Skip to content

Commit

Permalink
update master branch (#2)
Browse files Browse the repository at this point in the history
* More complex models for velocity estimation are added:
 - polynomial functions with customized polynomial degree
 - periodic function(s) with customized period(s) for seasonal fitting
 - step function(s) with customized date(s) for coseismic fitting
These models are not callable using smallbaseline.py
But they have been made available using timeseries2velocity.py with arguments
Check timeseries2velocity.py -h for more information under "velocity models of interest"

* generic STD estimation for time func params

+ stack.timeseries():
   - rename get_design_matrix4average_velocity() to get_design_matrix4time_func() for more meaningful name, and inside sub-functions.
   - set model as dict type to support future time functions.
   - update usage of get_design_matrix4*() in objects.gps, simulation.simulation.

+ timeseris2velocity:
   - set model as dict type, consistent with objects.stack()
   - add standard error estimation for all time function parameters, based on error propagation law, as a generic extension to the linear velocity STD from equation (10) in Fattahi and Amelung (2015, JGR). This is currently support via non-bootstrap approach only.

* add writefil.write_isce_xml/file()

+ add utils.writefile.write_isce_xml() to support xml/vrt file writing

+ add writefile.write_isce_file() to write binary data file and metadata files in isce format.

+ ts2vel: add Yuan-Kai to the author list

* rename file for the num of triplets with non-zero int ambiguity

from `numNonzeroIntClosure.h5` to `numTriNonzeroIntAmbiguity.h5`.

* add docs/FAQs page

* indexing readfile.read() call in view.py

Support fancy indexing / slicing in utils.readfile.read() with x/ystep, to merge the reading and multilooking process into one.

This significantly reduce the memory load of view.py for large 3D dataset, from 9 GB to less than 500 MB in the case of GalapagosSenDT128 dataset.

* isce_utils.py: bug fix for spacecraftname in metadata (insarlab#430)

* tropo_pyaps3: add --custom-height for testing purpose (insarlab#431)

+ view: auto update vlim after masking for multi-subplots for better display when the non-zero values are largely off from zero, which is the case of the absolute tropo delay in ERA5.h5 file.

+ tropo_pyaps3.py: add --custom-height for testing purpose

+ test_smallbaselineApp.py: take codacy suggestion

+ docs/README: one line for the badges

+ docs/dask.md: fix inproper display of dask performance figure on readthedocs

* utils: add med_abs_dev()

separate median_abs_deviation() from median_abs_deviation_threshold() to support 2D matrix calculation for MAD.

* isce_utils: update calc for CENTER_LINE_UTC and HEADING

+ use sensingMid for CENTER_LINE_TUC calculation, instead of the previous startUTC. Difference is in seconds, with no expected impact on tropo_pyaps3.py with GAM resolution of hours.

+ calc HEADING from orbit instead of reading los file to be more generic, precise and simple

+ use meta instead of metadata to shorter code

* save_hdfeos5.py and smallbaselineApp.py: add avgSpatialCoh.h5 to S1*.he5 file (insarlab#439)

+ smallbaselineApp: add avgSpatialCoh.h5 to the default geocode file list

+ save_hdfeos5: add `quality/avgSpatialCoh`
   - add `--asc / --avg-spatial-coh` for the average spatial coherence
   - rename `-t` to `--tc / --temp-coh` for the temporal coherence to avoid confusion with the newly added avgSpatialCoh.h5

+ update docs/hdfeos5.md for the updated HDF-EOS5 data structure

* bugfix for view.py --coastline option (insarlab#440)

* bugfix for view.py --coastline option

* bugfix in view.py --multilook-num (insarlab#441)

* bugfix in view.py --multilook-num, which causes the initiation shape mismatch due to the recently introduced multilooking via indexing.

* view: set coastline default to None instead of 'no'

* Improve memory efficiency in the steps of calculating ramp, deramp, residual_RMS (insarlab#436)

* To enhance the memory efficiency in calculating ramp, deramp, and time-series RMS, by reading/estimating(/writing) one date at a time.

* Fixed a syntax error in readfile.py at line 799
Before: is not "none"
Now:    != "none"

* writefile.py: add `print_msg=True` to `layout_hdf5()` and `write_hdf5_block()`.

* isce_utils.py: fix typo burst to frame

* timeseries: add precise UTC time info to `self.times`

from CENTER_LINE_UTC

* view: tie --coastline with --lalo-label

If --coastline option is turned ON, enable --lalo-label as well, to show the lat/lon label; otherwise, it's blank by default by cartopy.

* add writefile.layout_hdf5(ref_file)

utils.writefile.layout_hdf5():
+ support ref_file as an alternative of ds_name_dict and metadata, for easy use

+ support auxliary data writing, for less call of write_hdf5_block()

+ use snake_case to replace the camelCase for consistency

Simplify the code using the new `writefile.layout_hdf5()` in the following functions:
+ utils.utils1.run_deramp()
+ ifgram_inversion.ifgram_inversion()
+ prep_fringe.prepare_timeseries()

pyaps: force era5 correctio in the 1st pass

* skip pixels with zero avgSpatialCoh/tempCoh to speedup

+ ifg_inv:
   - skip pixel with zero avgSpatialCoh, which is caused by data missing in the processing (usually if not all)
   - msg bugfix in calc_temp_coh() when the patch is very small
   - change default --cluster value to 'no', to be consistent with default template value.

+ unwrap_phase_phase_closure.calc_T_int(): mask out pixels with zero average spatial coherence, which is caused by missing data

+ dem_err: skip pixel with zero temporal coherence, due to water body, shadow, zero average spatial coherence, etc.

+ decorrelation: update msg

+ multilook: ignore runtime warnning msg

* ifgramStack_coherence_spatialCoh.txt to coherenceSpatialAvg.txt

+ rename ifgramStack_coherence_spatialCoh.txt to coherenceSpatialAvg.txt for shorter filename

+ use camelCase for output figure name in plot_network.py

+ msg update for load_data.py when checking inconsistent file size

* improved auto skip for inv_net and quick_overview

+ ifg_inv: run if TS is partly written only, due to the incomplete running of ifgram_inversion.

+ unwrap_error_phase_closure: add checking of modification time, and ref_y/x metadata for numTriNonzeroIntAmbiguity calculation

* readfile.read_hdf5_file(): speedup x/ystep indexing

+ utils.readfile.read_hdf5_file(): speedup x/ystep indexing by using numpy indexing instead of h5py indexing. The latter has significant slow down for large index. For a time-series with 20*2400*5500, it's 1 sec vs 44 secs.

+ .gitignore: add *.dSYM*

+ view:
   - update ref_y/x for multilooking
   - support date1_date2 for TS file to be able to reconstruct interferogram from time-series on the fly

* decor: use coh_step to replace epsilon

decor: use coh_step to replace epsilon to avoid dividing a number close to zero

* decor.cal_coh4phase_pdf_bias: numerically robust

+ use poly_deg of 10 for L < 10; and 40 for 10 <= L < 20, for more suitable redundancy

+ add poly_rcond for easy modification

+ limit output coh_cal to [0, 1]

+ attach coh_sim/est = 1 to the end of the bias data for a more robust calibration for values close to 1

* tropo_pyaps3: check date_list in existing tropo h5 file

* new ASCII logo

+ version: add a new ASCII logo to replace the previous one

+ docs/README: add mintpy prounciation in International Phonetic Alphabet (IPA) style.

+ docs/resources/colormaps/README: update link to sci colormap png

+ delete the unused mintpy/.travis.yml

+ delete the impractical docs/examples/dev, which is meant for early stage develop only

+ .gitignore: add mintpy/objects/solid_f.*.so*

* ifg_inv: bugfix for custom output filenames

ifg_inv: bugfix for custom output filenames and update example usage for offset

sbApp: print plot_netowrk cmd always

* time function fitting: k! in the denominator of the polynomial function (insarlab#450)

* time function fitting: k! in the denominator of the polynomial function

* dem_err: rm duplicated design matrix func for time func

+ remove the duplicated design matrix func for time func and use the one from stack.timeseries objects instead, because it's more generic and more clear coding.

+ use G for the design matrix symbol instead of A, to be consistent with the convension in Yunjun et al. (2019)

* bugfix on the prep_fringe.py for the updated isce_utils.extract_geometry_metadata() args (insarlab#455)

There is a bug on line 140 of the prep_fringe.py causing the code to stop working during the extraction of the metadata on the reference xml files.

* asc_desc2horz_vert: add --dset option

add --dset option to be able to customize the dataset to read/use to asc/desc 2 horz/vert conversion, for input file with more than one dataset.

* net.select_pairs_*(): use date_format instead of date12_format for simplicity

* move resources from docs to mintpy

+ move resources folder from docs to mintpy as they are used in the code, to make mintpy folder independent from docs folder

+ rename docs/resources/colormaps/README.md to docs/api/colormaps.md and update its usage in mkdocs.yml

+ update the location of resources files in save_kmz_timeseries.py, objects/colors.py

+ remove unused test/unwrap_error_bridging in .gitignore

* prep_aria: support subset / update mode / compression

+ writefile.layout_hdf(): add compression argument

+ smallbaselineApp: do not forcely pass --update to prep_aria.py

+ prep_aria:
   - add read_subset_box() to support loading part of ARIA products in 'load_data' step using `mintpy.subset.yx/lalo` options
   - add --compression option
   - more robust run_or_skip() considering the corrupted h5 files

* prep_aria: bugfix when no template file input

+ bugfix when there is no template file input
+ update example usage

* add pip to docs/conda.txt

+ add pip to conda.txt since it's used in the installation of pykml
+ remove ${CONDA_PREFIX} while calling conda since it's now removed in the custom environment from the latest version of conda distribution

* prep_gamma: support *-* folder style

while searching for *.par files to grab LAT/LONO_REF1/2/3/4 metadata

* prep_aria: update metadata after subseting

* view: break date1_date2 into date1\ndate2

for number subplots in (20, 50] for clearer display

* ifg_inv: NaN as no-data value for offset/phase

+ ifgram_inversion;
   - use np.nan value as the no-data value for observations (phase or offset) to replace the previous version of using both (zero and nan), by:
      - setting masked out pixel values as np.nan instead of 0 and
      - setting zero phase (not offset) value to nan.

   - The above changes results in the following improvements:
      - simplified checking of masks and pixels with no-data values in some (not all) interferograms.
      - remove `skip_zero_value` from estimate_timeseries() due to the above change

   - skip pixels with zero avgSpatialSnr values for offset

   - more comments

+ objects/stack.ifgramStack.temporal_average():

   - split the big 3D matrix in row direction instead of ifgram/time dimension.

   - the above change allows to use nanmean to replace manual mean calculation, to better handle the nan value, for improved avgSpatialSnr.h5 calculation.

+ adjust printout msg in writefile.layout_hdf5() and view.py

* support for alosStack in ISCE (insarlab#464)

Adds support for ALOS-2 (and ALOS-4 in the future) stack processed with contrib/stack/alosStack in ISCE-2. The data loading part is similar to the product from topsStack and stripmapStack, whose sub-type is identified automatically based on the input metadata file via mintpy.load.metaFile.

Detailed changes are as follows:

+ utils/isce_utils.py:
   - modify get_processor() to detect alosStack based on f1_*/*.frame.xml file.
   - add the following new functions to extract metadata for alosStack:
      - extract_alosStack_metadata()
      - alos2_acquisition_mode()
      - extract_image_size_alosStack()
      - load_track()
      - read_alosStack_baseline()

+ prep_isce.py:
   - adjust prepare_stack() and prepare_geometry() for alosStack.
   - cmd_line_parse(): 1) use argparse required argument to replace the checking in cmd_line_parse(); 2) translate wildcard in input metaFile

+ add docs/examples/input_files/NCalAlos2Scan.txt as the example template file for loading alosStack products

+ objects/stackDict.geometry.write2hdf5(): convert waterBody to waterMask while loading data

+ utils/readfile.read_binary_file(): use band2 by default for *.cor file with 2 bands to support reading complex coherence files generated by isce-2.

* dir structure of ISCE/alosStack (insarlab#465)

* docs/dir_structure: directory structure of ISCE/alosStack added

* docs/examples/input_files: Rename template file of ISCE/alosStack

* ifg_inv for offset: consider A/RLOOKS for az/rg_pixel_size

+ ifgram_inversion: update az/rg_pixel_size to the resolution before multilooking for meaningful value

+ add utils.utils0.vtec2range_delay() to convert the zenith TEC to the predicted range delay in SAR, based on Chen and Zebker (2012)

+ add utils.isce_utils.get_IPF() for Sentinel-1 data

+ view: show lat/lon in the status bar for file in radar coord, if the geometryRadar.h5 file is in the default location.

+ tsview: make REF_Y/X optional to support off time-series

+ update HDF-EOS5 usage in save_hdfeos5.py and docs/hdfeos5.md

+ remove ECMWF from smallbaselineApp.cfg since it's not been updated anymore and is replaced by ERA-5; and update comments in tropo_pyaps(3).py

* ifg_recon: bugfix

+ stack.ifgramStack.get_design_matrix4timeseries(): add refDate='no' to disable the column redunction due to temporal referencing.

+ ifgram_reconstruction.py:
   - fix the bug of design matrix due to the reference date
   - update example usage

* fix bool array index bug for h5 dataset

+ readfile & stack: fix a bug of fancy indexing with numpy boolean array in h5py dataset (https://docs.h5py.org/en/stable/high/dataset.html#fancy-indexing), which is used in utils/readfile.py and objects/stack.py. This seems to be a bug in h5py side because it was working before.

+ prep_aria: fix a bug in gdal*.ReadAsArray() from numpy array int64 type to native python int16 type

* bugfix for loading SNAP DEM with subset enabled

+ load_data: fix a bug of subsetting geometry file in geo-coord if input dataset is in geo-coord as well.

+ stackDict: acknowledge the -32768 no data for DEM

* example dataset & template file for SNAP

+ add an example input dataset processed with SNAP provided by Andre Theron
+ add mintpy/data/input_files/WCapSenAT29.txt
+ add SNAP dataset to test_smallbaselineApp.py and test/configs/WCapSenAT29.txt

+ view: do not update disp_min/max if --wrap is enabled for multiple subplots, to be consistent with the one for single subplot.

* add setup.py for pip install and pypi distribution

+ add pyproject.toml to use setuptools and wheel for build system

+ add MANIFEST.in

+ add setup.py with the following updates:
   - auto grabbed version, (long_)description
   - dependencies info
   - data files

+ move /sh to /mintpy/sh for easy registration with setup.py

+ move /docs/examples/input_files into /mintpy/data/input_files

+ move /mintpy/resources into /mintpy/data

+ version: add release_description to replace the previous description; and add new description.

+ bump python min version in doc/setup from 3.5 to 3.6 to be consistent with the docs/conda.txt

* example dataset for ARIA on San Francisco

+ update mintpy/data/input_files/SanFranSenDT42.txt: enable unwrapping error correction with bridging and disable network modification via interferogram index.

+ docs/demo_dataset: add ARIA section with zenodo link using the San Francisco dataset prepared by Heresh Fattahi using ARIA-tools

+ view.prep_slice(): update example comments regarding the cartopy projection

* aria support in load_data with autoPath

+ load_data:
   - move the call of prep_aria.main() from smallbaselineApp.py into load_data.py, so that prep_aria could appears to be the same as other prep_*.py in the user side, a.k.a. "load_data.py -t template file" works for ARIA as well.
   - support 'mintpy.load.autoPath' for aria
   - ignore failed prep_aria and continue load_data, becuase 1) the ARIA product could have been loaded into mintpy and deleted already and 2) the potential missing loaded HDF5 files will be reported laterward in smallbaselineApp.py, so it's okay to ignore it here.

+ utils1.update_template_file(): fix a bug when the option name contains the option value, i.e. mintpy.load.autoPath = auto.

+ add SanFranSenDT42.txt to the test_smallbaselinApp.py

+ mintpy/sh/plot_smallbaselineApp: plot ts_demErr.h5

* add setup.py for pip install and pypi distribution

+ add pyproject.toml to use setuptools and wheel for build system

+ add MANIFEST.in

+ add setup.py with the following updates:
   - auto grabbed version, (long_)description
   - dependencies info
   - data files

+ move /sh to /mintpy/sh for easy registration with setup.py

+ move /examples from /docs to /mintpy for easy management via pip/setup.py

+ version: add release_description to replace the previous description; and add new description.

* more comments on extract_alosStack_metadata()

+ more comments on isce_utils.extract_alosStack_metadata()
+ update docs/api/attribute.md

* run_isce_stack: support topsStack

+ add run_isce_stack.py to drive the isce-2 topsStack and stripmapStack processors. Compared with process_isce_stack_v1.py:
   - remove the job submission code since I can not test and use it
   - add topsStack
   - simplify the code
   - add dem prep based on dem(_gsi).py and auto opt values
   - replace cmd call of stackSentinel/StripMap.py with python call
   - add --text_cmd to sh file execution to avoid running load_tops/stripmap_stack

+ add fill_and_translate_template_auto_value() to replace mintpy.utils.check_template_auto_value()
   - support input template option value of auto, which will be translated into default value

+ add isce.numThread option for topsStack.topo.py

+ delete obsolete process_isce_stack*.py

Co-authored-by: Yuankailiu <[email protected]>
Co-authored-by: Zhang Yunjun <[email protected]>
Co-authored-by: Sara Mirzaee <[email protected]>
Co-authored-by: Zhang Yunjun <[email protected]>
Co-authored-by: ranneylxr <[email protected]>
Co-authored-by: Yuan-Kai Liu <[email protected]>
Co-authored-by: Sara Mirzaee <[email protected]>
Co-authored-by: Bryan Marfito <[email protected]>
Co-authored-by: CunrenLiang <[email protected]>
  • Loading branch information
10 people authored Nov 14, 2020
1 parent 60d7cc9 commit 3bb2c62
Show file tree
Hide file tree
Showing 112 changed files with 3,635 additions and 1,375 deletions.
7 changes: 4 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,15 @@
*.pyc
*.idea/*
*ipynb_checkpoints*
*.egg*
build/
dist/
docs/api_docs/
docs/examples/dev/*
docs/resources/*logo*
docs/resources/*.ai
docs/resources/*.pdf
docs/deps.svg
isce.log
mintpy/dev/*
mintpy/modis*
mintpy/objects/solid/solid_f.*.so*
site/
test/unwrap_error_bridging/
6 changes: 6 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Include the README
include docs/README.md

# Include the license file
include LICENSE

13 changes: 13 additions & 0 deletions docs/FAQs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
## Frequently Asked Questions

#### 1. What's the sign convention of the line-of-sight data?

For line-of-sight (LOS) phase in the unit of radians, i.e. 'unwrapPhase' dataset in `ifgramStack.h5` file, positive value represents motion away from the satellite. We assume the "date1_date2" format for the interferogram with "date1" being the earlier acquisition.

For LOS displacement in the unit of meters, i.e. 'timeseries' dataset in `timeseries.h5` file positive value represents motion toward the satellite (uplift for pure vertical motion).

#### 2. How to prepare the input for MintPy if I am using InSAR software rather than ISCE stack processors and ARIA-tools?




12 changes: 6 additions & 6 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
[![Language](https://img.shields.io/badge/python-3.5%2B-blue.svg)](https://www.python.org/)
[![Language](https://img.shields.io/badge/python-3.6%2B-blue.svg)](https://www.python.org/)
[![Docs Status](https://readthedocs.org/projects/mintpy/badge/?version=latest)](https://mintpy.readthedocs.io/?badge=latest)
[![CircleCI](https://img.shields.io/circleci/build/github/insarlab/MintPy.svg?color=green&logo=circleci)](https://circleci.com/gh/insarlab/MintPy)
[![Latest version](https://img.shields.io/badge/latest%20version-v1.2.3-yellowgreen.svg)](https://github.com/insarlab/MintPy/releases)
[![Version](https://img.shields.io/badge/version-v1.2.3-yellowgreen.svg)](https://github.com/insarlab/MintPy/releases)
[![License](https://img.shields.io/badge/license-GPLv3-yellow.svg)](https://github.com/insarlab/MintPy/blob/main/LICENSE)
[![Forum](https://img.shields.io/badge/forum-Google%20Group-orange.svg)](https://groups.google.com/forum/#!forum/mintpy)
[![Citation](https://img.shields.io/badge/doi-10.1016%2Fj.cageo.2019.104331-blue)](https://doi.org/10.1016/j.cageo.2019.104331)

## MintPy ##

The Miami INsar Time-series software in PYthon (MintPy) is an open-source package for Interferometric Synthetic Aperture Radar time series analysis. It reads the stack of interferograms (coregistered and unwrapped) in [ISCE](https://github.com/isce-framework/isce2), [ARIA](https://github.com/aria-tools/ARIA-tools), [FRInGE](https://github.com/isce-framework/fringe), [SNAP](http://step.esa.int/), [GAMMA](https://www.gamma-rs.ch/no_cache/software.html) or ROI_PAC format, and produces three dimensional (2D in space and 1D in time) ground surface displacement in line-of-sight direction. It includes a routine time series analysis (`smallbaselineApp.py`) and some independent toolbox.
The Miami INsar Time-series software in PYthon (MintPy as /mɪnt paɪ/) is an open-source package for Interferometric Synthetic Aperture Radar (InSAR) time series analysis. It reads the stack of interferograms (coregistered and unwrapped) in [ISCE](https://github.com/isce-framework/isce2), [ARIA](https://github.com/aria-tools/ARIA-tools), [FRInGE](https://github.com/isce-framework/fringe), [SNAP](http://step.esa.int/), [GAMMA](https://www.gamma-rs.ch/no_cache/software.html) or ROI_PAC format, and produces three dimensional (2D in space and 1D in time) ground surface displacement in line-of-sight direction. It includes a routine time series analysis (`smallbaselineApp.py`) and some independent toolbox.

This package was called PySAR before version 1.1.1. For version 1.1.2 and onward, we use MintPy instead.

Expand Down Expand Up @@ -45,7 +45,7 @@ Configuration parameters for each step are initiated with default values in a cu
wget https://zenodo.org/record/3952953/files/FernandinaSenDT128.tar.xz
tar -xvJf FernandinaSenDT128.tar.xz
cd FernandinaSenDT128/mintpy
smallbaselineApp.py ${MINTPY_HOME}/docs/examples/input_files/FernandinaSenDT128.txt
smallbaselineApp.py ${MINTPY_HOME}/mintpy/data/input_files/FernandinaSenDT128.txt
```

<p align="left">
Expand All @@ -67,7 +67,7 @@ save_kmz_timeseries.py #generate Goodle Earth KMZ file in points for time-se

#### 2.2 Customized processing recipe ####

MintPy is a toolbox with individual utility scripts. Simply run the script with `-h / --help` to see its usage, you could build your own customized processing recipe! [Here](../sh/compare_velocity_with_diff_tropo.sh) is an example to compare the velocities estimated from displacement time-series with different tropospheric delay corrections.
MintPy is a toolbox with individual utility scripts. Simply run the script with `-h / --help` to see its usage, you could build your own customized processing recipe! [Here](../mintpy/sh/compare_velocity_with_diff_tropo.sh) is an example to compare the velocities estimated from displacement time-series with different tropospheric delay corrections.

#### 2.3 Build on top of `mintpy` module ####

Expand All @@ -84,7 +84,7 @@ Algorithms implemented in the software are described in details at [Yunjun et al

+ [Quick start with example datasets](./demo_dataset.md)
+ [Example data directory](./dir_structure.md)
+ [Example template files for InSAR processors](./examples/input_files/README.md)
+ [Example template files for InSAR processors](./../mintpy/data/input_files/README.md)
+ [Tutorials in Jupyter Notebook](https://github.com/insarlab/MintPy-tutorial)

### 4. Contact us ###
Expand Down
21 changes: 12 additions & 9 deletions docs/api/attributes.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
MintPy mainly uses attribute names from [ROI_PAC](http://www.geo.cornell.edu/eas/PeoplePlaces/Faculty/matt/pub/winsar/InSAR_textbook_for_web_2014.pdf), with some additional attributes generated by MintPy itself.
MintPy mainly uses attribute names from [ROI_PAC](http://www.geo.cornell.edu/eas/PeoplePlaces/Faculty/matt/pub/winsar/InSAR_textbook_for_web_2014.pdf), with some additional self-generated attributes.

### Required attributes ###

If using ROI_PAC as InSAR processor, both **baseline parameter RSC** file (i.e. *100416-100901_baseline.rsc*) and **basic metadata file** (i.e. *filt_100416-100901-sim_HDR_4rlks_c10.unw.rsc*) will be imported into MintPy. The following attributes for each interferogram are required in order to run MintPy:
If using ROI_PAC as the InSAR processor, both **baseline parameter RSC** file (i.e. *100416-100901_baseline.rsc*) and **basic metadata file** (i.e. *filt_100416-100901-sim_HDR_4rlks_c10.unw.rsc*) will be imported into MintPy. The following attributes for each interferogram are required in order to run MintPy:

+ FILE_LENGTH = number of rows.
+ WIDTH = number of columns.
Expand All @@ -15,30 +15,33 @@ If using ROI_PAC as InSAR processor, both **baseline parameter RSC** file (i.e.
+ CENTER_LINE_UTC = Time at middle of interferogram in seconds, used in tropo correction using PyAPS.
+ HEIGHT = Height of satellite in meters, used in dem_error, incidence_angle, convert2mat.
+ STARTING_RANGE = Distance from satellite to first ground pixel in meters, used in incidence_angle calculation
+ DATE12 = (date1)-(date2), reference - secondary date of interferogram in 6 digit number.
+ PLATFORM = satellite/sensor name, used in Local Oscillator Drift correction for Envisat.
+ ORBIT_DIRECTION = ascending, or descending.
+ ALOOKS/RLOOKS = multilook number in azimuth/range direction, used in weighted network inversion.

The following attributes vary for each interferogram:

+ DATE12 = (date1)-(date2), reference - secondary date of interferogram in 6 digit number.
+ P_BASELINE_TOP_HDR = Perpendicular baseline at top (first line) of interferogram in meters.
+ P_BASELINE_BOTTOM_HDR = Perpendicular baseline at bottom (last line) of interferogram in meters.
+ ALOOKS/RLOOKS = multilook number in azimuth/range direction, used in weighted network inversion.


### Optional attributes ###

+ ANTENNA_SIDE = -1 for right looking radar, used in save_unavco
+ AZIMUTH_PIXEL_SIZE = Azimuth pixel size at orbital altitude (multiply by Re/(Re+h) for ground size (m), where Re is the local earth radius), used in baseline_error/trop and multilook.
+ HEADING = Spacecraft heading at peg point (degree), used in asc_desc, los2enu
+ PRF = Pulse repetition frequency (Hz), used in save_unavco

### Attributes generated by MintPy automatically ###
### Self-generated attributes ###

+ FILE_TYPE = file type.
- for HDF5 files, it's the root level dataset name, such as `velocity, timeseries, ifgramStack, temporalCoherence, mask, HDFEOS, dem, coherence, etc.`;`
- for binary files, it's the file extension name, such as `.unw, .cor, .int, .amp, .mli, .dem, .hgt, .unw.conncomp, .UTM_TO_RDC, .trans, etc.`, except for ISCE geometry files, which is the file base name such as `hgt, lat, lon, los, shadowMask, incLocal`.
+ FILE_PATH = absolute file path
+ LENGTH = row number, equivalent to FILE_LENGTH
+ PROCESSOR = processing software, i.e. isce, roipac, gamma
+ PROCESSOR = processing software, i.e. isce, aria, snap, gamma, roipac etc.
+ DATA_TYPE = data type, i.e. float32, int16, etc., for isce product read using GDAL
+ UNIT = data unit, i.e. m, m/yr, radian, and 1 for file without unit, such as coherence
+ UNIT = data unit, i.e. m, m/yr, radian, and 1 for file without unit, such as coherence [[source]](https://github.com/insarlab/MintPy/blob/main/mintpy/objects/stack.py#L75)
+ REF_DATE = reference date
+ REF_X/Y/LAT/LON = column/row/latitude/longitude of reference point
+ SUBSET_XMIN/XMAX/YMIN/YMAX = start/end column/row number of subset in the original coverage
Expand All @@ -47,4 +50,4 @@ If using ROI_PAC as InSAR processor, both **baseline parameter RSC** file (i.e.

### Reference ###

+ Pritchard et al., (2014), Open-source software for geodetic imaging: ROI_PAC for InSAR and pixel tracking, pp 44-48. [PDF](http://www.geo.cornell.edu/eas/PeoplePlaces/Faculty/matt/pub/winsar/InSAR_textbook_for_web_2014.pdf)
+ Pritchard et al., (2014), Open-source software for geodetic imaging: ROI_PAC for InSAR and pixel tracking, pp 44-48. [PDF](http://www.geo.cornell.edu/eas/PeoplePlaces/Faculty/matt/pub/winsar/InSAR_textbook_for_web_2014.pdf)
4 changes: 2 additions & 2 deletions docs/resources/colormaps/README.md → docs/api/colormaps.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ MintPy support the following colormaps:

+ [Matplotlib colormaps](https://matplotlib.org/3.1.0/tutorials/colors/colormaps.html)
+ Custom colormaps: `cmy` and `dismph`
+ Custom colormaps in **.cpt** (color palette tables) format. To add your own colormap, drop the corresponding .cpt file in `$MINTPY/docs/resources/colormaps`.
+ Custom colormaps in **.cpt** (color palette tables) format. To add your own colormap, drop the corresponding .cpt file in `$MINTPY/mintpy/data/colormaps`.

We recommend to use cyclic colormap `cmy` for wrapped phase/displacement measurement.

Expand Down Expand Up @@ -63,5 +63,5 @@ The following colormaps is included by default:
+ More at [Scientific Color-Maps](http://www.fabiocrameri.ch/colourmaps.php) ([Crameri, 2018](https://doi.org/10.5194/gmd-11-2541-2018))

<p align="left">
<img width="600" src="http://www.fabiocrameri.ch/resources/ScientificColourMaps_FabioCrameriCompact.png">
<img src="http://www.fabiocrameri.ch/resources/ScientificColourMaps_FabioCrameri.png">
</p>
2 changes: 1 addition & 1 deletion docs/api/module_hierarchy.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Hierarchy of sub-modules within MintPy. Level N modules depends on level N-1, N-
/utils
plot (objects/{stack, coord, colors}, utils/{ptime, utils0, utils1, readfile, network})
utils (objects/{stack, coord}, utils/{ptime, utils0, utils1, readfile})
isce_utils (utils/{readfile, writefile, utils1})
isce_utils (utils/{ptime, readfile, writefile, utils1})
------------------ level 6 --------------------
/objects
insar_vs_gps (objects/{stack, giant}, utils/{readfile, gps, plot, utils})
Expand Down
1 change: 1 addition & 0 deletions docs/conda.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ matplotlib
netcdf4
numpy
openmp
pip
pygrib
pyhdf
pykdtree
Expand Down
2 changes: 1 addition & 1 deletion docs/dask.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ A typical run time without local cluster is 30 secs and with 8 workers 11.4 secs

To show the run time improvement, we test three datasets (South Isabela, Fernandina, and Kilauea) with different number of cores and same amount of allocated memory (4 GB) on a compute node in the [Stampede2 cluster's skx-normal queue](https://portal.tacc.utexas.edu/user-guides/stampede2#overview-skxcomputenodes). Results are as below:

![Dask LocalCluster Performance](https://github.com/insarlab/MintPy-tutorial/blob/main/docs/dask_local_cluster_performance.png)
![Dask LocalCluster Performance](https://yunjunzhang.files.wordpress.com/2020/08/dask_local_cluster_performance.png)

#### 1.5 Known problems ####

Expand Down
40 changes: 37 additions & 3 deletions docs/demo_dataset.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Size: ~750 MB
wget https://zenodo.org/record/3952953/files/FernandinaSenDT128.tar.xz
tar -xvJf FernandinaSenDT128.tar.xz
cd FernandinaSenDT128/mintpy
smallbaselineApp.py ${MINTPY_HOME}/docs/examples/input_files/FernandinaSenDT128.txt
smallbaselineApp.py ${MINTPY_HOME}/mintpy/data/input_files/FernandinaSenDT128.txt
```

<p align="left">
Expand All @@ -21,6 +21,27 @@ Relevant literature:

+ Yunjun, Z., H. Fattahi, and F. Amelung (2019), Small baseline InSAR time series analysis: Unwrapping error correction and noise reduction, _Computers & Geosciences, 133,_ 104331, doi:10.1016/j.cageo.2019.104331.

#### Sentinel-1 on San Francisco Bay with ARIA ####

Area: San Francisco Bay, California, USA
Data: Sentinel-1 A/B descending track 42 during May 2015 - March 2020 (114 acquisitoins; [Zenodo](https://zenodo.org/record/4265413))
Size: ~2.7 GB

```bash
wget https://zenodo.org/record/4265413/files/SanFranSenDT42.tar.xz
tar -xvJf SanFranSenDT42.tar.xz
cd SanFranSenDT42/mintpy
smallbaselineApp.py ${MINTPY_HOME}/mintpy/data/input_files/SanFranSenDT42.txt
```

<p align="left">
<img width="650" src="https://yunjunzhang.files.wordpress.com/2020/11/sanfransendt42_transect.jpg">
</p>

Relevant literature:

+ Chaussard, E., R. Bürgmann, H. Fattahi, R. M. Nadeau, T. Taira, C. W. Johnson, and I. Johanson (2015), Potential for larger earthquakes in the East San Francisco Bay Area due to the direct connection between the Hayward and Calaveras Faults, Geophysical Research Letters, 42(8), 2734-2741, doi:10.1002/2015GL063575.

#### Envisat of the 2008 Wells earthquake with Gamma ####

Area: Wells, Nevada, USA
Expand All @@ -31,7 +52,7 @@ Size: ~280 MB
wget https://zenodo.org/record/3952950/files/WellsEnvD2T399.tar.xz
tar -xvJf WellsEnvD2T399.tar.xz
cd WellsEnvD2T399/mintpy
smallbaselineApp.py ${MINTPY_HOME}/docs/examples/input_files/WellsEnvD2T399.txt
smallbaselineApp.py ${MINTPY_HOME}/mintpy/data/input_files/WellsEnvD2T399.txt
```

<p align="left">
Expand All @@ -42,6 +63,19 @@ Relevant literature:

+ Nealy, J. L., H. M. Benz, G. P. Hayes, E. A. Bergman, and W. D. Barnhart (2017), The 2008 Wells, Nevada, Earthquake Sequence: Source Constraints Using Calibrated Multiple‐Event Relocation and InSARThe 2008 Wells, Nevada, Earthquake Sequence: Source Constraints Using Calibrated Multiple‐Event Relocation, _Bulletin of the Seismological Society of America_, 107(3), 1107-1117, doi:10.1785/0120160298.

#### Sentinel-1 on Western Cape, South Africa with SNAP ####

Area: West coast of Western Cape province, South Africa
Data: Sentinel-1 ascending track 29 during March - June 2019 (10 acquisitions; [Zenodo](https://zenodo.org/record/4127335))
Size: ~560 MB

```bash
wget https://zenodo.org/record/4127335/files/WCapeSenAT29.tar.xz
tar -xvJf WCapeSenAT29.tar.xz
cd WCapeSenAT29
smallbaselineApp.py ${MINTPY_HOME}/mintpy/data/input_files/WCapeSenAT29.txt
```

#### ALOS-1 on Kuju with ROI_PAC ####

Area: Kuju volcano at Kyushu island, SW Japan
Expand All @@ -52,7 +86,7 @@ Size: ~240 MB
wget https://zenodo.org/record/3952917/files/KujuAlosAT422F650.tar.xz
tar -xvJf KujuAlosAT422F650.tar.xz
cd KujuAlosAT422F650/mintpy
smallbaselineApp.py ${MINTPY_HOME}/docs/examples/input_files/KujuAlosAT422F650.txt
smallbaselineApp.py ${MINTPY_HOME}/mintpy/data/input_files/KujuAlosAT422F650.txt
```

<p align="left">
Expand Down
Loading

0 comments on commit 3bb2c62

Please sign in to comment.