Merge pull request #32 from TUDelftGeodesy/sync_group_to_main

Sync group to main

.github/workflows/build.yml

@@ -7,14 +7,7 @@ on:
   push:
     branches: [ "main" ]
   pull_request:
-    branches:
-      - "main"
-      - "dev"
-    types:
-      - opened
-      - synchronize
-      - reopened
-      - ready_for_review
+    branches: [ "main" ]
 
 jobs:
   build:
@@ -27,9 +20,9 @@ jobs:
         python-version: ["3.10", "3.11", "3.12"]
 
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
     - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v3
+      uses: actions/setup-python@v5
       with:
         python-version: ${{ matrix.python-version }}
     - name: Install dependencies

.github/workflows/build_group.yml

@@ -0,0 +1,35 @@
+# Build and test for the private repository for TUDelft Radar group
+# Only test on PR for dev branch, not triggered for draft PR
+
+name: Build and pytest
+
+on:
+  pull_request:
+    types: [opened, synchronize, reopened, ready_for_review]
+    branches: [ "dev" ]
+
+jobs:
+  build:
+
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: ['ubuntu-latest']
+        python-version: ["3.10", "3.11", "3.12"]
+
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v5
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        python -m pip install build
+        python -m pip install ".[dev,docs]"
+    - name: Build the package
+      run: python -m build
+    - name: Test with pytest
+      run: python -m pytest

.github/workflows/doc_deploy.yml

@@ -0,0 +1,28 @@
+# Deploy documentation when new release created
+
+name: Deploy docs
+
+on:
+  release:
+    types:
+      - published
+
+
+jobs:
+  build:
+    name: Deploy docs
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout main
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - name: Set up Python 3.12
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - name: Install dependencies
+        run: |
+          python -m pip install .[docs] 
+      - name: Deploy docs
+        run: mkdocs gh-deploy --force

.github/workflows/ruff.yml

@@ -0,0 +1,15 @@
+# Lint and format check
+# For dev branch on private repo, only test on PR, not triggered for draft PR
+
+name: Ruff
+on:
+    push:
+      branches: [ "main" ]
+    pull_request:
+      branches: [ "main", "dev" ]
+jobs:
+  ruff:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/ruff-action@v3

README.md

@@ -1,42 +1,11 @@
-# DePSI 
-This is the WIP repository for the Python version of DePSI *(), a Python package for inteferometric SAR processing. The software is inspired by the MATLAB version DePSI. In this repository, we implement classic DePSI algorithms and new InSAR developments in Python.
 
-## Installation for development
+# DePSI
 
-It is assumed that you have `mamba` installed. If not, you can find the installation instructions [here](https://mamba.readthedocs.io/en/latest/installation/mamba-installation.html). Other package managers like `conda` or `venv` can be used as well.
+DePSI (van Leijen, 2014) is an open source software for processing Persistent Scatterer Interferometry (PS-InSAR) data, originally implemented in MATLAB. From 2024, TUDelft and Netherlands eScience Center are collaborating to develop a Python version of DePSI, with recent advances in PS-InSAR. 
 
-Clone this repository 
+## Developer Guide
 
-Then`cd` into it:
-
-```bash
-cd DePSI
-```
-
-Create a new conda environment (here we give an example name `depsi-dev`) with `mamba`.:
-
-```bash
-mamba create -c conda-forge -n depsi-dev python=3.12
-```
-
-Here we use Python 3.12 since we aim to support python 3.10 and above.
-
-Activate the environment:
-
-```bash
-mamba activate depsi-dev
-```
-
-Install this package in development mode:
-
-```bash
-pip install -e ".[dev,docs]"
-```
-
-In the end, install the pre-commit hooks:
-```bash
-pre-commit install
-```
+Please refer to the [Developer Guide](docs/dev_guide.md) for installation instructions, testing, and other development-related information.
 
 ## Useful reading material
 
@@ -49,3 +18,7 @@ pre-commit install
 Copyright (c) 2023 - 2025, Netherlands eScience Center & Delft University of Technology
 
 Apache Software License 2.0
+
+## References
+
+[1] Van Leijen, Frederik Johannes. "Persistent scatterer interferometry based on geodetic estimation theory." (2014).

docs/api_reference.md

@@ -0,0 +1,24 @@
+## **Classification methods**:
+
+::: depsi.classification
+    options:
+      show_root_heading: false
+      show_source: true
+      heading_level: 3
+
+
+## **IO methods**:
+
+::: depsi.io
+    options:
+      show_root_heading: false
+      show_source: true
+      heading_level: 3
+
+## **slc methods**:
+
+::: depsi.slc
+    options:
+      show_root_heading: false
+      show_source: true
+      heading_level: 3

docs/dev_guide.md

@@ -0,0 +1,79 @@
+# Developer Guide
+
+## Installation guide
+
+The Python implementation of DePSI is under development. At present you can only install it from the GitHub repository.
+
+It is assumed that you have `mamba` installed. If not, you can find the installation instructions [here](https://mamba.readthedocs.io/en/latest/installation/mamba-installation.html). Other package managers like `conda` or `venv` can be used as well.
+
+Before you start, make sure you have access to the correct DePSI repository. You can contribute to the pubilic DePSI repository by forking it to your own GitHub account. If you are a member of the TUDelftGeodesy organization, you can also contribute to the group DePSI repository.
+
+Clone this GitHub repository, then `cd` into the cloned repository.
+
+```bash
+cd DePSI
+```
+
+Create a new conda environment (here we give an example name `depsi-dev`) with `mamba`.:
+
+```bash
+mamba create -c conda-forge -n depsi-dev python=3.12
+```
+
+Here we use Python 3.12 since we aim to support python 3.10 and above.
+
+Activate the environment:
+
+```bash
+mamba activate depsi-dev
+```
+
+Install this package in development mode, with extra dependencies for development and documentation:
+
+```bash
+pip install -e ".[dev,docs]"
+```
+
+In the end, install the pre-commit hooks, which will run the checks before each commit:
+```bash
+pre-commit install
+```
+
+## Linting and formatting
+
+We use `ruff` for linting and formatting. If the pre-commit hooks are installed, the checks will be run automatically before each commit.
+
+To manually run the checks, use the following command in the root directory of the repository:
+
+```bash
+ruff check .
+```
+
+## Testing
+
+We use `pytest` for testing. All tests are located in the `tests` directory.
+
+To run the tests, use the following command in the root directory of the repository:
+
+```bash
+pytest tests
+```
+
+The [GitHub Actions](https://github.com/TUDelftGeodesy/DePSI/blob/main/.github/workflows/build.yml) will run the tests automatically for each push and pull-request
+on the `main` branch.
+
+## Documentation
+
+We use `mkdocs` for documentation. 
+
+To check the documentation at local, use the following command in the root directory of the repository:
+
+```bash
+mkdocs serve
+```
+
+This will build and render the documentation at a local server. Follow the link provided in the terminal to view the documentation in the browser.
+
+## Parallelization
+
+We use `dask` in many functions for delayed computation and parallelization. Since DePSI operates with Xarray, in most cases, we us Xarray's interface with Dask Arrays, such as `xarray.apply_gufunc` or `xarray.map_blocks` to perform parallel computation. Please refer to the [Xarray Tutorial of Parallelizing Custom Functions](https://tutorial.xarray.dev/advanced/parallel-intro.html) as the best practices for implementing parallelization in DePSI.  

docs/index.md

@@ -0,0 +1,16 @@
+# DePSI: Delft PS-InSAR processing package
+
+DePSI (van Leijen, 2014) is an open source software for processing Persistent Scatterer Interferometry (PS-InSAR) data, originally implemented in MATLAB. From 2024, TUDelft and Netherlands eScience Center are collaborating to develop a Python version of DePSI, with recent advances in PS-InSAR. 
+
+For the stable version of DePSI implemented in MATLAB, please refer to the [stable branch](https://github.com/TUDelftGeodesy/DePSI/tree/stable).
+
+## Installation
+
+The Python implementation of DePSI is under development. At present you can only
+install it from the GitHub repository:
+
+`pip install git+https://github.com/TUDelftGeodesy/DePSI.git@main`
+
+## References
+
+[1] Van Leijen, Frederik Johannes. "Persistent scatterer interferometry based on geodetic estimation theory." (2014).

docs/usages/slc.md

@@ -0,0 +1,23 @@
+# SLC related methods
+
+## Converting coregistered interferogram stack to SLC stack
+
+After reading the coregistered interferogram stack and the mother SLC with `sarxarray`, the SLC stack can be reconstructed using the [`ifg_to_slc`](https://tudelftgeodesy.github.io/DePSI/api_reference/#depsi.slc.ifg_to_slc) method. The method takes the mother SLC and the coregistered interferogram stack as input and returns the reconstructed SLC stack.
+
+```python
+import sarxarray
+from depsi.slc import ifg_to_slc
+from pathlib import Path
+
+f_mother_slc = 'path/to/mother_slc.raw' # Path to the mother SLC binary file
+f_ifgs = list(sorted(Path('dir_ifgs').rglob("2*/ifnteferogram.raw")))  # List of paths of coregistered interferograms
+shape = (10768, 40588) # Shape of the stack, (nrows, ncols)
+reading_chunks = (2000, 2000)  # Reading chunks for lazy loading, (nrows, ncols)
+
+# Lazy loading mother SLC and ifg stack
+mother = sarxarray.from_binary([f_mother_slc], shape, dtype=np.complex64, chunks=reading_chunks)
+ifgs = sarxarray.from_binary(f_ifgs, shape, dtype=np.complex64, chunks=reading_chunks)
+
+# Generate reconstructed SLCs
+slc_recon = ifg_to_slc(mother, ifgs)
+```

mkdocs.yml

@@ -0,0 +1,58 @@
+site_name: "DePSI: Delft PS-InSAR processing package"
+repo_url: https://github.com/TUDelftGeodesy/DePSI
+repo_name: DePSI
+
+nav:
+  - Getting started: 
+    - About DePSI: index.md
+  - Usage:
+    - SLC methods: usages/slc.md
+  - API Reference: api_reference.md
+  - Developer Guide: dev_guide.md
+
+
+theme:
+  name: material
+  palette:
+    # Palette toggle for light mode
+    - scheme: default
+      toggle:
+        icon: material/weather-sunny
+        name: Switch to dark mode
+      primary: blue
+      accent: white
+
+    # Palette toggle for dark mode
+    - scheme: slate
+      toggle:
+        icon: material/weather-night
+        name: Switch to light mode
+      primary: black
+      accent: pink
+  features:
+    - navigation.instant
+    - navigation.tabs
+    - navigation.tabs.sticky
+    - content.code.copy
+
+plugins:
+- mkdocs-jupyter:
+      include_source: True
+- search
+- mkdocstrings:
+    handlers:
+      python:
+        options:
+          docstring_style: numpy
+          docstring_options:
+            ignore_init_summary: no
+          merge_init_into_class: yes
+          show_submodules: no
+
+markdown_extensions:
+  - pymdownx.highlight:
+      anchor_linenums: true
+  - pymdownx.superfences
+
+extra:
+  generator: false

pyproject.toml

@@ -74,7 +74,8 @@ lint.select = [
   "PLE",  # Pylint error https://github.com/charliermarsh/ruff#error-ple
 ]
 lint.ignore = [
-"D100", "D101", "D104", "D105", "D106", "D107", "D203", "D213", "D413"
+"D100", "D101", "D104", "D105", "D106", "D107", "D203", "D213", "D413",
+"N803", "N806" # Allow capital letters in variables
 ] # docstring style
 
 line-length = 120