torch20 (#90)

* fix deleted method * format * format * deprecated scipy * add None check * update Dataset * add registers for collate * minor formatting * remove model tensor reshaping * rename args * network name changes * name changes * add keyword arg * move normalization method * update tests * convert to tensors * update README * format * format * format * format * format * use nn. * format * format * format * format * format * update loss * fix dataset test * move to modules * pin changes * stash * run train session * fix train data storage * add enum * change default transforms to None * change time series start/end get method * update script flow * add tower * add worker check * implement * start on predict data * fix inference steps * replace deprecated * fix predict window * cleanup * set defaults * comment * comment * changes * add foj loss * add foj loss * fix dropout * cleanup params * cleanup transfer * cleanup args * fix deep supervision * fix deep supervision * need to update the spatial gpkg * update dependencies * add 0 check * add multipolygon method * add pool_first arg * add pool_first arg * add pool_first arg * add pool_first arg * add pool_first arg * remove args * script changes * add optional StdConv * add pyogrio * replace gw polygon_to_array * add CLI arg for unknown labels * make read method * update progress bars * fix augmentations * fix augmentations * fix augmentation bdist * add args * add args * transfer updates * move methods * move scale * pass conv args * move methods * move data methods * add module import * modfy CLI defaults * update create methods * replace tqdm with rich * add batch file store * remove old code * support weak supervision * fix tests * test mask-rcnn * stash * change conv args * change conv args * test zoom and support score masking * modify tower unet * change arg name * cleanup unused and add order option * cleanup unused and add order option * change CLI * update CLI args * update test * change module names * stash * stash * add temp exception * reconfigure resconv * remove files * modify transfer model * fix mask scoring * use transfer name * separate spatial splits * add missing method * transfer loss * all_touched=True * add data methods * add edge check * add transfer arg * finetune * re-test params * get utm zone from lat/lon * change time scaler param * pass activation param * experimental final layers * cleanuup refinee * start/end * enums * enums * topology loss * cleanup * test attention * vit * residual in residual * attention * remove CLI option * predict start/end * temp test * libraries * reconfigure augmentations * random rng support * simplify augmenter * add data check for geometry collection * augment probability handling * force new spatial data for transfer model * formatting * test new loss * move modules * move modules * move modules * profiler * module imports * docs: update README (#75) * feat: transfer ltae v2 (#76) * update README * remove import * pass all_touched arg * add lon/lat to data batch * pass persistent workers arg * 👕 formatting * pass lon/lat coordinates * cleanup loss * rename network names * remove ViT * try/except import natten * 👕 formatting * pass all_touched from CLI * add all_touched CLI arg * fix: torch20 tests (#77) * 🔨 modify CI workflow * ⚡ remove install line * ⚡ update augmentation test names * 📝 update torch version in README * 🔨 modify CI workflow * 🔨 setup lib requirements * update augmentation tests * ⚡ update data tests * ⚡ update data tests * ⚡ better einops characters * 👕 formatting * ➕ raise minimum torchmetrics version * 🎨 formatting * fix: remove instances of torch_geometric (#78) * fix: test batch save step (#80) * docs: torch20 docs (#79) * 📝 update README * 📝 add sub-md * 📝 add install line * 📝 remove line * 📝 better comments * 📝 add README links * fix: torch20 cleanup (#81) * 🗑️ remove redundant method * 🎨 formatting * 🔥 py3 super() and remove unused methods * ✅ add tests for transformer * 🎨 formatting * ✅ add tests for tower unet * 🎨 formatting * 🔥 remove FOJ * 🔥 py3 super() and remove unused methods * 🎨 formatting * 🔥 py3 super() and remove unused methods * fix: add fiona dependency (#82) * fix: use cpu device (#83) * 🐛 add supported attention * 🐛 revert attention * 🐛✅ attention tests * 🙈 hide test for now * fix: torch20 train test (#84) * ✏️ change variable name * 🎨 formatting * ✅ test other weights * 🙈 block train CLI test * fix: torch20 network c (#85) * 🔥 remove network c module * 🎨 formatting * torch20 transfer (#86) * 🎨 formatting * 🔥 remove gain from dask storage * 🎨 formatting * 🎨 formatting * ⚡️ change CLI defaults * ✨ new loss enum * ⚡️ relative import * ✅ update tests * 🔒️ upgrade setuptools * fix: torch20 ex (#87) * ⚡️ sum weights * ⚡️ move sum * ✅ update loss tests * 🎨 formatting * ⚡️ use common method * 🎨 formatting * ➖ make kornia optional * ➕ increase dependency versions * ➖ cleanup imports * fix: ➖ remove boundary loss (#88) * ➖ remove topology loss * 📝 add comments for prior change * ⚡️ change default loss * 🎨 * ⚡️ add CLI data-pattern; remove args * 🎨 * 🔥 cleanup unused methods * 📝 new relative imports * ⚡️ change defaults * 🎨 * 🎨 * ⚡️ import loss * 🎨 * 🎨 * 🎨 * 🎨 * ✅ tests * 🎨 * 🎨 * 📝 change var name * 📝 change var name * change to class * ⚡️ predict callback return 3 bands * 🔥 remove unused import * 🎨 * ⚡️ relative loss imports * ⚡️ update enums * 🔥 remove arg in predict method * 🔥 remove temporal transformer * 🎨 * ⚡️ update main model network * ⚡️ ensure grad * 🎨 * 🎨 * 🎨 * 🔥 remove file * ✨ new relative import * 🔥 remove imports * 🎨 * 🔥 remove file * ⚡️ tests * ⚡️ scripts * 🔥 remove keyword arg * ⚡️ add keywrod arg * ⚡️ comment * 🎨 * 🎨 * 🎨 * 🎨 * 🔥 remove code * 🎨 * 🔥 remove imports * 🚑️ comment * 🎨 * 🔥 remove unused modules * 🎨 * 🎨 * 🐛 change default classes * 🔥 remove arg * 📝 * ⚡️ imports * 🐛 better check * ⚡️ change default values * ⚡️ rearrange convolutions * 🎨 * ⚡️ simplify pre * 🔥 remove unused losses * 🔥 remove unused module * 📝 static version --------- Co-authored-by: Ubuntu <[email protected]>
jgrss · Sep 23, 2024 · b51ca73 · b51ca73
1 parent e18e52e
commit b51ca73
Show file tree

Hide file tree

Showing 79 changed files with 9,102 additions and 12,576 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -11,17 +11,16 @@ on:
 jobs:
   Tests:
     runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.9", "3.10"]
     steps:
-      - uses: actions/checkout@v2
-      - uses: actions/setup-python@v2
+      - uses: actions/checkout@v3
+      - name: Setup Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v4
         with:
-          python-version: 3.8.12
-      - uses: syphar/restore-virtualenv@v1
-        id: cnetenv
-        with:
-          requirement_files: setup.cfg
-      - uses: syphar/restore-pip-download-cache@v1
-        if: steps.cnetenv.outputs.cache-hit != 'true'
+          python-version: ${{ matrix.python-version }}
+          cache: pip # caching pip dependencies based on changes to pyproject.toml
       - name: Install GDAL binaries
         run: |
           # Temporary? dpkg fix: https://askubuntu.com/questions/1276111/error-upgrading-grub-efi-amd64-signed-special-device-old-ssd-does-not-exist
@@ -39,19 +38,19 @@ jobs:
       - name: Install Python packages
         run: |
           # Install Python GDAL
-          pip install -U pip setuptools wheel
-          pip install -U Cython "numpy<=1.21.0"
+          pip install -U pip
+          pip install -U setuptools wheel
+          pip install -U numpy==1.24.4
+          pip install setuptools==57.5.0
           GDAL_VERSION=$(gdal-config --version | awk -F'[.]' '{print $1"."$2"."$3}')
           pip install GDAL==$GDAL_VERSION --no-binary=gdal
       - name: Install PyTorch
         run: |
           TORCH_CPU="https://download.pytorch.org/whl/cpu"
-          TORCH_VERSION="1.13.0"
-          pip install --upgrade --no-cache-dir setuptools>=0.59.5
-          pip install torch==${TORCH_VERSION} torchvision torchaudio --extra-index-url $TORCH_CPU
-          PYG_TORCH_CPU="https://data.pyg.org/whl/torch-${TORCH_VERSION}+cpu.html"
-          pip install torch-scatter torch-sparse torch-cluster torch-spline-conv torch-geometric -f $PYG_TORCH_CPU
-        if: steps.cnetenv.outputs.cache-hit != 'true'
+          TORCH_VERSION="2.2.2"
+          pip install -U --no-cache-dir setuptools>=65.5.1
+          pip install torch==${TORCH_VERSION} torchvision==0.17.2 torchaudio==${TORCH_VERSION} --extra-index-url $TORCH_CPU
+          pip install natten==0.17.1+torch220cpu -f https://shi-labs.com/natten/wheels
       - name: Install cultionet
         run: |
           pip install .
@@ -60,52 +59,3 @@ jobs:
           pip install pytest
           cd tests/
           python -m pytest
-
-#   Version:
-#     needs: Tests
-#     if: github.event_name == 'push' && github.ref == 'refs/heads/main' && !contains(github.event.head_commit.message, 'chore(release):')
-#     runs-on: ubuntu-latest
-#     concurrency: release
-#     steps:
-#       - uses: actions/checkout@v2
-#         with:
-#           fetch-depth: 0
-#           token: ${{ secrets.CULTIONET_TOKEN }}
-#       - uses: actions/setup-python@v2
-#         with:
-#           python-version: 3.8.12
-#       - name: Python Semantic Release
-#         run: |
-#           python -m pip install python-semantic-release
-#           # Add credentials
-#           git config user.name "github-actions"
-#           git config user.email "[email protected]"
-#           # Bump cultionet version
-#           semantic-release publish
-#         env:
-#           GH_TOKEN: ${{ secrets.CULTIONET_TOKEN }}
-
-#   # https://github.com/fnkr/github-action-ghr
-#   Release:
-#     needs: Version
-#     if: github.event_name == 'push' && github.ref == 'refs/heads/main' && !contains(github.event.head_commit.message, 'chore(release):')
-#     name: Upload release
-#     runs-on: ubuntu-latest
-#     steps:
-#       - uses: actions/checkout@v2
-#       - uses: actions/setup-python@v2
-#         with:
-#           python-version: 3.8.12
-#       - uses: syphar/restore-virtualenv@v1
-#         id: cnetenv
-#         with:
-#           requirement_files: setup.cfg
-#       - uses: syphar/restore-pip-download-cache@v1
-#         if: steps.cnetenv.outputs.cache-hit != 'true'
-#       - name: Checkout
-#         uses: fnkr/github-action-ghr@v1
-#         if: startsWith(github.ref, 'refs/tags/')
-#         env:
-#           GHR_PATH: .
-#           GHR_COMPRESS: gz
-#           GITHUB_TOKEN: ${{ secrets.CULTIONET_TOKEN }}
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -3,12 +3,14 @@ repos:
     rev: 22.3.0
     hooks:
     - id: black
+      exclude: src/cultionet/utils/stats.py
       language_version: python3
       args: [--skip-string-normalization]
   - repo: https://github.com/myint/docformatter
     rev: v1.4
     hooks:
     - id: docformatter
+      exclude: src/cultionet/utils/stats.py
       args: [
         --in-place,
         --wrap-summaries,
@@ -20,3 +22,11 @@ repos:
     rev: 6.0.0
     hooks:
     - id: flake8
+      exclude: src/cultionet/utils/stats.py
+  - repo: https://github.com/pycqa/isort
+    rev: 5.11.5
+    hooks:
+      - id: isort
+        exclude: src/cultionet/utils/stats.py
+        name: isort (python)
+        args: [--settings-path=pyproject.toml]