Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update tutorial for transforms and pretrained weights #2454

Merged
merged 7 commits into from
Dec 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 23 additions & 9 deletions docs/tutorials/pretrained_weights.ipynb
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
{
"cells": [
{
"cell_type": "markdown",
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "p63J-QmUrMN-"
},
"outputs": [],
"source": [
"Copyright (c) Microsoft Corporation. All rights reserved.\n",
"\n",
"Licensed under the MIT License."
"# Copyright (c) Microsoft Corporation. All rights reserved.\n",
"# Licensed under the MIT License."
]
},
{
Expand All @@ -19,6 +20,8 @@
"source": [
"# Pretrained Weights\n",
"\n",
"_Written by: Nils Lehmann_\n",
"\n",
"In this tutorial, we demonstrate some available pretrained weights in TorchGeo. The implementation follows torchvisions' recently introduced [Multi-Weight API](https://pytorch.org/blog/introducing-torchvision-new-multi-weight-support-api/). We will use the [EuroSAT](https://torchgeo.readthedocs.io/en/stable/api/datasets.html#eurosat) dataset throughout this tutorial. Specifically, a subset containing only 100 images.\n",
"\n",
"It's recommended to run this notebook on Google Colab if you don't have your own GPU. Click the \"Open in Colab\" button above to get started."
Expand Down Expand Up @@ -147,9 +150,11 @@
"source": [
"## Weights\n",
"\n",
"Available pretrained weights are listed on the model documentation [page](https://torchgeo.readthedocs.io/en/stable/api/models.html). While some weights only accept RGB channel input, some weights have been pretrained on Sentinel 2 imagery with 13 input channels and can hence prove useful for transfer learning tasks involving Sentinel 2 data.\n",
"Pretrained weights for `torchgeo.models` are available and sorted by satellite or sensor type: sensor-agnostic, Landsat, NAIP, Sentinel-1, and Sentinel-2. Refer to the [model documentation](https://torchgeo.readthedocs.io/en/stable/api/models.html#pretrained-weights) for a complete list of weights. Choose from the provided pre-trained weights based on your specific use case.\n",
"\n",
"While some weights only accept RGB channel input, some weights have been pretrained on Sentinel-2 imagery with 13 input channels and can hence prove useful for transfer learning tasks involving Sentinel-2 data.\n",
"\n",
"To access these weights you can do the following:"
"To use these weights, you can load them as follows:"
]
},
{
Expand All @@ -169,7 +174,16 @@
"id": "EIpnXuXgrMOM"
},
"source": [
"This set of weights is a torchvision `WeightEnum` and holds information such as the download url link or additional meta data. TorchGeo takes care of the downloading and initialization of models with a desired set of weights. Given that EuroSAT is a classification dataset, we can use a `ClassificationTask` object that holds the model and optimizer object as well as the training logic."
"This set of weights is a torchvision `WeightEnum` and holds information such as the download url link or additional meta data. TorchGeo takes care of the downloading and initialization of models with a desired set of weights. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`torchgeo.trainers` provides specialized task classes that simplify training workflows for common geospatial tasks. Depending on your objective, you can select the appropriate trainer class, such as `ClassificationTask` for classification, `SemanticSegmentationTask` for semantic segmentation, or other task-specific trainers. Check the [trainers documentation](https://torchgeo.readthedocs.io/en/stable/api/trainers.html) for more information.\n",
"\n",
"Given that EuroSAT is a classification dataset, we can use a `ClassificationTask` object that holds the model and optimizer as well as the training logic."
]
},
{
Expand Down Expand Up @@ -495,7 +509,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.8"
"version": "3.13.0"
},
"vscode": {
"interpreter": {
Expand All @@ -504,5 +518,5 @@
}
},
"nbformat": 4,
"nbformat_minor": 0
"nbformat_minor": 4
}
51 changes: 42 additions & 9 deletions docs/tutorials/transforms.ipynb
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
{
"cells": [
{
"cell_type": "markdown",
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "DYndcZst_kdr"
},
"outputs": [],
"source": [
"Copyright (c) Microsoft Corporation. All rights reserved.\n",
"\n",
"Licensed under the MIT License."
"# Copyright (c) Microsoft Corporation. All rights reserved.\n",
"# Licensed under the MIT License."
]
},
{
Expand All @@ -20,6 +21,13 @@
"# Transforms"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"_Written by: Isaac A. Corley_"
]
},
{
"cell_type": "markdown",
"metadata": {
Expand Down Expand Up @@ -408,7 +416,7 @@
"id": "p28C8cTGE3dP"
},
"source": [
"Transforms are able to operate across batches of samples and singular samples. This allows them to be used inside the dataset itself or externally, chained together with other transform operations using `nn.Sequential`. "
"`torchgeo.transforms` work seamlessly with both singular samples and batches of data. They can be applied within datasets or externally and combined with other transforms using `nn.Sequential`. Built for multispectral imagery, they are fully compatible with `torchvision.transforms` and `kornia.augmentation`."
]
},
{
Expand All @@ -429,13 +437,24 @@
"print(x.dtype, x.min(), x.max())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Appending Indices\n",
"\n",
"`torchgeo.transforms` support appending indices to a specified channel dimension.\n",
"\n",
"For detailed usage of all available transforms, refer to the [transforms documentation](https://torchgeo.readthedocs.io/en/stable/api/transforms.html)."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "KRjb-u0EEmDf"
},
"source": [
"Indices can also be computed on batches of images and appended as an additional band to the specified channel dimension. Notice how the number of channels increases from 13 -> 14."
"The following example shows how indices can be computed on batches of images and appended as an additional band to the specified channel dimension. Notice how the number of channels increases from 13 -> 14."
]
},
{
Expand Down Expand Up @@ -500,7 +519,9 @@
"id": "w4ZbjxPyHoiB"
},
"source": [
"It's even possible to chain indices along with augmentations from Kornia for a single callable during training."
"It's even possible to chain indices along with augmentations from Kornia for a single callable during training.\n",
"\n",
"When using Kornia with a dictionary input, you must explicitly set `data_keys=None` during the creation of the augmentation pipeline."
]
},
{
Expand Down Expand Up @@ -689,6 +710,18 @@
"print(f\"Class Label: {dataset.classes[sample['label']]}\")\n",
"image.resize((256, 256), resample=Image.BILINEAR)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Additional Reading\n",
"\n",
"To learn more about preprocessing and data augmentation transforms, the following external resources may be helpful:\n",
"\n",
"* [Kornia augmentations](https://kornia.readthedocs.io/en/latest/augmentation.html)\n",
"* [torchvision transforms](https://pytorch.org/vision/main/transforms.html)"
]
}
],
"metadata": {
Expand Down Expand Up @@ -717,9 +750,9 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.8"
"version": "3.13.0"
}
},
"nbformat": 4,
"nbformat_minor": 0
"nbformat_minor": 4
}
Loading