Semantic Segmentation Task using custom dataset

[gtgrp-user]

Hi,

I have surface water dataset which I would like to use to train a semantic segmentation model. Both training and validation datasets are loaded in dataloaders. I would like to add a few indices and augmentation so I created the appropriate transforms object.

Now, do I need a custom data module derived from LightningDataModule to run the SemanticSegmentationTask?

[isaaccorley]

Creating a LightningDataModule isn't required but we do have some useful features built into our datamodules. You can train with only dataloaders like so:

model = SemanticSegmentationTask()

trainer = Trainer()
trainer.fit(model, train_dataloader, val_dataloader)
trainer.test(test_dataloader)

You can find more thorough documentation of this here.

[adamjstewart]

ValueError: A frozen dataclass was passed to `apply_to_collection` but this is not allowed.

This could be due to the non-Tensor values in each mini-batch (crs/bbox). See here for how we handle this in our GeoDataModule base class. It may be easiest just to make your own data module.

Another issue I have is adding a transform in the mask dataset. The AugmentationSequential allows data_keys ['image','mask'] together. So the best option would be to apply the transform after loading the datasets in a data loader. But I need to learn how to do it and whether it's the best practice!

Yes, this is best practice. If you set self.aug (or self.train_aug, self.val_aug, etc. if you need them to differ) in your data module, they will be automatically applied to the entire mini-batch.

I would look at L7IrishDataModule and L8BiomeDataModule for similar example data modules. Let me know if you hit any snags.

[gtgrp-user]

Thanks, I will try this today.

[gtgrp-user]

I apologize for the long delay. I encountered a problem in my attempts to work with the L7IrishDataModule and L8BiomeDataModule, I encountered a problem. It appears that these modules are not available in the torchgeo version 0.4.1 that I have. As an alternative, I used the InriaDataModule, which is a NonGeoDataModule, and created my own data module. I was able to train a model successfully and managed to run the test and validate methods with the following metrics:

[{'val_loss': 0.08770599216222763,
  'val_MulticlassAccuracy': 0.9624531269073486,
  'val_MulticlassJaccardIndex': 0.9399352073669434}]

[{'test_loss': 0.0920742079615593,
  'test_MulticlassAccuracy': 0.9709367752075195,
  'test_MulticlassJaccardIndex': 0.9543617367744446}]

I am now able to perform predictions. However, I'm wondering if the torchgeo library also offers a method to perform inference on full-size Sentinel-2 images?

[adamjstewart]

The method to perform inference on full-size uncurated raster files would be to use a GeoDataModule instead of a NonGeoDataModule. L7IrishDataModule and L8BiomeDataModule are available in main if you want to take a look and use them for reference. ChesapeakeCVPRDataModule and NaipChesapeakeDataModule are also GeoDataModules.

[geografif]

Hi, recently I tried to replicate the OP's workflow. Upon trying to fit the pytorch lightning trainer, it does not recognize SemanticSegmentationTask as a LightningModule. Is there any way to resolve this?

TypeError: modelmust be aLightningModuleortorch._dynamo.OptimizedModule, got SemanticSegmentationTask`

[adamjstewart]

Hi @geografif, can you share the code you are running that results in this error? A minimal reproducible example would help me debug this.

[isaaccorley]

@geografif this is probably because you are using pytorch_lightning.Trainer. You need to update to lightning and use lightning.Trainer instead.

[geografif]

Hi @isaaccorley @adamjstewart I used lightning.Trainer and it worked. Thanks!