Improvement in crop mapping from satellite image time series by effectively supervising deep neural networks
The official implementation code for our paper "Improvement in crop mapping from satellite image time series by effectively supervising deep neural networks".
Our paper has been accepted to ISPRS Journal of Photogrammetry and Remote Sensing and is publicly available at: ScienceDirect.
To learn more discriminative feature representations to detect crop types and reveal the importance of proper supervision of deep neural networks in improving performance, we propose to supervise intermediate layers of a designed 3D Fully Convolutional Neural Network (FCN) by employing two middle supervision methods: Cross-entropy loss Middle Supervision (CE-MidS) and a novel middle supervision method, namely Supervised Contrastive loss Middle Supervision (SupCon-MidS). SupCon-MidS pulls together features belonging to the same class in embedding space, while pushing apart features from different classes. We demonstrate that SupCon-MidS enhances feature discrimination and clustering throughout the network, thereby improving the network performance. In addition, we employ two output supervision methods, namely F1 loss and Intersection Over Union (IOU) loss, which outperfrom the widely used cross-entropy loss.
1- Download the preprocessed training data for the four sites from google drive using the following links: Site_A , Site_B , Site_C , Site_D - Then put them in the data folder.
2- Train the model with the loss function(s) of your choice using the follwing code:
python main.py --out_supervision 'IOU' --mid_supervision 'SupCon' --site 'A' --validation_fold 5 --data_dir 'data' --save_dir 'save'
-
out_supervision: You can select either of 'IOU', 'F1', or 'Cross-entropy'.
-
mid_supervision: You can select either of 'SupCon','Cross-entropy', or'off' (i.e. No middle supervision - Only output supervision).
-
site: You can select either of 'A','B','C','D'.
-
validation_fold: An integer between 1 and 5 used to select the validation fold. Train the model using all five data splits to obtain five sets of weights for the model (obtained by perfroming early stopping).
-
save_dir: Set the path for saving the weights.
-
data_dir: Path where the data of the four sites are located.
3- Take the average of the softmax outputs of the five obtained weights for the model using the test data to obtain the final prediction probabilities. The test data will be uploaded soon.
@article{mohammadi2023improvement,
title={Improvement in crop mapping from satellite image time series by effectively supervising deep neural networks},
author={Mohammadi, Sina and Belgiu, Mariana and Stein, Alfred},
journal={ISPRS Journal of Photogrammetry and Remote Sensing},
volume={198},
pages={272--283},
year={2023},
publisher={Elsevier}
}