Skip to content

Latest commit

 

History

History
265 lines (253 loc) · 7.39 KB

README.md

File metadata and controls

265 lines (253 loc) · 7.39 KB

DINO-MC: Self-supervised Contrastive Learning for Remote Sensing Imagery with Multi-sized Local Crops

PyTorch implementation and pretrained models for DINO-MC and DINO-TP. For details, please see our paper.

Pretrained models

Our models are pre-trained on SeCo-100K, and we list their k-nn and linear probing accuracy on EuroSAT. You can download the full checkpoint of pre-trained model with training infomation as well as weights of teacher and student networks used for the downstream tasks.

model arch params k-nn linear download
DINO-MC ViT-S/8 21M 93.41% 94.09% pre-trained ckpt
DINO-MC ResNet-50 23M 93.94% 95.59% pre-trained ckpt
DINO-MC WRN-50 69M 95.65% 95.70% pre-trained ckpt
DINO-MC Swin-t 28M 93.22% 90.54% pre-trained ckpt
DINO-TP ViT-S/8 21M 93.15% 93.89% pre-trained ckpt
DINO-TP ResNet-50 23M 79.05% 86.70% pre-trained ckpt
DINO-TP WRN-50 69M 86.27% 88.15% pre-trained ckpt
DINO-TP Swin-t 28M 92.83% 91.94% pre-trained ckpt

Training

Our codes refer to DINO and SeCo. If you want to pre-train DINO-MC based on your datasets:

python run_with_submitit.py --nodes 1 --ngpus 4 --arch vit_small --data_mode mc --data_path /path/to/dataset/train --output_dir /path/to/saving_dir

Fine-tuning

After pre-training, you can evaluate the representations on three end-to-end fine-tuning downstream tasks.

model arch EuroSAT download
DINO ViT-S/8 97.98% EuroSAT
DINO-MC ViT-S/8 98.15% EuroSAT
DINO-MC Swin-tiny 98.43% EuroSAT
DINO-MC ResNet-50 98.69% EuroSAT
DINO-MC WRN-50-2 98.78% EuroSAT
model arch BigEarthNet-10% download BigEarthNet download
DINO ResNet-50 79.67% BigEarthNet-10% ckpt 85.38% BigEarthNet ckpt
DINO-TP ResNet-50 80.10% BigEarthNet-10% ckpt 85.20% BigEarthNet ckpt
DINO-MC ResNet-50 82.55% BigEarthNet-10% ckpt 86.86% BigEarthNet ckpt
DINO-MC WRN-50-2 82.67% BigEarthNet-10% ckpt 87.22% BigEarthNet ckpt
DINO-MC Swin-tiny 83.84% BigEarthNet-10% ckpt 88.75% BigEarthNet ckpt
DINO-MC ViT-S/8 84.20% BigEarthNet-10% ckpt 88.69% BigEarthNet ckpt
model arch Pre. Rec. F1 download
DINO ResNet-50 57.37 44.21 49.53 OSCD
DINO-MC ResNet-50 51.94 54.04 52.46 OSCD
DINO-TP ResNet-50 51.10 49.03 49.74 OSCD
DINO WRN-50-2 53.58 52.28 52.41 OSCD
DINO-MC WRN-50-2 49.99 56.81 52.70 OSCD
DINO-TP WRN-50-2 55.77 47.30 50.61 OSCD

Citation

If you find this repository useful, please consider giving a star ⭐ and citation:

@misc{wanyan2023dinomc,
      title={DINO-MC: Self-supervised Contrastive Learning for Remote Sensing Imagery with Multi-sized Local Crops}, 
      author={Xinye Wanyan and Sachith Seneviratne and Shuchang Shen and Michael Kirley},
      year={2023},
      eprint={2303.06670},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}