Skip to content

Latest commit

 

History

History
121 lines (90 loc) · 5.8 KB

README.md

File metadata and controls

121 lines (90 loc) · 5.8 KB

Robust Re-Identification by Multiple Views Knowledge Distillation

This repository contains Pytorch code for the ECCV20 paper "Robust Re-Identification by Multiple Views Knowledge Distillation" [arXiv]

VKD - Overview

@inproceedings{porrello2020robust,    
    title={Robust Re-Identification by Multiple Views Knowledge Distillation},
    author={Porrello, Angelo and Bergamini, Luca and Calderara, Simone},
    booktitle={European Conference on Computer Vision},
    pages={93--110},
    year={2020},
    organization={Springer}
}

Installation Note

Tested with Python3.6.8 on Ubuntu (17.04, 18.04).

  • Setup an empty pip environment
  • Install packages using pip install -r requirements.txt
  • Install torch1.3.1 using pip install torch==1.3.1+cu92 torchvision==0.4.2+cu92 -f https://download.pytorch.org/whl/torch_stable.html
  • Place datasets in .datasets/ (Please note you may need do request some of them to their respective authors)
  • Run scripts from commands.txt

Please note that if you're running the code from Pycharm (or another IDE) you may need to manually set the working path to PROJECT_PATH

VKD Training (MARS [1])

Data preparation

  • Create the folder ./datasets/mars
  • Download the dataset from here
  • Unzip data and place the two folders inside the MARS [1] folder
  • Download metadata from here
  • Place them in a folder named info under the same path
  • You should end up with the following structure:
PROJECT_PATH/datasets/mars/
|-- bbox_train/
|-- bbox_test/
|-- info/

Teacher-Student Training

First step: the backbone network is trained for the standard Video-To-Video setting. In this stage, each training example comprises of N images drawn from the same tracklet (N=8 by default; you can change it through the argument --num_train_images.

# To train ResNet-50 on MARS (teacher, first step) run:
python ./tools/train_v2v.py mars --backbone resnet50 --num_train_images 8 --p 8 --k 4 --exp_name base_mars_resnet50 --first_milestone 100 --step_milestone 100

Second step: we appoint it as the teacher and freeze its parameters. Then, a new network with the role of the student is instantiated. In doing so, we feed N views (i.e. images captured from multiple cameras) as input to the teacher and ask the student to mimic the same outputs from fewer (M=2 by default,--num_student_images) frames.

# To train a ResVKD-50 (student) run:
python ./tools/train_distill.py mars ./logs/base_mars_resnet50 --exp_name distill_mars_resnet50 --p 12 --k 4 --step_milestone 150 --num_epochs 500

Model Zoo

We provide a bunch of pre-trained checkpoints through two zip files (baseline.zip containing the weights of the teacher networks, distilled.zip the student ones). Therefore, to evaluate ResNet-50 and ResVKD-50 on MARS, proceed as follows:

  • Download baseline.zip from here and distilled.zip from here (~4.8 GB)
  • Unzip the two folders inside the PROJECT_PATH/logs folder
  • Then, you can evaluate both networks using the eval.py script:
python ./tools/eval.py mars ./logs/baseline_public/mars/base_mars_resnet50 --trinet_chk_name chk_end
python ./tools/eval.py mars ./logs/distilled_public/mars/selfdistill/distill_mars_resnet50 --trinet_chk_name chk_di_1

You should end up with the following results on MARS (see Tab.1 of the paper for VeRi-776 and Duke-Video-ReID):

Backbone top1 I2V mAP I2V top1 V2V mAP V2V
ResNet-34 80.81 70.74 86.67 78.03
ResVKD-34 82.17 73.68 87.83 79.50
ResNet-50 82.22 73.38 87.88 81.13
ResVKD-50 83.89 77.27 88.74 82.22
ResNet-101 82.78 74.94 88.59 81.66
ResVKD-101 85.91 77.64 89.60 82.65
Backbone top1 I2V mAP I2V top1 V2V mAP V2V
ResNet-50bam 82.58 74.11 88.54 81.19
ResVKD-50bam 84.34 78.13 89.39 83.07
Backbone top1 I2V mAP I2V top1 V2V mAP V2V
DenseNet-121 82.68 74.34 89.75 81.93
DenseVKD-121 84.04 77.09 89.80 82.84
Backbone top1 I2V mAP I2V top1 V2V mAP V2V
MobileNet-V2 78.64 67.94 85.96 77.10
MobileVKD-V2 83.33 73.95 88.13 79.62

Teacher-Student Explanations

As discussed in the main paper, we have leveraged GradCam [2] to highlight the input regions that have been considered paramount for predicting the identity. We have performed the same analysis for the teacher network as well as for the student one: as can be seen, the latter pays more attention to the subject of interest compared to its teacher.

Model Explanation

You can draw the heatmaps with the following command:

python -u ./tools/save_heatmaps.py mars <path-to-teacher-net> --chk_net1 <teacher-checkpoint-name> <path-to-student-net> --chk_net2 <student-checkpoint-name> --dest_path <output-dir>

References

  1. Zheng, L., Bie, Z., Sun, Y., Wang, J., Su, C., Wang, S., Tian, Q.: Mars: A video benchmark for large-scale person re-identification. In: European Conference on Computer Vision (2016)
  2. Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision (pp. 618-626).