Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework

This repo covers an reference implementation for the paper Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework in PyTorch, using Deep Fashion In-Store as an illustrative example: Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework.

Requirements

• Pytorch 1.7.0
• tensorboard_logger 0.1.0

Typo

• We want to correct a typo in the paper. In Eq. 6 and 7, should be .

Running

• This code is built upon two codebases: Supervised Contrastive Learning and MoCo.

• Train pre-trained model on Deep Fashion In-store dataset

  • Perpare train-listfile, val-listfile. The format is as follows:

   {
     "images": [
       "/deep_fashion_in_store/img/WOMEN/Dresses/id_00000002/02_1_front.jpg",
       "/deep_fashion_in_store/img/WOMEN/Dresses/id_00000002/02_2_side.jpg",
       "/deep_fashion_in_store/img/WOMEN/Dresses/id_00000002/02_4_full.jpg",
       "/deep_fashion_in_store/img/WOMEN/Dresses/id_00000002/02_7_additional.jpg",
       "/deep_fashion_in_store/img/WOMEN/Blouses_Shirts/id_00000004/03_1_front.jpg"
     ],
     "categories": [
       "Dresses",
       "Dresses",
       "Dresses",
       "Dresses",
       "Blouses_Shirts"
     ]
   }
  
  

  • Class map can be downloaded from class_map.json

  • Repeating product ids can be downloaded from repeating_product_ids.csv

  • If experiment on the model transfer ability from seen classes to unseen classes, the two classes maps can be downloaded from class_map_seen.json and class_map_unseen.json.

  • To train the model on Deep Fashion In-store dataset, run

   python train_deepfashion.py --data ./deepfashion/ 
   --train-listfile ./train_listfile.json 
   --val-listfile ./val_listfile.json 
   --test-listfile ./test_listfile.json 
   --class-map-file ./classmap.json 
   --num-classes 17 
   --learning_rate 0.5 --temp 0.1
   --ckpt /pretrained_model/
   --dist-url 'tcp://localhost:10001' 
   --multiprocessing-distributed 
   --world-size 1 --rank 0 --cosine
  
  

  • To evaluate the model, run

   python eval_deepfashion.py --data ./deepfashion/ 
   --train-listfile ./train_listfile.json --val-listfile ./val_listfile.json 
   --class-map-file ./classmap.json 
   --num-classes 17 
   --learning_rate 0.5 --temp 0.1
   --ckpt /trained_model/
  
  

Reference

@inproceedings{hierarchicalContrastiveLearning,
      title={Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework}, 
      author={Shu Zhang and Ran Xu and Caiming Xiong and Chetan Ramaiah},
      year={2022},
      booktitle={CVPR},
}