Skip to content

Latest commit

 

History

History
141 lines (113 loc) · 7.28 KB

README.md

File metadata and controls

141 lines (113 loc) · 7.28 KB
Raw

LargeScaleIncrementalLearning

This is the implementation of CVPR 2019 paper "Large Scale Incremental Learning". If the paper and code helps you, we would appreciate your kindly citations of our paper.

@inproceedings{wu2019large,
  title={Large Scale Incremental Learning},
  author={Wu, Yue and Chen, Yinpeng and Wang, Lijuan and Ye, Yuancheng and Liu, Zicheng and Guo, Yandong and Fu, Yun},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  pages={374--382},
  year={2019}
}

Abstract

In this paper, we proposed a new method to address the imbalance issue in incremental learning, which is critical when the number of classes becomes large. Firstly, we validated our hypothesis that the classifier layer (the last fully connected layer) has a strong bias towards the new classes, which has substantially more training data than the old classes. Secondly, we found that this bias can be effectively corrected by applying a linear model with a small validation set. Our method has excellent results on two large datasets with 1,000+ classes (ImageNet ILSVRC 2012 and MS-Celeb-1M), outperforming the state-of-the-art by a large margin (11.1% on ImageNet ILSVRC 2012 and 13.2% on MS-Celeb-1M).

Environment Setup

Words before the code: most codes and experiments are finished in late 2017 and earlier 2018. It is hard to retrieve exact the same environment for experiments, which I remember that the system was in Ubuntu 14. CUDA and tensorflow are all with earlier versions. I re-installed the system several times last year (2018) because some conficts in setting up environment for pytorch, which was original fit for caffe and tensorflow. And also, I upgraded the system form Ubuntu 14 to Ubuntu 16.

The resnet implementation is the official tensorflow official models at:

https://github.com/tensorflow/models

In the latest repo, the most similar implementaion is:

https://github.com/tensorflow/models/blob/master/official/r1/resnet/imagenet_main.py

Unfortunately, we were not able to run our code with the latest tensorflow-2.0 or tensorflow-1.14.

We understand that how important it is to reproduce the results of published papers. I find a compatible tensorflow-1.5 version that is able to run the code with my current CUDA settings. To not mess up with my current software enviroment for pytorch mostly, I use the virtualenv to set up a seperate environment for experiments. The dependency of the code is lite and should be able work in most cases if you get the tensorflow version set up correctly. I summarize my current environment for reference.

System Information
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial

CUDA version: 9.0
CUDNN version: 7.0.5
GPU: TITAN X (Pascal) 12GB

Python version: 3.5.2
Environemnt is setup using virtualenv.

virtualenv venv
pip install scipy
pip install tensorflow-gpu==1.5

To activate the environment:

source venv/bin/activate

To run the code

Dataset:
We mainly clean the code for ImageNet dataset and leave the other two datasets (CIFAR and MS-Celeb-1M) in future work.

Data preparation:
We have put essential files in the repo so that what you need to do is to download the ImageNet-1000 images. For the images, we used the ILSVRC2016_CLS-LOC.tar.gz file, which should be kept unchanged since 2012.
Some links to handle the 50K validation images to split validation images in class folders.

https://github.com/facebook/fb.resnet.torch/blob/master/INSTALL.md#download-the-imagenet-dataset

After downloading and processing, the structure in the data forlder (dataImageNet100 and dataImageNet1000) should be like:

|--train
|  |--n01440764
|  |--n01443537
|  |.......
|--train.txt
|--val
|  |--n01440764
|  |--n01443537
|  |.......
|--val.txt

Experiments run command:

CUDA_VISIBLE_DEVICES=0 python imagenet_main.py 1>log 2>&1

Results

We run the code once and report the result here. Top-5 accuracy is reported on ImageNet dataset. We observe similar results on ImageNet-100 form this repo with new environment and what we reported in paper. On ImageNet-1000, we notice some performance drop in several first increments but the final results are even better than what we reported in our paper, which might be caused by different operation systems and software environments.

Training on ImageNet-100 takes around 15 hours. Results are:

10 20 30 40 50 60 70 80 90 100
In paper (%) 98.40 96.20 94.00 92.90 91.10 89.40 88.10 86.50 85.40 84.40
This Repo (%) 98.79 96.10 95.06 93.50 90.96 89.73 89.02 87.22 85.97 84.24
Before Bias Correction (%) - 95.70 93.06 90.75 86.60 86.06 83.60 81.25 78.28 76.32
\beta 1.0 0.5900 0.5140 0.4742 0.4839 0.4648 0.4389 0.4297 0.4335 0.3941
\gamma 0.0 -0.4416 -0.5401 -0.4701 -0.5323 -0.4672 -0.4830 -0.5349 -0.5804 -0.4609
Training Samples 12800 14417 14600 14600 14600 14441 14600 14620 14331 14566
Val Samples 200 400 300 240 250 240 210 160 180 200
Test Samples 500 1000 1500 2000 2500 3000 3500 4000 4500 5000

Training on ImageNet-100 takes around 100 hours. Results are:

100 200 300 400 500 600 700 800 900 1000
In paper (%) 94.10 92.50 89.60 89.10 85.70 83.20 80.20 77.50 75.00 73.20
This Repo (%) 93.72 91.46 88.70 86.63 84.64 83.08 81.37 79.82 78.22 76.76
Before Bias Correction (%) - 89.64 84.50 80.84 78.07 74.89 72.66 70.38 67.93 63.34
\beta 1.0 0.7873 0.7382 0.7053 0.6884 0.6704 0.6609 0.6515 0.6239 0.6334
\gamma 0.0 -0.5586 -0.5759 -0.5015 -0.5505 -0.5137 -0.4677 -0.4471 -0.4118 -0.4064
Training Samples 126856 144159 144505 144301 143776 145238 143391 144418 143277 143046
Val Samples 2000 4000 3000 2400 2500 2400 2100 1600 1800 2000
Test Samples 5000 10000 15000 20000 25000 30000 35000 40000 45000 50000

Results are from one run of the model on ImageNet-100 and ImageNet-1000. Log files are located at

./logs/log-ImageNet100
./logs/log-ImageNet1000

Implementation notes

Class Order:
To keep the same order with iCaRL (https://github.com/srebuffi/iCaRL), we use the same random seed (1993) from numpy to generate the order.

Distilling Loss:
We store the previous network for distilling loss.

Bias Correction:
After learning the Bias Correction parameters (\beta and \gamma), classifier after correction is used for the distilling loss in the next incremental training.

Validation Samples from exemplars:
10% selection is limited on exemplars (old classes). Samples from new classes will match the same number of validation samples.

Useful links

Awesome-Incremental-Learning: https://github.com/xialeiliu/Awesome-Incremental-Learning

Contact

If you found any issue of the code, please contact Yue Wu (@wuyuebupt, Email: [email protected] or [email protected])