About Mixup

[May-forever]

Hi, @AlexeyAB .

How to use mixup in training networks?

How to see the changes in training images and save them?

[AlexeyAB]

Just set in cfg-file:

[net]
mixup=1

---

How to see the changes in training images and save them?

Use flag -show_imgs at the end of training command
./darknet detector train ... -show_imgs it will show and save augmented images near with executable file

if you don't want to show them (just for saving jpg files) use:
./darknet detector train ... -show_imgs -dont_show

---

For Classifier there are:

• mixup=1 BoF (Bag of Freebies) - Visually Coherent Image Mixup ~+4 AP@[.5, .95] #3272
• cutmix=1 Classifier - CutMix: Regularization Strategy to Train Strong Classifiers #4419
• mosaic=1 Classifier trainin - Mosaic data augmentation #4432

For Detector there are:

• mixup=1 BoF (Bag of Freebies) - Visually Coherent Image Mixup ~+4 AP@[.5, .95] #3272
• mosaic=1 Detector - Mosaic data augmentation  #4264

[May-forever]

Just set in cfg-file:

[net]
mixup=1

How to see the changes in training images and save them?

Use flag -show_imgs at the end of training command
./darknet detector train ... -show_imgs it will show and save augmented images near with executable file

if you don't want to show them (just for saving jpg files) use:
./darknet detector train ... -show_imgs -dont_show

For Classifier there are:

• mixup=1 BoF (Bag of Freebies) - Visually Coherent Image Mixup ~+4 AP@[.5, .95] #3272
• cutmix=1 Classifier - CutMix: Regularization Strategy to Train Strong Classifiers #4419
• mosaic=1 Classifier trainin - Mosaic data augmentation #4432

For Detector there are:

• mixup=1 BoF (Bag of Freebies) - Visually Coherent Image Mixup ~+4 AP@[.5, .95] #3272
• mosaic=1 Detector - Mosaic data augmentation  #4264

Hi @AlexeyAB , thank you very much for your help.

Can't cutmix=1 be used to train a detector yet?

Looking forward to hearing from you, thanks a lot.

[nyj-ocean]

@AlexeyAB
Can I set mixup=1 and mosaic=1 together in yolov3.cfg or Gaussian+CIoU+yolov3.cfg to improve mAP?

[net]
batch=32
subdivisions=16
width=416
height=416
channels=3
momentum=0.9
decay=0.0005
angle=0
saturation = 1.5
exposure = 1.5
hue=.1
mixup=1
mosaic=1

[AlexeyAB]

@nyj-ocean

If you set both - then will be used only Mosaic

mixup=1
mosaic=1

Because in the most tests MixUp doesn't give any gain in AP during training Detector (MixUp gives boost of accuracy only for training Classifier with mixup=1, and then this pre-trained weights file with higher Top1 is used for training Detector with mixup=0 - so it gives higher ~+4 AP)

Mosaic should increase accuracy for both training Classifier and Detector.

---

So is recommended:

• cutmix=1 mosaic=1 for training Classifier
• mosaic=1 for training Detector

[nyj-ocean]

@AlexeyAB
If I only set mosaic=1 in yolov3.cfg.
Then can I directly use the pre-weights darknet53.conv.74 to train mosaic+yolov3?

Or should I train Classifier with mosaic=1 at first,and then this pre-trained weights file with higher Top1 is used for training Detector with mosaic=1?

[AlexeyAB]

Or should I train Classifier with mosaic=1 at first,and then this pre-trained weights file with higher Top1 is used for training Detector with mosaic=1?

Yes, you can. And it will increase accuracy.
But for an even greater increase in accuracy, first train the Classifier with a mosaic and cutmix, and then the Detector with a mosaic.

[nyj-ocean]

@AlexeyAB
I try mosaic+yolov3 in the latest repo.
I only set mosaic=1 in yolov3.cfg.Then I directly use my own dataset to train mosaic+yolov3 with the pre-weights darknet53.conv.74.
It can train normally at the beginning ,but produce NAN later as following.

But I can train mixup+yolov3 normally,and it do not produce NAN

[AlexeyAB]

@nyj-ocean

• Can you attach your cfg-file?
• Does training work well without mosaic=1?
• What dataset do you use?
• Do you use GPU=1 CUDNN=1?

[nyj-ocean]

@AlexeyAB

• Can you attach your cfg-file?
  mosaic+yolov3.cfg.txt

• Does training work well without mosaic=1?
  Yes,I can train yolov3 without mosaic=1 as shown in How to calculate mAP for (Gaussian+GIOU+YOLOv3) in this repository? #4357 (comment)

• What dataset do you use?
  I use my own dataset.
  1500 images for training set ,other 1500 images for val set.

• Do you use GPU=1 CUDNN=1?
  Yes, I set the Makefile like following

GPU=1
CUDNN=1
CUDNN_HALF=1
OPENCV=1
AVX=0
OPENMP=0
LIBSO=0
ZED_CAMERA=0

[nyj-ocean]

@AlexeyAB
But I can train mixup+yolov3 normally,and it do not produce NAN as following

[nyj-ocean]

@AlexeyAB
mosaic+yolov3 produce Nan about 20k steps as shown in #4446 (comment)
So I use the mosaic+yolov3_16000.weigts in backup file to continue training.
But it produce Nan again about 25k steps as shown following
(it breaks at 24k steps )

[AlexeyAB]

@nyj-ocean

I improved mosaic=1 for Detector. Try new mosaic.
Two commits: 87f36b7 and 13f064f

[nyj-ocean]

@AlexeyAB
Thanks.
By the way, can I set mosaic=1 and activation=swish in yolov3.cfg ,and train with darknet53.conv.74 to further improve mAP like following?

[net]
batch=32
subdivisions=16
width=416
height=416
channels=3
momentum=0.9
decay=0.0005
angle=0
saturation = 1.5
exposure = 1.5
hue=.1
mosaic=1
...
...
[convolutional]
batch_normalize=1
filters=32
size=3
stride=1
pad=1
activation=swish

[nyj-ocean]

@AlexeyAB
I only replace activation=swish with activation=leaky in yolov3.cfg,and use darknet53.conv.74 to train.
It do not get any improvement in mAP, but decrease a bit.

[AlexeyAB]

@nyj-ocean
darknet53.conv.74 is trained with leaky-relu. You should retrain darknet53 classifier with swish.

Re-train Darknet53.cfg classifier with activation=swish and mosaic=1 cutmix=1 on ImageNet then do partial
/.darknet partial cfg/darknet53_448.cfg darknet53_448.weights darknet53.conv.74 74

and use this new pre-trained weights darknet53.conv.74 for training detector with activation=swish and mosaic=1

[nyj-ocean]

@AlexeyAB
I set mixup=1 in yolov3.cfg, and use my own dataset to train with darknet53.conv.74.

After training, I use the best.weights in the backup file to calculate mAP with the following command:

./darknet detector map my_obj.data mixup+yolov3.cfg backup/best.weights -points 0

It gets a 0.6% improvement in mAP

I am not sure this improvement in mAP is just for random or for the mixup=1

[AlexeyAB]

@nyj-ocean In such cases, they usually train 3 times with mixup=1 and 3 times without, and take average accuracy.

But instead I suggest to train Detector with new mosaic=1

[nyj-ocean]

@AlexeyAB
I will try to set mosaic=1 in yolov3.cfg, and train with darknet53.conv.74 in latest repo.

[nyj-ocean]

@AlexeyAB
I set mosaic=1 in yolov3.cfg, and train with darknet53.conv.74 in latest repo.
(dataset: 1500 images for training set ,other 1500 images for val set)
After training, I use the best.weights in the backup file to calculate mAP with the following command:

./darknet detector map my_obj.data mosaic+yolov3.cfg backup/best.weights -points 0

It gets a 0.2% improvement in mAP compared with yolov3

mixup+yolov3 gets a 0.6% improvement in mAP compared with yolov3 as shown in #4446 (comment)

[AlexeyAB]

@nyj-ocean Do you use the latest version of Darknet?

[nyj-ocean]

@AlexeyAB
#4446 (comment)

@nyj-ocean
I improved mosaic=1 for Detector. Try new mosaic.
Two commits: 87f36b7 and 13f064f

I download this repo and start to train mosaic+yolov3 just after when you improved mosaic=1 #4446 (comment)