shufflenet v2+ssd performance loss too much

[DL-85]

Thanks for the great tools!
When I use this tool to generate table from shufflenet v2+ssd prototxt and caffemodel, and use the caffe2ncnn tool to get int8 model, the performance loss too much, but mobilenet v1+ssd is fine with this tool. Why is it, any idea?

[DL-85]

Do I have to merge the BatchNorm layer and Scale layer to Convolution layer (modified the bias_term: to true and convert the caffemodel) before use this tool?

[BUG1989]

This effect will be better

[leeburt]

@DL-85 hi，i follow the step to convert my mobilenet-ssd to int8 model,but the cost time is more and the performance is bad. Do you know the reason？It is very grateful if you give me any suggest.

[ndcuong91]

@BUG1989 @DL-85 i evaluated result of mobilenet_ssd on VOC2007 and i got the same problem:

• fp32 model: mAP: 66.76%
• int8 model: mAP=54.75%
  I used the model here and confident threshold=0.01.
  Any suggestion?

[BUG1989]

@titikid For this model(mobilenet_v1_ssd 300),I find that if you close the conv1 layer Int8 forward,and using Float32 forward inplace it,the mAP maybe increase.Please help me confirm it.
Thanks a lot!
no-retrained int8 model with a little accuracy loss is a difficult task.I'm always trying new ways to meet the int8 requirement.

[ndcuong91]

@BUG1989 How do you know it?
Anw, i forgot to modify confident_threshold param in file mobilenet_ssd_voc_ncnn.param. Here are updated result for mobilenet_ssd_300: (change confident_thres from 0.25 to 0.01)

• fp32 model: mAP: 72.33%
• int8 model: mAP=60.15%

[BUG1989]

@titikid We tried it out layer by layer ^_^

[BUG1989]

Recently i have been trying a new implement : kld + fine-tuning.It can reduce the precision loss(mAP loss < %).But i do the work in my spare time, so I don't know when I can finish it (

[ndcuong91]

@BUG1989 yes.kld+fine-tuning should be better.
Sorry. This is my first time to use ncnn framework. How can i "close the conv1 layer Int8 forward,and using Float32 forward inplace it" as your recommendation?

[BUG1989]

@titikid mixed-precision-inference

[ndcuong91]

@BUG1989 is there some mistake here: "before converting your model files, delete the layer weight scale line in table file, and that layer will do the float32 inference" ???. i think table file is output of converting process.

[ndcuong91]

@BUG1989 i add 7=1 in conv1 in param file and the accuracy is significantly increase (mAP=66.35%). Cheer!

[BUG1989]

@titikid Thank you very much for helping me with this experiment

[BUG1989]

@DL-85 @titikid I test a new case,with output channel split,[k, k, cin, cout] cut into "cout" buckets.quantize all layer in mobilenet_v1_ssd,the mAP from 60.38 to 68.85(the mAP of FP32 is 70.49)

[ndcuong91]

@BUG1989 Do you have any document for new quantization method (base on number of group)? btw, i couldn't see any MobileNet_v1_dev.table in repository

[BUG1989]

@titikid Hi,I have update those files,base on number of output channel number.

• using caffe-int8-convert-tools-dev-weight.py to generate the calibration.table,maybe you can use those tables ;
• using this ncnn to adapt it(caffe2ncnn,examples,and so on).

[ndcuong91]

@BUG1989 i will try it. Thanks for your work!

[andeyeluguo]

how did you get the quantization performance? @titikid

[andeyeluguo]

what do you mean "add 7=1 in conv1 in param file"