Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add High-res FasterRCNN MobileNetV3 and tune Low-res for speed #3265

Merged
merged 6 commits into from
Jan 19, 2021

Conversation

datumbox
Copy link
Contributor

@datumbox datumbox commented Jan 19, 2021

  • Rename low-res model.
  • Tune low-res model for speed (2x faster) by sacrificing 0.02 mAP.
  • Add High-res model.

Low Resolution: Faster R-CNN MobileNetV3-Large 320 FPN

Heavily optimized for speed. Good for actual mobile usage.

Weight checkpoint:

PR3253/3rd_training/34945231/model_23.pth

Validate:

python -m torch.distributed.launch --nproc_per_node=2 --use_env train.py\
 --dataset coco --model fasterrcnn_mobilenet_v3_large_320_fpn --test-only --pretrained

Accuracy metrics:

IoU metric: bbox
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.228
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.380
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.232
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.026
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.219
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.444
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.215
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.290
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.294
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.037
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.295
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.570

Speed Benchmark: 0.1679 sec per image on CPU

High Resolution: Faster R-CNN MobileNetV3-Large FPN

Offers good balance between speed and accuracy, significantly faster than the model with the resnet backbone without sacrificing too much accuracy.

Weight checkpoint:

PR3253/high_res/34974236/model_25.pth

Validate:

python -m torch.distributed.launch --nproc_per_node=2 --use_env train.py\
 --dataset coco --model fasterrcnn_mobilenet_v3_large_fpn --test-only --pretrained

Accuracy metrics:

0: IoU metric: bbox
0:  Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.328
0:  Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.525
0:  Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.343
0:  Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.127
0:  Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.363
0:  Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.502
0:  Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.287
0:  Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.427
0:  Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.445
0:  Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.196
0:  Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.499
0:  Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.648

Speed Benchmark: 0.8409 sec per image on CPU

…hat makes it 2x faster sacrificing a bit of mAP.
@datumbox datumbox changed the title [WIP] Add High-res FasterRCNN MobileNetV3 and tune Low-res for speed Add High-res FasterRCNN MobileNetV3 and tune Low-res for speed Jan 19, 2021
@datumbox datumbox requested a review from fmassa January 19, 2021 12:26
Copy link
Member

@fmassa fmassa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks!

@datumbox datumbox mentioned this pull request Jan 19, 2021
13 tasks
@datumbox datumbox force-pushed the mobilenetv3_highres branch 2 times, most recently from 518c422 to 66d861f Compare January 19, 2021 14:14
@datumbox datumbox merged commit ae0d80b into pytorch:master Jan 19, 2021
@datumbox datumbox deleted the mobilenetv3_highres branch January 19, 2021 14:39
facebook-github-bot pushed a commit that referenced this pull request Jan 21, 2021
Summary:
* Tag fasterrcnn mobilenetv3 model with 320, add new inference config that makes it 2x faster sacrificing a bit of mAP.

* Add a high resolution fasterrcnn mobilenetv3 model.

* Update tests and expected values.

Reviewed By: datumbox

Differential Revision: D25954564

fbshipit-source-id: f6b64981d2bc83e3577435481a569df38b427b20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants