In SIGNet, instance masks need to be generated in '.raw' format in order to be parsed for use in training depth, pose, and rigid flow. For more information about formatting, check the README.md file in the carla-SIGNet repository.
We compute the instance masks using a modified version of Facebook AI Research's Detectron.
This repository contains the modified code in order to generate these instance masks. Note that the masks need to be generated for sequences of 3 images (that are concatenated into one image) during training, and for singular images during testing.
Specifically, look at the infer_simple-eigen script in the tools folder, and the vis-raw file in the detectron/utils folder.
It is easiest to build Detectron from it's docker image in case you do not have Caffe2 installed and running already with the right CUDA version (please see INSTALL.md.
Create a folder called models and place the Mask-RCNN model inside it: https://drive.google.com/open?id=1fvfVgZLND9MT_6jaSXcKvNVirR4reexk
Example running command:
python tools/infer_simple-eigen.py \
--cfg configs/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml \
--output-dir /dir/to/store/results/ \
--image-ext png \
--output-ext png \
--wts models/mask_rcnn_R-101-FPN_2x.pkl \
--kitti_eigen_file /path/to/list/of/images.txt \
demo/kitti
Review the infer_simple-* files for further clarifications.
Below, you can find the original README.md content of the original Detectron repository. Please review it in case of troubleshooting issues with the platform.
Detectron is Facebook AI Research's software system that implements state-of-the-art object detection algorithms, including Mask R-CNN. It is written in Python and powered by the Caffe2 deep learning framework.
At FAIR, Detectron has enabled numerous research projects, including: Feature Pyramid Networks for Object Detection, Mask R-CNN, Detecting and Recognizing Human-Object Interactions, Focal Loss for Dense Object Detection, Non-local Neural Networks, Learning to Segment Every Thing, Data Distillation: Towards Omni-Supervised Learning, DensePose: Dense Human Pose Estimation In The Wild, and Group Normalization.
Example Mask R-CNN output.
The goal of Detectron is to provide a high-quality, high-performance codebase for object detection research. It is designed to be flexible in order to support rapid implementation and evaluation of novel research. Detectron includes implementations of the following object detection algorithms:
- Mask R-CNN -- Marr Prize at ICCV 2017
- RetinaNet -- Best Student Paper Award at ICCV 2017
- Faster R-CNN
- RPN
- Fast R-CNN
- R-FCN
using the following backbone network architectures:
- ResNeXt{50,101,152}
- ResNet{50,101,152}
- Feature Pyramid Networks (with ResNet/ResNeXt)
- VGG16
Additional backbone architectures may be easily implemented. For more details about these models, please see References below.
- 4/2018: Support Group Normalization - see
GN/README.md
Detectron is released under the Apache 2.0 license. See the NOTICE file for additional details.
If you use Detectron in your research or wish to refer to the baseline results published in the Model Zoo, please use the following BibTeX entry.
@misc{Detectron2018,
author = {Ross Girshick and Ilija Radosavovic and Georgia Gkioxari and
Piotr Doll\'{a}r and Kaiming He},
title = {Detectron},
howpublished = {\url{https://github.com/facebookresearch/detectron}},
year = {2018}
}
We provide a large set of baseline results and trained models available for download in the Detectron Model Zoo.
Please find installation instructions for Caffe2 and Detectron in INSTALL.md.
After installation, please see GETTING_STARTED.md for brief tutorials covering inference and training with Detectron.
To start, please check the troubleshooting section of our installation instructions as well as our FAQ. If you couldn't find help there, try searching our GitHub issues. We intend the issues page to be a forum in which the community collectively troubleshoots problems.
If bugs are found, we appreciate pull requests (including adding Q&A's to FAQ.md and improving our installation instructions and troubleshooting documents). Please see CONTRIBUTING.md for more information about contributing to Detectron.
- Data Distillation: Towards Omni-Supervised Learning. Ilija Radosavovic, Piotr Dollár, Ross Girshick, Georgia Gkioxari, and Kaiming He. Tech report, arXiv, Dec. 2017.
- Learning to Segment Every Thing. Ronghang Hu, Piotr Dollár, Kaiming He, Trevor Darrell, and Ross Girshick. Tech report, arXiv, Nov. 2017.
- Non-Local Neural Networks. Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. Tech report, arXiv, Nov. 2017.
- Mask R-CNN. Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. IEEE International Conference on Computer Vision (ICCV), 2017.
- Focal Loss for Dense Object Detection. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. IEEE International Conference on Computer Vision (ICCV), 2017.
- Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour. Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. Tech report, arXiv, June 2017.
- Detecting and Recognizing Human-Object Interactions. Georgia Gkioxari, Ross Girshick, Piotr Dollár, and Kaiming He. Tech report, arXiv, Apr. 2017.
- Feature Pyramid Networks for Object Detection. Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
- Aggregated Residual Transformations for Deep Neural Networks. Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
- R-FCN: Object Detection via Region-based Fully Convolutional Networks. Jifeng Dai, Yi Li, Kaiming He, and Jian Sun. Conference on Neural Information Processing Systems (NIPS), 2016.
- Deep Residual Learning for Image Recognition. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Conference on Neural Information Processing Systems (NIPS), 2015.
- Fast R-CNN. Ross Girshick. IEEE International Conference on Computer Vision (ICCV), 2015.