Fine-tuning Mask-RCNN on my own dataset

[raninbowlalala]

I want to fine-tuning Mask-RCNN on my own dataset.

1. I converted mask binary image to RLE format, and generated train.json val.json same as coco format;
2. run: $ python2 tools/train_net.py --cfg configs/getting_started/e2e_mask_rcnn_X-101-64x4d-FPN_1x_protrait.yaml OUTPUT_DIR /tmp/detectron-output
   and I got error as below:

E0227 16:22:45.168256 15153 pybind_state.h:422] Exception encountered running PythonOp function: ValueError: could not convert string to float: c

At:
/home/liufang/detectron_venv/detectron/lib/utils/segms.py(134): polys_to_boxes
/home/liufang/detectron_venv/detectron/lib/roi_data/mask_rcnn.py(46): add_mask_rcnn_blobs
/home/liufang/detectron_venv/detectron/lib/roi_data/fast_rcnn.py(207): _sample_rois
/home/liufang/detectron_venv/detectron/lib/roi_data/fast_rcnn.py(112): add_fast_rcnn_blobs
/home/liufang/detectron_venv/detectron/lib/ops/collect_and_distribute_fpn_rpn_proposals.py(60): forward
terminate called after throwing an instance of 'caffe2::EnforceNotMet'
what(): [enforce fail at pybind_state.h:423] . Exception encountered running PythonOp function: ValueError: could not convert string to float: c

I think detectron only support polygon format, am I right?
If it's true, do you know how to convert RLE to polygon or convert mask binary image to polygon?
Thanks in advance.

[zy2008cn]

I also have problems with my own dataset when training.

[francoto]

Hello,
have you tried with pycocotools API to convert RLE to polygon ?
I haven't test it because I already have polygons but I think it could convert them for you.

[raninbowlalala]

@francoto I checked the pycocotools API and there is no API for converting RLE to polygon. Now, I modified the segms.py to load RLE format and fine-tuning is processing on GPU, but I found the segmentation result is bad when I checked the intermediate checkpoint.

[francoto]

@raninbowlalala You can check issue #100 I think they solved your issue there.

[raninbowlalala]

@francoto Tanks, your answer helps me a lot!

[raninbowlalala]

@francoto Thank you very much. Your answer helps me a lot.

[waspinator]

how did you generate uncompressed RLE masks for iscrowd images?

[enemni]

@waspinator I am using your pycococreator but I am not sure how to use the annotation with the matterport MaskRCNN when is_crowd=1 because it seems that RLE is not compatible with the MaskRCNN implementation. Did you find any solution?