About Image backbone checkpoint

[sunnyHelen]

Hi, thanks a lot for sharing your great work.
I want to know where to find the pretrained R50 on COCO and NuImage for the initialization of the backbone of the camera branch.

[Alexander0Yang]

The pretrained image backbone weights can be downloaded from https://download.openmmlab.com/mmdetection3d/v0.1.0_models/nuimages_semseg/cascade_mask_rcnn_r50_fpn_coco-20e_20e_nuim/cascade_mask_rcnn_r50_fpn_coco-20e_20e_nuim_20201009_124951-40963960.pth. Or you can train it by yourself using the corresponding config.

[VijayLuo]

Hi, awesome work!
the DeepInteraction-large uses Swin-Tiny as image backbone. but i didn't find the pretrained image backbone weights in the paper and this repository. Where can i download it or how can i train it.

[MaiRajborirug]

Hi, thank you for providing everything necessary

@Alexander0Yang I wonder in the tools/test.py code, which checkpoint should I use? There are r50_fpn_voxel.pth, Fusion_0075_refactor.pth, and the one you mention earlier cascade_mask_rcnn_r50_fpn_coco-20e_20e_nuim_20201009_124951-40963960.pth.

From my understanding Fusion_0075_refactor.pth is the lidar weight and cascade_mask_rcnn_r50_fpn_coco-20e_20e_nuim_20201009_124951-40963960.pth. is the camera weight. However, how should I put both of them together in the inference code though?

Much appreciate!