CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration (NeurIPS 2021)

PyTorch implementation of the paper:

CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration by:

Hao Yu, Fu Li, Mahdi Saleh, Benjamin Busam and Slobodan Ilic.

Introduction

We study the problem of extracting correspondences between a pair of point clouds for registration. For correspondence retrieval, existing works benefit from matching sparse keypoints detected from dense points but usually struggle to guarantee their repeatability. To address this issue, we present CoFiNet - Coarse-to-Fine Network which extracts hierarchical correspondences from coarse to fine without keypoint detection. On a coarse scale and guided by a weighting scheme, our model firstly learns to match down-sampled nodes whose vicinity points share more overlap, which significantly shrinks the search space of a consecutive stage. On a finer scale, node proposals are consecutively expanded to patches that consist of groups of points together with associated descriptors. Point correspondences are then refined from the overlap areas of corresponding patches, by a density-adaptive matching module capable to deal with varying point density. Extensive evaluation of CoFiNet on both indoor and outdoor standard benchmarks shows our superiority over existing methods. Especially on 3DLoMatch where point clouds share less overlap, CoFiNet significantly outperforms state-of-the-art approaches by at least 5% on Registration Recall, with at most two-third of their parameters.

News

• 09.03.2022: Add support for training and testing on KITTI.

• 28.10.2021: Paper available on arxiv.

• 27.10.2021: Release training and testing code of 3DMatch and 3DLoMatch.

Installation

• Clone the repository:

  git clone https://github.com/haoyu94/Coarse-to-fine-correspondences.git
  cd Coarse-to-fine-correspondences
  

• Create conda environment and install requirements:

  conda create -n {environment name} python=3.8
  pip install -r requirements.txt
  

• Compile C++ and CUDA scripts:

  cd cpp_wrappers
  sh compile_wrappers.sh
  cd ..
  

Demo

TBD

3DMatch & 3DLoMatch

Pretrained model

Pretrained model is given in weights/.

Prepare datasets

sh scripts/download_data.sh

Train

sh scripts/train_3dmatch.sh

Test

• Point correspondences are first extracted by running:

sh scripts/test_3dmatch.sh

and stored on snapshot/tdmatch_enc_dec_test/3DMatch/.

• To evaluate on 3DLoMatch, please change the benchmark keyword in configs/tdmatch/tdmatch_test.yaml from 3DMatch to 3DLoMatch.

• The evaluation of extracted correspondences and relative poses estimated by RANSAC can be done by running:

sh scripts/run_ransac.sh

• The final results are stored in est_traj/3DMatch/{number of correspondences}/result and the results evaluated on our computer have been provided in est_traj/.

• To evaluate on 3DLoMatch, please change 3DMatch in scripts/run_ransac.sh to 3DLoMatch.

KITTI

Pretrained model

Pretrained model is given in weights/.

Prepare datasets

Please follow PREDATOR for data preperation.

There should be two folders poses and sequences under ./data/kitti/dataset/.

Train

sh scripts/train_kitti.sh

Test

sh scripts/test_kitti.sh

Acknowledgments

• The code is heavily borrowed from PREDATOR.

• Our backbone network is from KPConv.

• We use the Transformer implementation in SuperGlue.

• Sinkhorn implementation is from SuperGlue and RPM-Net.

We thank the authors for their excellent work!

Citiation

If you find this repository helpful, please cite:

@article{yu2021cofinet,
  title={CoFiNet: Reliable Coarse-to-fine Correspondences for Robust PointCloud Registration},
  author={Yu, Hao and Li, Fu and Saleh, Mahdi and Busam, Benjamin and Ilic, Slobodan},
  journal={Advances in Neural Information Processing Systems},
  volume={34},
  year={2021}
}