AlignNet-3D: Fast Point Cloud Registration of Partially Observed Objects

Introduction

This repository is code release for our 3DV 2019 paper (arXiv report here).

Citation

If you use our code or data, please cite

    @inproceedings{Gross193DV,
      author = {Johannes Gro{\ss} and Aljo\u{s}a O\u{s}ep and Bastian Leibe},
      title = {AlignNet-3D: Fast Point Cloud Registration of Partially Observed Objects},
      booktitle = {International Conference on 3D Vision (3DV)},
      year = {2019}
    }

If you use the data, please also cite the original dataset:

    @inproceedings{Geiger12CVPR,
      author = {Andreas Geiger and Philip Lenz and Raquel Urtasun},
      title = {Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite},
      booktitle = {Conference on Computer Vision and Pattern Recognition (CVPR)},
      year = {2012}
    }

Installation

• Install Tensorflow (tested with version 1.8.0)
• Clone Open3D fork from https://github.com/grossjohannes/Open3D
• Build Open3D, build and install the Open3D python package (see http://www.open3d.org/docs/compilation.html)
• Install dependencies with pip install -r requirements.txt
• Run with Python 3

Dataset preparation

• Download the datasets:
  • Synth20.zip
  • Synth20others.zip
  • SynthCars.zip
  • SynthCarsPersons.zip
  • KITTITrackletsCars.zip
  • KITTITrackletsCarsPersons.zip
  • KITTITrackletsCarsHard.zip
  • KITTITrackletsCarsPersonsHard.zip
• Extract the datasets to the same directory, e.g. /home/gross/data. The folder structure should look like

data
│
└───SynthCars
│   │
│   └───meta
│   │   │   00000000.json
│   │   │   00000001.json
│   │   │   ...
│   │
│   └───pointcloud1
│   │   │   00000000.npy
│   │   │   00000001.npy
│   │   │   ...
│   │
│   ...
│
└───SynthCarsPersons
    │   ...

Running ICP evaluations

• Specify your dataset folder (e.g. /home/gross/data) in make_icp_configs.py
• Prepare the icp configs by running python make_icp_configs.py
• Run all icp evaluations at once with ./eval_icp.sh

Training

• Adapt logging.basedir in configs/default.json
• Run e.g. python train.py --config configs/SynthCars.json
• Models and evaluation results will be written to the specified logging.basedir
• For models with pre-training from other models, adapt training.pretraining.model in the respective config files (e.g. configs/KITTITrackletsCars.json)

Evaluation

• To run the evaluation (again) with an existing model checkpoint, run e.g. python train.py eval_only --config configs/KITTITrackletsCarsHard.json --eval_epoch 28
  • The results will e.g. be in /home/gross/models/KITTITrackletsCarsHard/val/eval000028/
  • eval.json contains the results when the full angle is evaluated, eval_180.json the evaluation for the predicted angle/flipped angle closest to the ground truth angle
• To run the evaluation (again) with already computed inference outputs (pred_translations.npy, pred_angles.npy, ...), run e.g. python train.py eval_only --config configs/KITTITrackletsCarsHard.json --eval_epoch 28 --use_old_results
• To run the evaluation (again) with ICP refinement, run e.g. python train.py eval_only --config configs/KITTITrackletsCarsHard.json --eval_epoch 28 --refineICP --use_old_results
  • The evaluation results are written to a refined_p2p subfolder
• Some trained models and evaluation results can be found in models_alignnet.zip

License

Our code is released under BSD-3 License (see LICENSE file for details).

References

• This repository builds upon, thus borrows code heavily from PointNet, Frustum PointNets and DGCNN.