README

This repo is developed based on OpenPCDet. It additionally estimates uncertainties in each dimension of detection results with Laplace approximation.

Setup the environment

You can set up the environment via Dockerfile, which is adapted from the original Dockerfile of OpenPCDet. In the following, <root_dir> denotes the directory which contains det3/ and openpcuct/.

cd <root_dir>/openpcuct/docker
docker build -t corl2022:193 . -f Dockerfile
# If you are using RTX 3090, please try
# docker build -t corl2022:193 . -f RTX3090Dockerfile
docker run -it --name corl2022_193 --gpus all -v <root_dir>:/usr/app corl2022:193 bash
# inside the docker container, please run
cd /usr/app/openpcuct/tools
./scripts/setup.sh /usr/app/openpcuct

Prepare data and weights

Prepare KITTI data

You can prepare the KITTI dataset as OpenPCDet. Here we copy it from the original OpenPCDet repo for easy reading. Please download the official KITTI 3D object detection dataset and organize the downloaded files as follows:

openpcdet
├── data
│   ├── kitti
│   │   │── ImageSets
│   │   │── training
│   │   │   ├──calib & velodyne & label_2 & image_2
├── pcdet
├── tools

• Generate the data infos by running the following command:

cd <root_dir>
python -m pcdet.datasets.kitti.kitti_dataset create_kitti_infos tools/cfgs/dataset_configs/kitti_dataset.yaml

• Generate the ground-truth predictive distribution by running the following command:

cd <root_dir>/tools
python3 deploy_experiments.py \
    generate_ground_truth_spatial_uncertainty \
    --exp_name=gen_gt_uct \
    --data_dir=/usr/app/openpcdet/data/kitti/training

Prepare weights

Please download the weights into /usr/app/openpcdet/. They are provided by OpenPCDet and we copy it from the original OpenPCDet repo for easy reading.

	download
PointPillar	model-18M
SECOND	model-20M
PointRCNN	model-16M

Reproduce the results

You can reproduce the experiments by running

cd <root_dir>/tools
# mAPIoU & mAPJIoU results will be printed after each evaluation
python3 scripts/reproduce_experiments.py

Then please move all the directories ending with "-gen_pred_dist" from <root_dir>/output/{pointpillar,second,pointrcnn}/ to /usr/app/validate_data/experiment_data/, and organize the folder like

validate_data
├── experiment_data
│   ├── pointpillar-standard-seed666-full-net-gen_pred_dist
│   ├── ...
│   ├── second-standard-seed666-full-net-gen_pred_dist
│   ├── ...
│   ├── pointpillar-standard-seed666-full-net-gen_pred_dist
│   ├── ...
├── result_data

Then you can run the following commands

cd <root_dir>/tools
# plot calibration plots and the ECE table
python3 scripts/plot_calibration_plots.py

The results will be as the following two tables. They are the results of Table 1 of the manuscript.

	PP-deterministic	SC-deterministic	PR-deterministic
cls	0.115217	0.100307	0.113034

	classification-mean	classification-std	regression-mean	regression-std
PP-DT	0.115000	NaN	NaN	NaN
PP-MC	0.112385	0.0	0.763297	0.0
PP-LL(emp.)	0.115262	0.0	0.525599	0.0
PP-FU(emp.)	0.097388	0.0	0.422932	0.0
PP-LL(std.)	0.115223	0.0	0.301047	0.0
PP-FU(std.)	0.113344	0.0	0.321744	0.0
SC-DT	0.100000	NaN	NaN	NaN
SC-MC	0.125214	0.0	0.594675	0.0
SC-LL(emp.)	0.100338	0.0	0.516388	0.0
SC-FU(emp.)	0.087153	0.0	0.394806	0.0
SC-LL(std.)	0.100163	0.0	0.313392	0.0
SC-FU(std.)	0.093891	0.0	0.298234	0.0
PR-DT	0.113000	NaN	NaN	NaN
PR-MC	0.105214	0.0	0.742780	0.0
PR-LL(emp.)	0.113138	0.0	0.345880	0.0
PR-LM(emp.)	0.062914	0.0	0.304791	0.0
PR-FU(emp.)	0.060646	0.0	0.526193	0.0
PR-LL(std.)	0.112998	0.0	0.520265	0.0
PR-LM(std.)	0.100390	0.0	0.479749	0.0
PR-FU(std.)	0.074905	0.0	0.401306	0.0

Claims

This code is developed based on the following bases. We thank their efforts and contributions to the community:

• Code skeleton: OpenPCDet
• Generative model: Inferring Spatial Uncertainty in Object Detection

We implement the Laplace approximation of this CoRL2022 submission in the pcuct/laplace_approx folder. If there is any problem, please feel free to contact us (pyun [email protected]).

Notes

This repo is the very first version of our submission. Here we just release it for your reference. In the coming weeks, we will update this repo by fixing some bugs and adding more features.