Skip to content

Latest commit

 

History

History
71 lines (54 loc) · 4.13 KB

README.md

File metadata and controls

71 lines (54 loc) · 4.13 KB

ViDAR-UniAD Fine-tuning

This repo contains the code and configuration files for ViDAR fine-tuning on UniAD for end-to-end autonomous driving.

Results and Models

Stage1: Perception training

Downstream Model Dataset pre-train Config Detection
NDS		 Tracking
AMOTA		 Mapping
IoU-lane		 models & logs
UniAD-Stage1 (baseline) nuScenes (100% Data) BEVFormer-base: cfg / model base_track_map.py 46.69 36.0 28.7 -
ViDAR-UniAD-Stage1 nuScenes (100% Data) ViDAR-BEVFormer: cfg / model vidar_track_map.py 54.99 45.6 33.8 models / logs

Stage2: End-to-end training

Downstream Model Dataset pre-train Config Detection
NDS		 Tracking
AMOTA		 Mapping
IoU-lane		 Motion
minADE		 Occupancy
IoU-n.		 Planning
avg.Col.		 models & logs
UniAD-Stage2 (baseline) nuScenes (100% Data) UniAD-Stage1: cfg base_e2e.py 49.36 38.3 31.3 0.75 62.8 0.27 -
ViDAR-UniAD-Stage2 nuScenes (100% Data) ViDAR-UniAD-Stage1: cfg / model vidar_e2e.py 54.06 43.5 35.2 0.65 65.7 0.18 models / logs

Getting Started

Installation

  • First, refer to Installation to install ViDAR first.
  • Second, run pip install -r requirements.txt to install extra dependencies.

Data preprocessing

Please refer to Dataset for data preparation before the first run.

Training Command

# stage-1
CONFIG=./projects/configs/stage1_track_map/vidar_track_map.py
GPU_NUM=8
export PYTHONPATH=/PATH/TO/ViDAR/projects/mmdet3d_plugin/bevformer/:${PYTHONPATH}
./tools/uniad_dist_train.sh ${CONFIG} ${GPU_NUM}

# stage-2
CONFIG=./projects/configs/stage2_e2e/vidar_e2e.py
GPU_NUM=16
export PYTHONPATH=/PATH/TO/ViDAR/projects/mmdet3d_plugin/bevformer/:${PYTHONPATH}
./tools/uniad_dist_train.sh ${CONFIG} ${GPU_NUM}

Eval Command

CONFIG=path/to/uniad_config.py
CKPT=path/to/checkpoint.pth
GPU_NUM=8

./tools/uniad_dist_eval.sh ${CONFIG} ${CKPT} ${GPU_NUM}

Related Citations

@inproceedings{yang2023vidar,
    title={Visual Point Cloud Forecasting enables Scalable Autonomous Driving},
    author={Yang, Zetong and Chen, Li and Sun, Yanan and Li, Hongyang},
    booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
    year={2024}
}

@inproceedings{hu2023_uniad,
    title={Planning-oriented Autonomous Driving}, 
    author={Yihan Hu and Jiazhi Yang and Li Chen and Keyu Li and Chonghao Sima and Xizhou Zhu and Siqi Chai and Senyao Du and Tianwei Lin and Wenhai Wang and Lewei Lu and Xiaosong Jia and Qiang Liu and Jifeng Dai and Yu Qiao and Hongyang Li},
    booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
    year={2023}
}