CarlaFLCAV

CarlaFLCAV is an open-source FLCAV simulation platform based on CARLA simulator that supports:

• Multi-modal dataset generation: Including point-cloud, image, radar data with associated calibration, synchronization, and annotation

• Training and inference: Examples for CAV perception, including object detection, traffic sign detection, and weather classification

• Various FL frameworks: FedAvg, device selection, noisy aggregation, parameter selection, distillation, and personalization

• Optimization based modules: Network resource and road sensor pose optimization.

Demo

fedsecond.mp4

Federated SECOND for 3D point cloud object detection

fedyolo.mp4

Federated YOLOV5 for 2D image object detection

fedlstm.mp4

Federated LSTM for BEV trajectory prediction

fedfusion.mp4

Cooperative perceptioin with road sensors for federated distillation

Test Environment

• Ubuntu 20.04
• Python 3.8
• CARLA 0.9.13
• CUDA 11.3 (Nvidia Driver 470)
• Pytorch 1.10.0

Citation

CarlaFLCAV can reproduce results in the following papers:

@article{CarlaFLCAV,
  title={Federated deep learning meets autonomous vehicle perception: Design and verification},
  author={Shuai Wang and Chengyang Li and Derrick Wing Kwan Ng and Yonina C. Eldar and H. Vincent Poor and Qi Hao and Chengzhong Xu},
  journal={IEEE Network},
  doi={10.1109/MNET.104.2100403},
  year={2022}
}

@article{CarlaFLOTA,
  title={Edge federated learning via unit-modulus over-the-air computation},
  author={Shuai Wang and Yuncong Hong and Rui Wang and Qi Hao and Yik-Chung Wu and Derrick Wing Kwan Ng},
  journal={IEEE Transactions on Communications},
  year={2022},
  volume={70},
  number={5},
  pages={3141--3156},
  publisher={IEEE}
}

CarlaFLCAV Arxiv version: http://arxiv.org/abs/2206.01748

CarlaFLOTA Arxiv version: https://arxiv.org/abs/2101.12051

Acknowledgement

• Carla
• Carla-ROS-Bridge
• CarlaINVS

Authors

Shuai Wang

Chengyang Li