Virtual to Real Reinforcement Learning for Autonomous Driving

In BMVC 2017 (Spotlight Presentation)

By Xinlei Pan*, Yurong You*, Ziyan Wang, Cewu Lu

News!

We updated with a novel TORCS environment for reinforcement learning here, with support for changing dynamics of the environment, getting segmentations and more!

Introduction

Virtual to real reinforcement learning proposed a realistic translation network that can preprocess virtual images from simulator and translate to their realistic counterparts. It has been proved by experiments that our method can train a driving policy entirely in virtual environments that can adapt to real world driving data.

Virtual to real reinforcement learning for autonomous driving was initially described in an arXiv technical report.

This repo contains a Torch implementation of our virtual to real reinforcement learning framework. The code contains two main parts: TORCS car racing simulator and image translation network. Stay tuned for more updates.

Citing Virtual to Real Reinforcement Learning for Autonomous Driving

If you find this work or our implementation useful in your research, please cite:

@article{you2017virtual,
    title={Virtual to Real Reinforcement Learning for Autonomous Driving},
    author={You, Yurong and Pan, Xinlei and Wang, Ziyan and Lu, Cewu},
    journal={arXiv preprint arXiv:1704.03952},
    year={2017}
}

Installation of TORCS simulation environments

The TORCS car racing simulator can be installed to collect images, their segmentations. It can also be installed as a reinforcement learning training environment.

For detailed instructions, see here for more.

Install TORCS to collect virtual images

Follow the directions in README.md to install TORCS.

Install TORCS to collect virtual images' segmentation

Replace folder TORCS/torcs-1.3.6/data/tracks/e-track-1 with data/TORCS/e-track-1; replace folder TORCS/torcs-1.3.6/ data/tracks/road/g-track-2 with data/TORCS/g-track-2. Remove previous installed TORCS and reinstall TORCS following README.md.

Training virtual-to-seg and seg-to-real networks

Example pictures of virtual-to-seg and seg-to-real translation has been put in data/examples/seg-to-real.png.

To train segmentation to realistic image translation network, download the segmented data from here and here. Then follow pix2pix network to train the seg-to-real network.

We used the segnet from here to train and segment driving images. For image translation, we use the code from Pix2Pix, so we don't include again here.