Introduction

This repo is about modelling the interaction between Autonomous Agents [AA] and Human Agents [HA] in a mixed traffic environment. We simulate various scenarios such as: selfishness vs cooperativeness in AAs, behavior of AAs with varying number of HAs and other.

If you wish to learn more about the papers and the project goal refer to project report or, for a quiker summary, refer to the Project_definiton.md.

Instructions

Download

To download the repo use

git clone --recursive https://gitlab.com/nicofirst1/mas_traffic

Setup

To get install the necessary packages use:

bash scripts/initial_config.sh

Note that the previous will install both git and anaconda. If you encounter any error regarding one of the two please install them separately and rerun the script.

Then configure the python package with (ensure that your conda environment is dmas):

python setup.py install

If you encounter an error regarding SUMO_HOME not being defined, run the appropriate sumo install script in sumo_setup with:

source ~/.bashrc

conda activate dmas

Supported OS:

• Ubuntu 18.04 LTS
• macOSX Mojave

If your OS is not supported or you encounter an error check the installation file for more detailed installation instructions.

Running

Once you are done you can run the first Tutorial with:

python FlowMas/Tutorials/1_GridMapNoLights.py

Training

For training you can use

python FlowMas/train.py {args}

The training results will be saved in the ray_result dir.

To get a list of possible arguments run

python FlowMas/train.py --help

A complete list of attributes will be then printed. Notice that each attribute is documented in the param class

NB: If you change some attributes of the Params class, but the changes does not seem to be working, use:

python setup.py install

Visualizing results

Once you have trained your agents you can use the flow visualization framework to visualize your results. Use :

tensorboard --logdir=FlowMas/data/ray_results

To use tensorboard visualization, this will display every training you have in the ray_result dir. If you rather use matplotlib you can use the following command:

python flow/visualize/plot_ray_results.py FlowMas/data/ray_results/experiment_dir/progress.csv

where experiment_dir is the directory containing thetraining instance you would like to visualize. This will show a list of possible parameter to visualize. To plot them simply add them to the command as:

python flow/visualize/plot_ray_results.py FlowMas/data/ray_results/experiment_dir/progress.csv episode_reward_max episode_reward_mean

Moreover, for visualizing the SUMO gui with your trained agent use:

python flow/visualize/visualizer_rllib.py FlowMas/data/ray_results/experiment_dir/result/directory 1

Visualizing multiple training instances

Once you have populated the ray_result dir with multiple training instances you can use the plot_results script to generate plots for different training parameters such as:

• Delays: selfish, cooperative, total
• Actions: selfish, cooperative, total
• Jerks: selfish, cooperative, total
• Rewards: selfish, cooperative, total

Usage

To use the script just run it with:

python FlowMas/utils/plot_results.py -input_dir path/to/your/dir [-output_dir custom/output/folder ]

For instance if you want to plot every training instance in your ray_result dir, use:

python FlowMas/utils/plot_results.py -input_dir FlowMas/data/ray_results

An out directory will be created in the ray_result dir containing your plots.

Repo structure

There repository is currently structured as follows:

• The flow git fork for flow framework
• FlowMas: the project's core containing the following dirs:
  • maps: a dir containing custom maps, you can follow the readme for further information.
  • Tutorials: which has an incrementing number of tutorials each being a step forward in the final implementation of the project. You can check the tutorial readme for more infos.
  • utils: contains utility scripts, check the README for more infos.
  • train.py: main script for training the environment.
  • train_osm.py: Failed attempt to work with OSM maps. If you're intrested in solving this problem please see this.
• MarkDown: a directory for useful markdowns:
  • The Error markdown file contains common errors encountered during the project development.
  • The Journal markdown file has both an useful_link section as well as a todo section in which the project'steps are enumerated.
  • The Project_definiton markdown file holds the project' specifications.
  • The Installation markdown file holds installation instruction.
• scripts: contains shell scripts mainly for installation purposes
  • The initial_configs dir holds package installation for both ubuntu18 and macosx
  • The sumo_setup dir holds sumo binary installation shell scripts
  • The initial_config.sh shell script is used in the setup.py file to start the installation process
  • The python_setup.sh shell script for setting up python environment.
  • The kill_dmas.sh script can be used to kill background processes running SUMO.