README.md

OpenAI Gym experiments

My implementations of normalized advantage functions (NAF) for continuous actions spaces and dueling network architecture (DUEL) for discrete action spaces.

Example results with NAF:

• InvertedPendulum-v1
• Pendulum-v0

Example results with DUEL:

• CartPole-v0
• MountainCar-v0
• Acrobot-v0

Prerequisites

You will need:

• Python 2.7
• OpenAI Gym
• Keras
• Numpy
• Scikit-Learn (if using imagination rollouts)

In Ubuntu that would be:

sudo apt-get install python-numpy python-sklearn
pip install --user gym keras

If you want to run Mujoco environments, you also need to acquire trial key and install the binaries. Then you can install Mujoco support for OpenAI Gym:

pip install --user gym[mujoco]

Running the code

There are three main starting points:

• python duel.py <envid> - run DUEL against environment with discrete action space,
• python naf.py <envid> - run NAF against environment with continuous action space,
• python nag_ir.py <envid> - run NAF with imagination rollouts.

You can override default hyperparameters with command-line options, use -h to see them or check out the code.

Some other utility scipts:

• python test.py <envid> - test script to run random actions against the environment,
• python naf_search.sh - example how to run crude hyperparameter search for NAF,
• python duel_search.sh - example how to run crude hyperparameter search for DUEL.