PPO Breakout Score

[eaplatanios]

I tried running the PPO2 example using the breakout game on my MacBook, without modifying anything in the scripts or configurations and I am only able to get up to 19.6 score. Why is that? Is there an implementation bug or do I need to tune PPO differently on my machine to get up to 400?

[DanielTakeshi]

You might need to be more specific in your request. See for example this issue: #938 When you are able to get to 19.6 score, how many environment steps does that correspond to?

[eaplatanios]

@DanielTakeshi sorry for not clarifying. I am executing the provided run.py script directly with default arguments. This sets nenv = 6 based on my CPU and runs for 10^6 steps. I haven't modified anything so running that script and explicitly setting nenv = 6 should give you the same result. The command I use to run the experiment is:

python3.7 -m baselines.run --alg=ppo2 --env=BreakoutNoFrameskip-v4

[christopherhesse]

Looking at the published graphs: http://htmlpreview.github.io/?https://github.com/openai/baselines/blob/master/benchmarks_atari10M.htm breakout doesn't get to 400 until 1e7 steps, have you tried training that long?