when network is deep, the training not stable

[raopku]

if i make the --layers 4, or larger( in my opinion, 4 layers is not very deep)
the performance is not stable.
Episode 259 finished after 192 timesteps, episode reward 192.0
Episode 260 finished after 76 timesteps, episode reward 76.0
Episode 261 finished after 127 timesteps, episode reward 127.0
Episode 262 finished after 26 timesteps, episode reward 26.0
Episode 263 finished after 200 timesteps, episode reward 200.0
Episode 264 finished after 200 timesteps, episode reward 200.0
Episode 265 finished after 10 timesteps, episode reward 10.0
Episode 266 finished after 200 timesteps, episode reward 200.0
Episode 267 finished after 200 timesteps, episode reward 200.0
Episode 268 finished after 34 timesteps, episode reward 34.0
Episode 269 finished after 62 timesteps, episode reward 62.0
Episode 270 finished after 113 timesteps, episode reward 113.0
Episode 271 finished after 107 timesteps, episode reward 107.0
Episode 272 finished after 119 timesteps, episode reward 119.0
Episode 273 finished after 115 timesteps, episode reward 115.0
Episode 274 finished after 54 timesteps, episode reward 54.0
Episode 275 finished after 200 timesteps, episode reward 200.0
Episode 276 finished after 170 timesteps, episode reward 170.0
Episode 277 finished after 200 timesteps, episode reward 200.0
Episode 278 finished after 150 timesteps, episode reward 150.0
Episode 279 finished after 13 timesteps, episode reward 13.0
Episode 280 finished after 153 timesteps, episode reward 153.0
Episode 281 finished after 21 timesteps, episode reward 21.0
Episode 282 finished after 94 timesteps, episode reward 94.0

why?

when --layers 1,the training is statble

Episode 218 finished after 200 timesteps, episode reward 200.0
Episode 219 finished after 200 timesteps, episode reward 200.0
Episode 220 finished after 187 timesteps, episode reward 187.0
Episode 221 finished after 200 timesteps, episode reward 200.0
Episode 222 finished after 200 timesteps, episode reward 200.0
Episode 223 finished after 200 timesteps, episode reward 200.0
Episode 224 finished after 200 timesteps, episode reward 200.0
Episode 225 finished after 200 timesteps, episode reward 200.0
Episode 226 finished after 200 timesteps, episode reward 200.0
Episode 227 finished after 200 timesteps, episode reward 200.0
Episode 228 finished after 200 timesteps, episode reward 200.0
Episode 229 finished after 200 timesteps, episode reward 200.0
Episode 230 finished after 200 timesteps, episode reward 200.0
Episode 231 finished after 200 timesteps, episode reward 200.0
Episode 232 finished after 200 timesteps, episode reward 200.0
Episode 233 finished after 200 timesteps, episode reward 200.0
Episode 234 finished after 200 timesteps, episode reward 200.0
Episode 235 finished after 200 timesteps, episode reward 200.0
Episode 236 finished after 200 timesteps, episode reward 200.0
Episode 237 finished after 200 timesteps, episode reward 200.0
Episode 238 finished after 200 timesteps, episode reward 200.0
Episode 239 finished after 200 timesteps, episode reward 200.0