Add sticky actions for Atari games

[qgallouedec]

Description

See:
#635 (comment)

Whether this default setting will change in the future is still an open question.

This is the implementation that uses the environment argument repeat_action_probability. But I find it confusing: why would repeat_action_probability be an argument to make_atari_env and not to AtariWrapper? I am open to change this implementation, but it requires adding a new wrapper RepeatActionWrapper.

Motivation and Context

• I have raised an issue to propose this change (required for new features and bug fixes)

closes #271

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)

Checklist

• I've read the CONTRIBUTION guide (required)
• I have updated the changelog accordingly (required).
• My change requires a change to the documentation.
• I have updated the tests accordingly (required for a bug fix or a new feature).
• I have updated the documentation accordingly.
• I have opened an associated PR on the SB3-Contrib repository (if necessary)
• I have opened an associated PR on the RL-Zoo3 repository (if necessary)
• I have reformatted the code using make format (required)
• I have checked the codestyle using make check-codestyle and make lint (required)
• I have ensured make pytest and make type both pass. (required)
• I have checked that the documentation builds using make doc (required)

Note: You can run most of the checks using make commit-checks.

Note: we are using a maximum length of 127 characters per line

[qgallouedec]

I finally opted for the natural way from a user point of view.

The implementation is strictly equivalent to that of ALE, namely:

• On reset, the previous action is initialized to NOOP:

https://github.com/mgbellemare/Arcade-Learning-Environment/blob/49154a3e96f0858e5e9a1f41c89cfe18e6741468/src/environment/stella_environment.cpp#L194

• On step, the previous action is repeated with a probability of action_repeat_probability:

https://github.com/mgbellemare/Arcade-Learning-Environment/blob/49154a3e96f0858e5e9a1f41c89cfe18e6741468/src/environment/stella_environment.cpp#L166

[qgallouedec]

Last thing to clarify: should I put the new arg at the end of the arg list?

[araffin]

Last thing to clarify: should I put the new arg at the end of the arg list?

I would say so, to avoid any bad surprises.

[araffin]

maybe skip that one if frame_skip<=1 (need to check if it is <=0 or <=1, but if I recall, it should have been called action repeat.

[araffin]

apart from minor comment, LGTM =)