Gym API Compliance

[ludgerpaehler]

The Gym API has just released its latest specifications, which interestingly for us contain breaking changes. They commit to maintaining all features from this point onward, but it would possibly make sense to have a look and see that we do not rely on any of the features which are affected by the breaking changes.

Here are the release notes:

https://github.com/openai/gym/releases/tag/0.26.0

Especially the step termination/truncation affects us directly probably.

[jcallaham]

Hmm, the Ray spec has "gym>=0.21.0,<0.24.0", which seems to resolve to 0.23.1 based on our current requirements.txt. Should we just go with the API expected by Ray for the time being? I assume they will eventually update as well though, so it'd be good to keep in mind either way.