[RLlib] Cleanup examples folder #16: Add missing docstrings to 2 connector example scripts.

[sven1977]

Cleanup examples folder #16: Add missing docstrings to 2 connector example scripts.

• mean/std filter and frame-stacking.
• Other major improvements of these 2 scripts.

Why are these changes needed?

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[simonsays1980]

LGTM

[simonsays1980]

Are the rest of the connectors now default?

[sven1977]

All off-the-shelf connectors now directly operate on the episodes now (except for framestacking for performance reasons):

• read last obs from episode
• alter that last obs
• directly write it back into the episode via episode.set_observations.

This way, RLlib's default pieces (which come after the user defined ones) only see the already altered episode and thus properly extract the observations from those and put them into the RLModule batch.

This way, things work more uniformly. Reading the last obs from an episode and writing it back is super fast, b/c we are still operating on simple lists in the EnvRunner (not numpy'ized yet).