[RLlib] TypeError with latest Pettingzoo version (unexpected keyword argument 'return_info')

[george-skal]

What happened + What you expected to happen

Hi, I am running a parameter sharing waterworld example and also MPE examples with latest Pettingzoo and Supersuit and I cannot use older versions, since there are bug fixes on Simple Spread. I get the the same TypeError on all. On windows:

TypeError: reset() got an unexpected keyword argument 'return_info'

and on Ubuntu:

TypeError: OrderEnforcingWrapper.reset() got an unexpected keyword argument 'return_info'

Please find attched the error file.
error.txt

Thanks,
George

Versions / Dependencies

Ray 2.4.0
Pettingzoo 1.23.0
Supersuit 3.8.0
Pytorch 1.13.1

Reproduction script

from ray import air, tune
from ray.rllib.algorithms.ppo import PPOConfig
from ray.tune.registry import register_env
from ray.rllib.env.wrappers.pettingzoo_env import PettingZooEnv
from pettingzoo.sisl import waterworld_v4

if __name__ == "__main__":
    def env_creator(args):
        return PettingZooEnv(waterworld_v4.env(n_pursuers=5, n_evaders=5, n_coop=2))


    env = env_creator({})
    register_env("waterworld", env_creator)

    obs_space = env.observation_space
    act_spc = env.action_space

    policies = {"shared_policy_1": (None, obs_space, act_spc, {})
                }


    def policy_mapping_fn(agent_id, episode, worker, **kwargs):
        return "shared_policy_1"


    config = (
        PPOConfig()
            .environment("waterworld")
            .resources(num_gpus=1, num_cpus_for_local_worker=2)
            .rollouts(num_rollout_workers=4)  # default = 2 (I should try it)
            .framework("torch")
            .multi_agent(
            policies=policies,
            policy_mapping_fn=policy_mapping_fn,
        )
    )

    tune.Tuner(
        "PPO",
        run_config=air.RunConfig(
            name="PPO shared waterworld_v4 ncoop=2 test trial 1",
            stop={"training_iteration": 1500},
            checkpoint_config=air.CheckpointConfig(
                checkpoint_frequency=10,
            ),
        ),
        param_space=config.to_dict(),
    ).fit()

Issue Severity

High: It blocks me from completing my task.

[Rohan138]

Currently RLlib is pinned to gymnasium==0.26.3 and is only compatible to the corresponding pettingzoo and supersuit versions; this will be fixed by #34696. Duplicate of #32889