Multi-agent extension?

[kevinstan]

Hi, is there an easy way to extend this environment to support the multi-agent setting? It seems like the MiniGridEnv class assumes that there is only one agent in the environment. Would it be possibly to have a wrapper that first instantiates a single grid environment, then calls the methods in MiniGridEnv only with respect to a particular agent?

I'm trying to get inspiration from Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments, but I'm having trouble seeing exactly how and where to extend MiniGrid.

A naive approach would be simply to copy paste the relevant methods, e.g. have dir_vec1 and dir_vec2, right_vec1 and right_vec2, etc... However some methods like reset, _reward, _rand_int, steps_remaining need not be copied.

What would be an easy and elegant way to go about extending to multiple (2 or more) agents?

Best,
Kevin

[maximecb]

MiniGrid distinguishes between the agent and the other objects in the environment, and in retrospect I think that this may have been a design mistake. I think that a better design choice would be to treat agents like other objects in the world, using the same class hierarchy, and maybe to keep a list of agents in the MiniGrid class. Then in the step method you may want to specify which agent you are stepping. In general, I feel like OpenAI Gym kind of assumes a single agent setup though.

Not sure we should make this refactoring to MiniGrid itself because I'm concerned about breaking existing code. It's unfortunate because there are a few things I would change if this wasn't an issue. The best solution may be for you to fork this repo and make the modifications you want. Maybe call it something like MultiGrid ;)

[ArnaudFickinger]

https://github.com/ArnaudFickinger/gym-multigrid

Here you go ;)

[maximecb]

@ArnaudFickinger Nice work, many people have been asking for this.

Just one nitpick: since you based your environment on MiniGrid, you should copy over the open source license: https://github.com/maximecb/gym-minigrid/blob/master/LICENSE