[Question] Is there support for MARL?

[smarianimore]

As per title: is there support for multi-agent learning environments, such as PettingZoo for instance?

[Toni-SM]

Hi @smarianimore

It is a work in progress for the next release...

I programmed the environment wappers for PettingZoo and the DexterousHands (bi-dexhands: a multi-agent environment on top of Isaac Gym). Also, I have a first implementation of IPPO and MAPPO...

If you are interested in collaboration, I can create a separate branch for this feature... and we can work on it.
Let me know if it is the case

[smarianimore]

I would be glad to have the time, but as a full time researcher it is difficult...will definitely let you know if I manage to find a way

[ranzuh]

I'm thinking of using SKRL to implement MARL training in my custom Omni-Isaac environment. If I understand IPPO correctly it is quite straightforward to do MARL with the regular PPO if parameter sharing is utilized. Next, I want to do it without parameter sharing (separate models for each policy). I think it should be pretty easy to implement with the nice modularity of skrl.

[Toni-SM]

Hi @ranzuh

I have just pushed, to the multi-agent branch, the IPPO and MAPPO and the wrappers for PettingZoo and Bi-Dexhands.

Also, I enabled the documentation for this branch (https://skrl.readthedocs.io/en/multi-agent/)
Relevant links in docs for multi-agents applications are:

• wrapper (multi-agent envs): https://skrl.readthedocs.io/en/multi-agent/api/envs/multi_agents_wrapping.html

• multi-agent algorithms: https://skrl.readthedocs.io/en/multi-agent/api/multi_agents.html

  Note: I need to check MAPPO implementation to be sure is right

---

In the .zip you can find a script for IPPO application for the Bi-Dexhands ShadowHand Over environment that does not share parameters.
I hope this helps with your implementation :)
ippo_bidexhands_shadow_hand_over.zip