Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: PPO doesn't work correctly with MultiDiscrete action spaces with "start" parameter #1718

Closed
5 tasks done
AlejandroCN7 opened this issue Oct 18, 2023 · 1 comment · Fixed by #1845
Closed
5 tasks done
Labels
custom gym env Issue related to Custom Gym Env documentation Improvements or additions to documentation help wanted Help from contributors is welcomed

Comments

@AlejandroCN7
Copy link

AlejandroCN7 commented Oct 18, 2023

🐛 Bug

Hello!

I am trying to run the PPO algorithm with one of the environments we have created in Sinergym.

The point is that I have defined a MultiDiscrete action space (which according to the documentation is compatible), but the actions performed do not take into account the "start" parameter of the space definition.

As can be seen in the Traceback, the last action variable should be an integer value between 25 and 35, but it takes the values from 0 to 10.

I do not include test code in order not to increase the complexity of the problem, since I am using Sinergym as I have commented. The problem is simpler, and it can be seen in Traceback.

Code example

No response

Relevant log output / Error message

[ENVIRONMENT] (WARNING) : Step: The action [1 0 0 1 1 1] is not correct for the Action Space MultiDiscrete([ 2  2  2  2  2 11], start=[ 0  0  0  0  0 25])
[ENVIRONMENT] (WARNING) : Step: The action [1 1 1 0 0 5] is not correct for the Action Space MultiDiscrete([ 2  2  2  2  2 11], start=[ 0  0  0  0  0 25])
[ENVIRONMENT] (WARNING) : Step: The action [0 0 1 1 0 9] is not correct for the Action Space MultiDiscrete([ 2  2  2  2  2 11], start=[ 0  0  0  0  0 25])
[ENVIRONMENT] (WARNING) : Step: The action [1 1 1 0 1 8] is not correct for the Action Space MultiDiscrete([ 2  2  2  2  2 11], start=[ 0  0  0  0  0 25])
[ENVIRONMENT] (WARNING) : Step: The action [0 1 0 1 1 8] is not correct for the Action Space MultiDiscrete([ 2  2  2  2  2 11], start=[ 0  0  0  0  0 25])
[ENVIRONMENT] (WARNING) : Step: The action [0 0 0 1 0 2] is not correct for the Action Space MultiDiscrete([ 2  2  2  2  2 11], start=[ 0  0  0  0  0 25])
[ENVIRONMENT] (WARNING) : Step: The action [0 0 1 1 0 3] is not correct for the Action Space MultiDiscrete([ 2  2  2  2  2 11], start=[ 0  0  0  0  0 25])
[ENVIRONMENT] (WARNING) : Step: The action [0 1 0 0 0 7] is not correct for the Action Space MultiDiscrete([ 2  2  2  2  2 11], start=[ 0  0  0  0  0 25])
[ENVIRONMENT] (WARNING) : Step: The action [0 1 0 0 1 3] is not correct for the Action Space MultiDiscrete([ 2  2  2  2  2 11], start=[ 0  0  0  0  0 25])
...

System Info

  • SB3 intalled by pip (stable-baselines3==2.0.0).
  • Python 3.10.6
  • torch==2.0.1
  • gymnasium==0.29.1

Checklist

@AlejandroCN7 AlejandroCN7 added the custom gym env Issue related to Custom Gym Env label Oct 18, 2023
@AlejandroCN7 AlejandroCN7 changed the title [Bug]: PPO doen't work correctly with MultiDiscrete action spaces with "start" parameter [Bug]: PPO doesn't work correctly with MultiDiscrete action spaces with "start" parameter Oct 18, 2023
@araffin araffin added more information needed Please fill the issue template completely check the checklist You have checked the required items in the checklist but you didn't do what is written... labels Oct 18, 2023
@araffin
Copy link
Member

araffin commented Oct 18, 2023

Probably similar to #1295, we need to update the env checker

edit: correct issue is #913 (comment)

@araffin araffin added help wanted Help from contributors is welcomed documentation Improvements or additions to documentation and removed more information needed Please fill the issue template completely check the checklist You have checked the required items in the checklist but you didn't do what is written... labels Oct 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
custom gym env Issue related to Custom Gym Env documentation Improvements or additions to documentation help wanted Help from contributors is welcomed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants