Input position and image into DQN policy

[CoolCoder54323]

Hi, I am currently trying to create an AI for a drone for my observation space I would like to add one box space for the image, and one for position and velocity. Is there any way I would be able to do this or is there a way I can combine them into one box space?

[Miffyli]

Duplicate of #133

You can use this trick do feed the 1D data as part of the image. Alternatively stable-baselines3 has almost-done PR that will add support for dictionary observations, which you could try out using.

Closing as duplicate of #133.

[CoolCoder54323]

@Miffyli I installed the forked version of the stable baselines as you mentioned, but I still received an error:

It appears that the VecTransposeImage only works with environments that have single box spaces. I have two box inputs but one is not an image and I'm not sure needs to be transposed.
Is there any solution for this?
Should I try to change the class to only take in single observations, or will that be added in the full release?
also if i do that will it hinder the training of my agent in any way. (I don't really know why it needs to be transposed in the first place)
My knowledge of this is limited sorry if I am asking anything stupid

[Miffyli]

@CoolCoder54323

The PR is still work-in-progress so issues like this still persist. In the current form you need to change the other "box" observation to be either 1D or a 3D array, where the latter needs to have either 1, 3 or 4 channels (this is the current limitation in the PR). I can not guarantee when this would be fixed, but I try to contribute to it the next week.

If that is not possible, you can try the trick I linked above where you combine the two images into one, and then separate them inside the tensorflow code.

[CoolCoder54323]

Ok I will try the trick thank, you for your help.

[CoolCoder54323]

@Miffyli
I have a np.array for my depth image with the dimensions [88,88,1] do you know how I would add the 1d data to this?

[araffin]

@Miffyli I installed the forked version of the stable baselines as you mentioned, but I still received an error:

It appears that the VecTransposeImage only works with environments that have single box spaces. I have two box inputs but one is not an image and I'm not sure needs to be transposed.
Is there any solution for this?
Should I try to change the class to only take in single observations, or will that be added in the full release?
also if i do that will it hinder the training of my agent in any way. (I don't really know why it needs to be transposed in the first place)
My knowledge of this is limited sorry if I am asking anything stupid

so the real issue here is that you need to have channel first images (we are planning to support channel last too, hence the WIP)
you have a full example here: https://github.com/DLR-RM/stable-baselines3/pull/243/files#diff-56297865c9e2f36cfbb1b907fe7dea14b65054a680bf2916509b73c1119c54f0

[CoolCoder54323]

@araffin
I made the image in my dictionary a channel first image, and it gave the same error I believe the function VecTransposeImage only works with single box image environments. Do you think I could just ignore this function or is it critical for the policy, I was told it was needed to allow stable baselines to make frame observations(I don't exactly know what that means). The other thing in my dict is a 2x3 box space representing position and velocity fyi.

[CoolCoder54323]

@araffin Sorry to ping you again but I am kind of in a hurry as I am doing this for a school project that is due soon. I would really appreciate an answer, again sorry for double pinging you.

[araffin]

@araffin Sorry to ping you again but I am kind of in a hurry as I am doing this for a school project that is due soon. I would really appreciate an answer, again sorry for double pinging you.

Please take a close look at the working example I linked.

[CoolCoder54323]

@araffin Ok I took a closer look at what you linked me and was able to din f what I needed, for the most part, but I have one more problem. I cannot figure out how to return a box space as it requests

I am inputting two NumPy arrays of the appropriate size and within the right bounds and it is still giving this error even though this is what I inputted before when it was just one box space. I even tried inputting box.sample like this:

but that still didnt work.

[Miffyli]

Look at the error message: it should contain gym.Spaces objects but you are sampling observations into there. This kind of tech support is not what we offer here, so in future such questions will go unanswered.

[CoolCoder54323]

@Miffyli where can I get some tech support for this, it seems not many people online know about this

[Miffyli]

@CoolCoder54323 Your best bet could be to ask on RL Discord, or alternatively on stackoverflow.

[CoolCoder54323]

Ok thank you for your time.