How to use Class FullyObsWrapper in 'wrappers.py'

[wanghuimu]

Hi:
It's really a great projet! After reading the realted issues about the FullyObs, i want to try it, but i don't know how to use the Class FullyObsWrapper in the files--'wrappers.py' . Should I import it or change the code of funciton 'gen_obs(self)' in the files--'minigrid.py' ?
Thank you!

[d3sm0]

Hey something like:

import gym
import gym_minigrid
from gym_minigrid.wrappers import  FullyObsWrapper

env = gym.make(env_id)
env = FullyObsWrapper(env)
state = env.reset()

It would be great if you could post the method and result of the experiment! :)

[wanghuimu]

Thanks for you help! Now I am trying to do the experiment with the FullyObsWrapper, but it converge slower than the method with Partial-Obs, may be I should adjust the hyperparameters:)

[maximecb]

What kind of frame rate are you getting with the FullyObsWrapper? Is it converging slower in terms of number of frames or in terms of wall clock time?

[wanghuimu]

env = FullyObsWrapper(env)

[d3sm0]

According to #26 the high value in the agent position arms learning of the features.
@maximecb Shall we fix this with using the self.env.grid_size instead of 255?

https://github.com/maximecb/gym-minigrid/blob/a6678d060dfa3afa7f572e9528f99a3d1fc3b356/gym_minigrid/wrappers.py#L108

[maximecb]

Hm, I wouldn't use the grid size because this could have the same id as a valid object. We should instead add an object type for "agent" and use that id.

[wanghuimu]

The frame I set is 128 per process, and it convege slower in the real time, with particallyObs, it convege in 5 mins, but with the FullyObs, it converge in 8 mins. the code I used for traning is : python3 -m scripts.train --algo ppo --env MiniGrid-Empty-8x8-v0 --model PPO --save-interval 100 --frames-per-proc 128

Did I have some misunderstanding for the FullyObsWrapper? After we import the FullyObsWrapper, should we change the codes of ‘step()’ or ‘gen_obs(self)’ in minigrid.py? Because I think that the obs we get in ‘step()’ is still the particallyObs (the image we get in gen_obs is based on the sub-grid observed by the agent --- image = grid.encode(vis_mask))

Can I get the same result of observations with the method FullyObsWrapper if I change these codes?
class MiniGridEnv(gym.Env):
def init():
self.observation_space = spaces.Box(
low=0,
high=255 →→→ high = grid_size
shape=OBS_ARRAY_SIZE, →→→ shape = [grid_size, grid_size,3]
dtype='uint8'
)

   def gen_obs(self):
      image = grid.encode(vis_mask)        
                         ↓
                         ↓
      image = self.Grid(self.grid_size,self.grid_size).encode()                                          
      image[self.agent_pos[0]][self.agent_pos[1]] = np.array([255, self.agent_dir, 0])

[maximecb]

You don't have to change any code. I think a change from 5 minutes to 8 minutes is pretty normal. It probably takes longer to generate and process the larger images produced by the fully observable wrapper.

[wanghuimu]

You don't have to change any code. I think a change from 5 minutes to 8 minutes is pretty normal. It probably takes longer to generate and process the larger images produced by the fully observable wrapper.

D'accord~I still have a question: after we import FullyObsWrapper, the shape of obs that we get from 'step()' is FullyObs or partiallyObs related to the agent view size?
In my opinion, I think that the obs got from 'step()' generate from function 'gen_obs()', and 'gen_obs()' generate the agent's view(it's still the partiallyObs). I am sorry to ask the question because i am not familiar with python~~sorry

[maximecb]

If you use FullyObsWrapper then the obs returned by step() will have the shape (grid_width, grid_height, 3).

[PriyeshV]

When I run the following code,

import gymnasium as gym
from minigrid.wrappers import FullyObsWrapper, FlatObsWrapper

env = gym.make("MiniGrid-Empty-5x5-v0", render_mode="human")
env = FullyObsWrapper(env)
obs, _ = env.reset()

for _ in range(100):
   action = env.action_space.sample()  # User-defined policy function
   observation, reward, terminated, truncated, info = env.step(action)

   if terminated or truncated:
      observation, info = env.reset()
env.close()

The image I see is still that of the agent's (partial) view

[pseudo-rnd-thoughts]

I'm guessing that the render function will use the base environment agent's view rather than the modified wrapper's agent view
@BolunDai0216 Is there an easy change we can make to fix this?

[BolunDai0216]

Maybe we can turn off the highlight?

[pseudo-rnd-thoughts]

The problem is that the rendering happens in the base environment and ignores wrappers (mainly because the environment cannot "see" them). Otherwise, the wrapper modifying the base environment would work but this will affect other wrappers that use the render observation

[PriyeshV]

In this case, having a render wrapper that corrects this would be nice. Do you think that it is possible to custom-create one?

[BolunDai0216]

Being able to change all rendering configurations would require many changes to the codebase, but for this specific case, turning off the highlight is very easy to do, we just need to set self.highlight = False in the wrapper.

@PriyeshV can you explain a bit more on what your expected behavior would be?

[PriyeshV]

HI @BolunDai0216, Thanks for your response.

• I tried setting self.highlight = False in the __init__() of class FullyObsWrapper but it didn't change anything. Am I doing it right?
• I want to learn to solve the maze problem with images as states and that too in the FullyObservable setting. Hence, I was looking for a way to remove the highlighting.

[BolunDai0216]

From my end, if you change __init__() in FullyObsWrapper to

def __init__(self, env):
    super().__init__(env)
    self.unwrapped.highlight = False

    new_image_space = spaces.Box(
        low=0,
        high=255,
        shape=(self.env.width, self.env.height, 3),  # number of cells
        dtype="uint8",
    )

    self.observation_space = spaces.Dict(
        {**self.observation_space.spaces, "image": new_image_space}
    )

it gets rid of the highlight. Let me know if this works for you. Another alternative is to simply set highlight=False when passing the env to the wrapper

env = gym.make("MiniGrid-Empty-5x5-v0", render_mode="human", highlight=False)
env = FullyObsWrapper(env)

Let me know if any of these work for you.

[PriyeshV]

I tried the latter, and that helped. Thanks a lot :)