Some issues on Observation and Action

[sunghoonhong]

I played with “case5_example” environment, and I found some cases that I can not understand.

Firstly, observation dimension is not same as Grid2Op documentation. The documentation says there are 31 vectors that consist Completeobservation (1 the year, 2 the month, 3 the day …. 31 actual_dispatch). Howerver when I type “env.observation_space.shape”, there are 32 vectors. I think there is a strange vector between 25(time_before_cooldown_line) and 26 (time_before_cooldown_sub), because there is a vector of size #powerlines between them, which is not mentioned in the documentation.

Secondly, an action that assign origin and extremity to bus 2 at the same time always gives game over. Initially, all the components in a grid are connected to bus 1. Therefore, for example, if I command :

action_space.reconnect_powerline(line_id=5,bus_or=2,bus_ex=2),

it makes the line 5 being isolated, and I thought it is same as disconnecting the line 5, since electricity does not flow through the line 5. However, it gives game over. I tried line_id 1 to 7 but all results were same.

Furthermore, if I disconnect the line 5 first, and then reconnect origin and extremity of the line 5 to bus 2, it works. Is this what you intended?

Finally, what is exactly Observation.time_before_cooldown_line and Observation.time_before_cooldown_sub ? Those values do not change all the time. If I disconnect the line, or the line is disconnected because of overflow, those value always give zeros.

[BDonnot]

Hello,

First, thanks for noticing the mismatch between the documentation and the code. This will be fixed in the next version of grid2op.

To know of what is composed the action / observation space, you can use the attr_list_vect member of each of them, for example:

import grid2op
env = grid2op.make()
obs = env.reset()
print(obs.attr_list_vect)

This should return something like

['year', 'month', 'day', 'hour_of_day', 'minute_of_hour', 'day_of_week', 'prod_p', 'prod_q', 'prod_v', 'load_p', 'load_q', 'load_v', 'p_or', 'q_or', 'v_or', 'a_or', 'p_ex', 'q_ex', 'v_ex', 'a_ex', 'rho', 'line_status', 'timestep_overflow', 'topo_vect', 'time_before_cooldown_line', 'time_before_cooldown_line', 'time_before_cooldown_sub', 'time_before_line_reconnectable', 'time_next_maintenance', 'duration_next_maintenance', 'target_dispatch', 'actual_dispatch']

And as you noticed, the "time before cooldown line" is present twice. I fixed that for next release.

I will look further at the other issues you pointed out, trying to reproduce the bug with the line set on bus 2 at both its ends. I think this is normal this lead to a game over. When you set up a powerline isolated on a bus, it creates a nodes. When you do it on both its extremity, you create an isolated "grid" with 2 buses: one for each end of the powerline. This is then perfectly normal you get a game over in this case.
However, if you reconnect a powerline on bus 2 after having disconnected it, the same behaviour (game over) should be expected. I'll look into that. Thanks for noticing.

The "cool down" depends on the rules of the game. In l2rpn 2019, to mimic operational constraints, you could not reconnect a powerline after having disconnected it at the previous time step (and vice versa) and you could not reconfigure a substation on which you just acted. Depending on what were your actions, this behaviour might be normal, but i'll double that too.

Thanks for your valuable comments

Benjamin

[BDonnot]

Hi again,

I had a look at the "issue" with "time before cooldown line" and "time before cooldown sub", which were always zero (and not moving). It is because, in the default parameters, you have these values:

        # number of timestep before a substation topology can be modified again
        self.NB_TIMESTEP_TOPOLOGY_REMODIF = 0
        self.NB_TIMESTEP_LINE_STATUS_REMODIF = 0

So actually, in the parameters, you don't prevent to reconnect a powerline, or to modify again a substation when you just act on it. This is why it always says you can act on these elements (by being always 0).
The documentation should better explain this. I agree :-)

I'll try now to reproduce the behaviour you noted on the case5_example.

[BDonnot]

Hello again Sunghoon,

I am not able to reproduce the bug with the powerlines you point out in your example.

Secondly, an action that assign origin and extremity to bus 2 at the same time always gives game over. Initially, all the components in a grid are connected to bus 1. Therefore, for example, if I command :
action_space.reconnect_powerline(line_id=5,bus_or=2,bus_ex=2),
it makes the line 5 being isolated, and I thought it is same as disconnecting the line 5, since electricity does not flow through the line 5. However, it gives game over. I tried line_id 1 to 7 but all results were same.
Furthermore, if I disconnect the line 5 first, and then reconnect origin and extremity of the line 5 to bus 2, it works. Is this what you intended?

Here are what I am doing:

env = grid2op.make("case5_example")
obs = env.reset()
act = env.action_space.disconnect_powerline(line_id=5)
obs, reward, done, info = env.step(act)
act_case1 = env.action_space.reconnect_powerline(line_id=5,bus_or=2,bus_ex=2)
obs, reward, done, info = env.step(act)
print(done)

and it prints False

And i also did:

env = grid2op.make("case5_example")
obs = env.reset()
act_case2 = env.action_space.reconnect_powerline(line_id=5,bus_or=2,bus_ex=2)
obs, reward, done, info = env.step(act)
print(done)

and it prints False too.

Can you tell me a bit more what i could do to reproduce it ?

Thank you

Benjamin

[sunghoonhong]

Thanks for reply.

It is weird that it produce different output.
I exactly used the same code as yours.
However, the second code gives 'True'.
It seems that you gave wrong act to env.step().
You should give act_case2 but you gave act which is defined in the first code.

env = grid2op.make("case5_example")
obs = env.reset()
act_case2 = env.action_space.reconnect_powerline(line_id=5,bus_or=2,bus_ex=2)
obs, reward, done, info = env.step(act) # you should give act_case2 here
print(done)

I'm looking forward to hearing from you.

Best regards

[BDonnot]

Oh you are correct. I forgot to check the actions (it's really bad to do multiple things at the same time).

I fixed my code, and now i have the same behaviour as you reported. I'll get back asap.

Benjamin

[BDonnot]

Hello again,

I found out what the problem was. The pandapower backend used was not reconnecting properly the buses in the first case. So the powerline was indeed not connected. This was a bug due to presence of disconnected elements.

However, in the second case, there were no disconnected elements. So the backend properly reconnected two buses at both ends of the powerline creating de facto a second powergrid isolated from the first one leading, as you noticed, to a game over.

This issue has been fixed, and will be solved in the next release of grid2op. I also made a test for that.

Fixing this issue, i found out another one. You actually "faked" the connection of a powerline already connected, and thus by pass the rules by changing the topology of more than 1 subtation. This should not be possible by the rules, and will be fixed in the next release too :-)

To sum it up, the correct behaviour of grid2op should have been :

• in the first case (disconnection at t = 0 plus reconnection at t = 1 on bus 2 for both ends) to give a game over
• in the second case (line is connected at t = 0, you fake its reconnection at t = 1 on bus 2 on both ends, thus changing the topology of 2 substations) to lead to an illegal action and not to a game over (but to do nothing instead)

Thanks for all your checking

Benjamin

ps. If you don't have any other comment i would have missed, i will close this issue in the next release of grid2op.

[BDonnot]

As promised, I am closing this issue.

Every error reported here has been fixed in the latest release, and some unit tests have been added to make sure these bugs did not appear again.

Let me know if, unfortunately, it didn't solve all of the issues

Best

Benjamin