obs.simulate and env.step check cooldown differently

[richardwth]

Environment

• Grid2op version: 1.7.2
• System: ubuntu

Bug description

There are two types of cooldown in the game: time_before_cooldown_sub and time_before_cooldown_line. When cooldown of a sub or a line is 1, obs.simulate thinks an action on this sub or line is OK, but env.step thinks it is illegal.

How to reproduce

import grid2op
from lightsim2grid import LightSimBackend as bk_cls
from grid2op.Parameters import Parameters

# First, create env with cooldown
param = Parameters()
param.NB_TIMESTEP_COOLDOWN_SUB = 3
param.NB_TIMESTEP_COOLDOWN_LINE = 3
env = grid2op.make('l2rpn_wcci_2022', param=param, backend=bk_cls())
action_space = env.action_space
do_nothing = action_space({})

# second, get some action that has no actual effect on the grid
set_sub_10 = action_space.get_all_unitary_topologies_set(action_space, sub_id=10)
set_sub10_to_bus1 = set_sub_10[0]
# can also get some line action
# line_actions = action_space.get_all_unitary_line_set_simple(action_space)
# connect_last_line = line_actions[-1]

# last, do the action multiple times
_ = env.reset()

obs, _, done, info = env.step(set_sub10_to_bus1)
print('Step {}:'.format(obs.current_step), obs.time_before_cooldown_sub[10], info['exception'])

obs, _, done, info = env.step(do_nothing)
print('Step {}:'.format(obs.current_step), obs.time_before_cooldown_sub[10], info['exception'])
obs_, _, done, info = obs.simulate(set_sub10_to_bus1)
print('Step {} simu:'.format(obs.current_step), info['exception'])

obs, _, done, info = env.step(do_nothing)
print('Step {}:'.format(obs.current_step), obs.time_before_cooldown_sub[10], info['exception'])
obs_, _, done, info = obs.simulate(set_sub10_to_bus1)
print('Step {} simu:'.format(obs.current_step), info['exception'])

obs, _, done, info = env.step(set_sub10_to_bus1)
print('Step {}:'.format(obs.current_step), obs.time_before_cooldown_sub[10], info['exception'])

obs, _, done, info = env.step(set_sub10_to_bus1)
print('Step {}:'.format(obs.current_step), obs.time_before_cooldown_sub[10], info['exception'])

Current output

Step 1: 3 []
Step 2: 2 []
Step 2 simu: [Grid2OpException IllegalAction IllegalAction('Substation with ids [10] have been modified illegally (cooldown)')]
Step 3: 1 []
Step 3 simu: []
Step 4: 0 [Grid2OpException IllegalAction IllegalAction('Substation with ids [10] have been modified illegally (cooldown)')]
Step 5: 3 []

Expected output

I expect obs.simulate and env.step to be consistent on cooldown check of sub or line. For example, following the logic of obs.simulate, the output should be:

Step 1: 3 []
Step 2: 2 []
Step 2 simu: [Grid2OpException IllegalAction IllegalAction('Substation with ids [10] have been modified illegally (cooldown)')]
Step 3: 1 []
Step 3 simu: []
Step 4: 3 []
Step 5: 2 [Grid2OpException IllegalAction IllegalAction('Substation with ids [10] have been modified illegally (cooldown)')]

[BDonnot]

Hello,

Indeed that's not a correct behavior. I'll have a look and implement a fix for the new release (hopefully sooner rather than later)

Thanks for the issue 😊

[richardwth]

May I ask which one is the correct behavior by design, obs.simulate and env.step?

[BDonnot]

Cooldowns are carefully checked and tested for the env.step method.

Unless there is an obvious bug in env.step (which can happen of course, this software, like every other is not perfect) it should be used as the reference.

So env.step is correct in this case.

[BDonnot]

Bug should be fixed, and i clarify the doc for the correct behaviour, see https://grid2op.readthedocs.io/en/latest/rules.html#behaviour and following subsection for more information :-)