Missing observations in runner logs after running RandomRedispatchingAgent

[marota]

Environment

• Grid2op version: 1.1.1
• System: osx

Bug description

When running the RandomRedispatchingAgent on 100 timesteps, the runner runs until the end of the scenario according to the progression bar.
But when reloading the logs of the runner, we only access 35 timesteps of observations

How to reproduce

import grid2op
from grid2op.Agent import RandomRedispatchAgent#, DoNothingAgent
from grid2op.Runner import Runner
from grid2op import make
from grid2op.Episode import EpisodeData
import os

path_agent_saved='/tmp/redispatch-bug'
os.makedirs(path_agent_saved,exist_ok = True)

#run redispatch agent on one scenario for 100 timesteps
dataset = "rte_case14_realistic"
nb_episode=1
nb_timesteps=100

with make(dataset) as env:
  agent = RandomRedispatchAgent(env.action_space)
  runner = Runner(**env.get_params_for_runner(),
                  agentClass=None,
                  agentInstance=agent)
  nb_episode=1
  runner.run(nb_episode=nb_episode,
             path_save=path_agent_saved,
             nb_process=1,
             max_iter=nb_timesteps,
             env_seeds=[0]*nb_episode,
             agent_seeds=[0]*nb_episode,
             pbar=True)
  env.close()

#NOTICE: the run goes until the end of the 100 timesteps

#reload the observations and check the size of the observation
#when doing that, the size was 35 instead of 100 that we expected

episode_data = EpisodeData.from_disk(path_agent_saved, '000')
assert(len(episode_data.observations)==nb_timesteps)

[Tezirg]

Update: The current behavior sets the game over to be at the first ambiguous action encountered when reloading, even if logs data keeps going. In that case, it happens at index 34, but data on disk is correct with a 100 timesteps.

For some reason, some env_actions are ambiguous..

import grid2op
from grid2op.Agent import DeltaRedispatchRandomAgent
from grid2op.Runner import Runner
from grid2op import make
from grid2op.Episode import EpisodeData
import os
import numpy as np

path_agent_saved='/tmp/redispatch-bug'
os.makedirs(path_agent_saved,exist_ok = True)

#run redispatch agent on one scenario for 100 timesteps
dataset = "rte_case14_realistic"
nb_episode=1
nb_timesteps=100

with make(dataset) as env:
  agent = DeltaRedispatchRandomAgent(env.action_space)
  runner = Runner(**env.get_params_for_runner(),
                  agentClass=None,
                  agentInstance=agent)
  nb_episode=1
  res = runner.run(nb_episode=nb_episode,
                   path_save=path_agent_saved,
                   nb_process=1,
                   max_iter=nb_timesteps,
                   env_seeds=[0],
                   agent_seeds=[0],
                   pbar=True)
  print ("{}/{} steps".format(res[0][3], res[0][4]))

# Load with ambiguous actions
episode_data = EpisodeData.from_disk(path_agent_saved, '000')

# They are nb_timesteps objects
print ("EP objs actions len", len(episode_data.actions.objects))
print ("EP objs obs len", len(episode_data.observations.objects))
# But limited lens due to gameover encountered by ambiguous env action at t34
print ("EP obs len", len(episode_data.observations))
print ("EP actions len", len(episode_data.actions))


# Hack to bypass ambiguous actions gameover
grid2op.Action.BaseAction.check_space_legit = grid2op.Space.GridObjects.check_space_legit
# Reload
episode_data = EpisodeData.from_disk(path_agent_saved, '000')

# They are nb_timesteps objects
print ("bypass EP objs actions len", len(episode_data.actions.objects))
print ("bypass EP objs obs len", len(episode_data.observations.objects))

# With bypass, lens are correct
print ("bypass EP obs len", len(episode_data.observations))
print ("bypass EP actions len", len(episode_data.actions))

# Show where the problems is
print ("Env ramps up", episode_data.actions.helper.gen_max_ramp_up)
print ("Env ramps down", episode_data.actions.helper.gen_max_ramp_down)
print ("Env action index 34:")
print (episode_data.env_actions[34])
print ("Env action index 34 ambiguous:")
print (episode_data.env_actions[34].is_ambiguous())

Outputs:

100/100 steps
EP objs actions len 100
EP objs obs len 101
EP obs len 35
EP actions len 34
bypass EP objs actions len 100
bypass EP objs obs len 101
bypass EP obs len 101
bypass EP actions len 100
Env ramps up [ 5. 10.  0.  0. 10.]
Env ramps down [ 5. 10.  0.  0. 10.]
Env action index 34:
This action will:
	 - set load_p to [23.1 78.3 40.8  6.  11.5 25.8  8.4  3.2  4.9 10.3 14.3]
	 - set prod_p to [73.2 70.4 14.3  0.  70. ]
	 - set load_q to [16.3 55.  28.7  4.1  7.9 17.8  5.9  2.2  3.4  6.9  9.9]
	 - Redispatch gen_1_0 of -4.501743793487549
	 - Redispatch gen_2_1 of -6.000365734100342
	 - Redispatch gen_0_4 of 10.502110481262207
	 - NOT force any line status
	 - NOT switch any line status
	 - NOT switch anything in the topology
	 - NOT force any particular bus configuration
Env action index 34 ambiguous:
(True, Grid2OpException AmbiguousAction InvalidRedispatching InvalidRedispatching('Some redispatching amount are above the maximum ramp up'))

Here we can see that on reload, env actions are doing redispatching (not expected), and above ramps (even worst) !!!

[BDonnot]

This is probably due to the data that does not respect the ramps or pmin / pmax. Thus the environment is forced to take actions, that are ambiguous (and not checked they are because grid2op supposes the input data are consistent, which apparently they are not)