Incorrect reward on end of episode without error

[bwitherspoon]

Environment

• Grid2op version: 1.4.0
• System: Fedora 32

Bug description

According to the documentation for the Reward object, if has_error is false and is_done is true this should indicate the end of an episode without error (i.e. success). However, it appears that is_done is not being set on the final step. Maybe related to #146.

How to reproduce

Code snippet

import grid2op
from grid2op.Reward import BaseReward
from grid2op.dtypes import dt_float
from grid2op.Exceptions import DivergingPowerFlow

class Reward(BaseReward):
    def __init__(self):
        super().__init__()
        self.reward_min = dt_float(-1.0)
        self.reward_max = dt_float(1.0)

    def __call__(self, action, env, has_error, is_done, is_illegal, is_ambiguous):
        if has_error:
            return self.reward_min
        elif is_done:
            return self.reward_max

        return dt_float(0.0)

env = grid2op.make("l2rpn_case14_sandbox", reward_class=Reward)
max_timestep = env.chronics_handler.max_timestep()

obs = env.reset()
env.fast_forward_chronics(max_timestep - 100)
obs = env.get_obs()

while True:
    obs, reward, done, info = env.step(env.action_space())

    for ex in info["exception"]:
        print(ex)

    if done:
        print("reward:", reward)
        break

Current output

reward: 0.0

Expected output

reward: 1.0

[BDonnot]

Hello,

I could reproduce this bug. Thanks a lot for the detail explanation and the code snippet :-)

I will push the fix in the next release of grid2op (probably before the end of January 2021, but i'm not sure).

Thanks

Benjamin

[BDonnot]

Fixed in release 1.5.0