Final fix before 1.5.0 let's see if tests pass

.gitignore

@@ -293,6 +293,11 @@ grid2op/data/update_envs_ls.py
 grid2op/data_save
 test_rllib2.py
 grid2op/tests/res_test
+test_bug_macos.py
+pp_error_non_connected_grid.py
+test_issue174.py
+actions2.npy
+bug_discord_0.py
 
 # profiling files
 **.prof

CHANGELOG.rst

@@ -4,7 +4,6 @@ Change Log
 [TODO]
 --------------------
 - [???] add multi agent
-- [???] model curtailment
 - [???] better logging
 - [???] shunts in observation too, for real (but what to do when backend is not shunt compliant to prevent the
   stuff to break)
@@ -70,6 +69,9 @@ Change Log
 - [FIXED] issue `Issue #173 <https://github.com/rte-france/Grid2Op/issues/173>`_: a full nan vector could be
   converted to action or observation without any issue if it had the proper dimension. This was due to a conversion
   to integer from float.
+- [FIXED] an issue preventing to load the grid2op.utils submodule when installed not in "develop" mode
+- [FIXED] some issue with the multiprocessing of the runner on windows
+- [ADDED] more complete documentation for the runner.
 - [ADDED] a convenient function to evaluate the impact (especially on topology) of an action on a state
   (`obs + act`)
 - [ADDED] a property to retrieve the thermal limits from the observation.

docs/gym.rst

@@ -1,4 +1,10 @@
+
+.. _BaseGymSpaceConverter.add_key: ./gym.html#grid2op.gym_compat.gym_space_converter._BaseGymSpaceConverter.add_key
+.. _BaseGymSpaceConverter.keep_only_attr: ./gym.html#grid2op.gym_compat.gym_space_converter._BaseGymSpaceConverter.keep_only_attr
+.. _BaseGymSpaceConverter.ignore_attr: ./gym.html#grid2op.gym_compat.gym_space_converter._BaseGymSpaceConverter.ignore_attr
+
 .. currentmodule:: grid2op.gym_compat
+
 .. _openai-gym:
 
 Compatibility with openAI gym
@@ -40,16 +46,28 @@ A simple usage is:
     print(f"Is gym_env and open AI gym environment: {isinstance(gym_env, gym.Env)}")
     # it shows "Is gym_env and open AI gym environment: True"
 
+.. note::
+
+    To be as close as grid2op as possible, by default (using the methode discribed above) the action
+    space will be encoded as a gym Dict with keys the attribute of a grid2op action. This might not
+    be the best representation to perform RL with (some framework do not really like it...)
+
+    For more customization on that side, please refer to the section :ref:`gym_compat_box_discrete` below
+
 Observation space and action space customization
 -------------------------------------------------
 By default, the action space and observation space are `gym.spaces.Dict` with the keys being the attribute
 to modify.
 
+Default Observations space
+******************************
 For example, an observation space will look like:
 
 - "a_ex": Box(`env.n_line`,) [type: float, low: 0, high: inf]
 - "a_or": Box(`env.n_line`,) [type: float, low: 0, high: inf]
 - "actual_dispatch": Box(`env.n_gen`,)
+- "curtailment": Box(`env.n_gen`,)  [type: float, low: 0., high: 1.0]
+- "curtailment_limit": Box(`env.n_gen`,)  [type: float, low: 0., high: 1.0]
 - "gen_p": Box(`env.n_gen`,)  [type: float, low: `env.gen_pmin`, high: `env.gen_pmax * 1.2`]
 - "gen_q": Box(`env.n_gen`,)  [type: float, low: -inf, high: inf]
 - "gen_v": Box(`env.n_gen`,)  [type: float, low: 0, high: inf]
@@ -86,7 +104,93 @@ represents the attribute `obs.line_status` which is a boolean vector (for each p
 `True` encodes for "connected" and `False` for "disconnected") See the chapter :ref:`observation_module` for
 more information about these attributes.
 
-We offer some convenience functions to customize these environment. They all more or less the same
+Default Action space
+******************************
+The default action space is also a type of gym Dict. As for the observation space above, it is a
+straight translation from the attribute of the action to the key of the dictionary. This gives:
+
+- "change_bus": MultiBinary(`env.dim_topo`)
+- "change_line_status": MultiBinary(`env.n_line`)
+- "curtail": Box(`env.n_gen`) [type: float, low=0., high=1.0]
+- "redispatch": Box(`env.n_gen`) [type: float, low=-`env.gen_max_ramp_down`, high=`env.gen_max_ramp_up`]
+- "set_bus": Box(`env.dim_topo`) [type: int, low=-1, high=2]
+- "set_line_status": Box(`env.n_line`) [type: int, low=-1, high=1]
+- "storage_power": Box(`env.n_storage`) [type: float, low=-`env.storage_max_p_prod`, high=`env.storage_max_p_absorb`]
+
+.. _base_gym_space_function:
+
+Customizing the action and observation space
+********************************************
+
+We offer some convenience functions to customize these spaces.
+
+If you want a full control on this spaces, you need to implement something like:
+
+.. code-block:: python
+
+    import grid2op
+    env_name = ...
+    env = grid2op.make(env_name)
+
+    from grid2op.gym_compat import GymEnv
+    # this of course will not work... Replace "AGymSpace" with a normal gym space, like Dict, Box, MultiDiscrete etc.
+    from gym.spaces import AGymSpace
+    gym_env = GymEnv(env)
+
+    class MyCustomObservationSpace(AGymSpace):
+        def __init__(self, whatever, you, want):
+            # do as you please here
+            pass
+            # don't forget to initialize the base class
+            AGymSpace.__init__(self, see, gym, doc, as, to, how, to, initialize, it)
+            # eg. Box.__init__(self, low=..., high=..., dtype=float)
+
+        def to_gym(self, observation):
+            # this is this very same function that you need to implement
+            # it should have this exact name, take only one observation (grid2op) as input
+            # and return a gym object that belong to your space "AGymSpace"
+            return SomethingThatBelongTo_AGymSpace
+            # eg. return np.concatenate((obs.gen_p * 0.1, np.sqrt(obs.load_p))
+
+    gym_env.observation_space = MyCustomObservationSpace(whatever, you, wanted)
+
+And for the action space:
+
+.. code-block:: python
+
+    import grid2op
+    env_name = ...
+    env = grid2op.make(env_name)
+
+    from grid2op.gym_compat import GymEnv
+    # this of course will not work... Replace "AGymSpace" with a normal gym space, like Dict, Box, MultiDiscrete etc.
+    from gym.spaces import AGymSpace
+    gym_env = GymEnv(env)
+
+    class MyCustomActionSpace(AGymSpace):
+        def __init__(self, whatever, you, want):
+            # do as you please here
+            pass
+            # don't forget to initialize the base class
+            AGymSpace.__init__(self, see, gym, doc, as, to, how, to, initialize, it)
+            # eg. MultiDiscrete.__init__(self, nvec=...)
+
+        def from_gym(self, gym_action):
+            # this is this very same function that you need to implement
+            # it should have this exact name, take only one action (member of your gym space) as input
+            # and return a grid2op action
+            return TheGymAction_ConvertedTo_Grid2op_Action
+            # eg. return np.concatenate((obs.gen_p * 0.1, np.sqrt(obs.load_p))
+
+    gym_env.action_space = MyCustomActionSpace(whatever, you, wanted)
+
+
+Customizing the action and observation space, using Converter
+**************************************************************
+However, if you don't want to fully customize everything, we encourage you to have a look at the "GymConverter"
+that we coded to ease this process.
+
+They all more or less the same
 manner. We show here an example of a "converter" that will scale the data (removing the value in `substract`
 and divide input data by `divide`):
 
@@ -112,17 +216,96 @@ and divide input data by `divide`):
     gym_env.observation_space = ob_space
 
 
+You can also add a specific keys into this observation space, for example say you want to compute
+the log of the loads instead of giving the direct value to your agent. This can be done with:
+
+.. code-block:: python
+
+    import grid2op
+    from grid2op.gym_compat import GymEnv
+    from grid2op.gym_compat import ScalerAttrConverter
+
+    env_name = "l2rpn_case14_sandbox"  # or any other grid2op environment name
+    g2op_env = grid2op.make(env_name)  # create the gri2op environment
+
+    gym_env = GymEnv(g2op_env)  # create the gym environment
+
+    ob_space = gym_env.observation_space
+    shape_ = (g2op_env.n_load, )
+    ob_space = ob_space.add_key("log_load",
+                                 lambda obs: np.log(obs.load_p),
+                                          Box(shape=shape_,
+                                              low=np.full(shape_, fill_value=-np.inf, dtype=float),
+                                              high=np.full(shape_, fill_value=-np.inf, dtype=float),
+                                              dtype=float
+                                              )
+                                       )
+
+    gym_env.observation_space = ob_space
+    # and now you will get the key "log_load" as part of your gym observation.
+
 A detailed list of such "converter" is documented on the section "Detailed Documentation by class". In
 the table below we describe some of them (**nb** if you notice a converter is not displayed there,
 do not hesitate to write us a "feature request" for the documentation, thanks in advance)
 
-======================================   ============================================================
-Converter name                           Objective
-======================================   ============================================================
-:class:`ContinuousToDiscreteConverter`   TODO
-:class:`MultiToTupleConverter`           TODO
-:class:`ScalerAttrConverter`             TODO
-======================================   ============================================================
+=============================================   ============================================================
+Converter name                                  Objective
+=============================================   ============================================================
+:class:`ContinuousToDiscreteConverter`          Convert a continuous space into a discrete one
+:class:`MultiToTupleConverter`                  Convert a gym MultiBinary to a gym Tuple of gym Binary and a gym MultiDiscrete to a Tuple of Discrete
+:class:`ScalerAttrConverter`                    Allows to scale (divide an attribute by something and subtract something from it
+`BaseGymSpaceConverter.add_key`_                Allows you to compute another "part" of the observation space (you add an information to the gym space)
+`BaseGymSpaceConverter.keep_only_attr`_         Allows you to specify which part of the action / observation you want to keep
+`BaseGymSpaceConverter.ignore_attr`_            Allows you to ignore some attributes of the action / observation (they will not be part of the gym space)
+=============================================   ============================================================
+
+
+.. note::
+
+    With the "converters" above, note that the observation space AND action space will still
+    inherit from gym Dict.
+
+    They are complex spaces that are not well handled by some RL framework.
+
+    These converters only change the keys of these dictionaries !
+
+
+.. _gym_compat_box_discrete:
+
+Customizing the action and observation space, into Box or Discrete
+*******************************************************************
+
+The use of the converter above is nice if you can work with gym Dict, but in some cases, or for some frameworks
+it is not convenient to do it at all.
+
+TO alleviate this problem, we developed 3 types of gym action space, following the architecture
+detailed in subsection :ref:`base_gym_space_function`
+
+===============================   ============================================================
+Converter name                    Objective
+===============================   ============================================================
+:class:`BoxGymObsSpace`           Convert the observation space to a single "Box"
+:class:`BoxGymActSpace`           Convert a gym MultiBinary to a gym Tuple of gym Binary and a gym MultiDiscrete to a Tuple of Discrete
+:class:`MultiDiscreteActSpace`    Allows to scale (divide an attribute by something and subtract something from it
+:class:`DiscreteActSpace`         Allows you to compute another "part" of the observation space (you add an information to the gym space)
+===============================   ============================================================
+
+They can all be used like:
+
+.. code-block:: python
+
+    import grid2op
+    env_name = ...
+    env = grid2op.make(env_name)
+
+    from grid2op.gym_compat import GymEnv, BoxGymObsSpace
+    gym_env = GymEnv(env)
+    gym_env.observation_space = BoxGymObsSpace(gym_env.init_env)
+    gym_env.action_space = MultiDiscreteActSpace(gym_env.init_env)
+
+
+We encourage you to visit the documentation for more information on how to use these classes. Each offer
+different possible customization.
 
 Recommended usage of grid2op with other framework
 --------------------------------------------------
@@ -137,6 +320,17 @@ Other frameworks
 Any contribution is welcome here
 
 
+Detailed Documentation by class
+--------------------------------
+.. automodule:: grid2op.gym_compat
+    :members:
+    :autosummary:
+
+.. autoclass:: grid2op.gym_compat.gym_space_converter._BaseGymSpaceConverter
+    :members:
+    :autosummary:
+
+
 Legacy version
 ---------------
 
@@ -183,11 +377,4 @@ We also implemented some "converter" that allow the conversion of some action sp
 `gym.spaces` (this is only available if gym is installed of course). Please check
 :class:`grid2op.gym_compat.GymActionSpace` for more information and examples.
 
-
-Detailed Documentation by class
---------------------------------
-.. automodule:: grid2op.gym_compat
-    :members:
-    :autosummary:
-
 .. include:: final.rst

docs/plot.rst

@@ -1,5 +1,5 @@
 .. currentmodule:: grid2op.PlotGrid
-.. |replaygif| image:: ../getting_started/path_agents/awesome_agent_logs/000/episode.gif
+.. |replaygif| image:: ./img/random_agent.gif
 .. |14bus_1| image:: ./img/14bus_1.png
 .. |14bus_2| image:: ./img/14bus_2.png
 .. |14bus_th_lim| image:: ./img/14bus_th_lim.png
@@ -57,7 +57,7 @@ the agent pretty easily, and allows easy saving into gif format (see below for m
     # execute this agent on 1 scenario, saving the results
     runner = Runner(**env.get_params_for_runner(), agentClass=CustomRandom)
     path_agent = os.path.join(path_agents, "RandomAgent")
-    res = runner.run(nb_episode=1, path_save=path_agent, pbar=tqdm, agent_seeds=[0, 1])
+    res = runner.run(nb_episode=1, path_save=path_agent, pbar=tqdm, agent_seeds=[0])
     # and now reload it and display the "movie" of this scenario
     plot_epi = EpisodeReplay(path_agent)
     plot_epi.replay_episode(res[0][1], gif_name="episode")

docs/runner.rst

@@ -78,6 +78,49 @@ you can also use the Runner pretty easily by passing it an instance of your agen
 Other tools are available for this runner class, for example the easy integration of progress bars. See bellow for
 more information.
 
+.. _runner-multi-proc-warning:
+
+Note on parallel processing
+----------------------------
+The "Runner" class allows for parallel execution of the same agent on different scenarios. In this case, each
+scenario will be run in independent process.
+
+Depending on the platform and python version, you might end up with some bugs and error like
+
+.. pull-quote::
+    AttributeError: Can't get attribute 'ActionSpace_l2rpn_case14_sandbox' on <module 'grid2op.Space.GridObjects'
+    from '/lib/python3.8/site-packages/grid2op/Space/GridObjects.py'> Process SpawnPoolWorker-4:
+
+or like:
+
+.. pull-quote::
+    File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/pool.py", line 125, in worker
+    result = (True, func(\*args, \*\*kwds))
+
+    File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/pool.py", line 51, in
+    starmapstar return list(itertools.starmap(args[0], args[1]))
+
+In this case this means grid2op has a hard time dealing with the multi processing part. In that case, it
+is recommended to disable it completely, for example by using, before any call to "runner.run" the following code:
+
+.. code-block:: python
+
+    import os
+    from grid2op.Runner import Runner
+
+    os.environ[Runner.FORCE_SEQUENTIAL] = "1"
+
+This will force (starting grid2op >= 1.5) grid2op to use the sequential runner and not deal with the added
+complexity of multi processing.
+
+This is especially handy for "windows" system in case of trouble.
+
+For information, as of writing (march 2021):
+
+- macOS with python <= 3.7 will behave like any python version on linux
+- windows and macOS with python >=3.8 will behave differently than linux but similarly to one another
+
+
 Detailed Documentation by class
 -------------------------------
 .. automodule:: grid2op.Runner

getting_started/03_Action.ipynb

@@ -682,10 +682,17 @@
    "source": [
     "storage_id = [0, 1]\n",
     "values = [-1.7, 2.3]\n",
-    "action_storage = action_space()\n",
-    "action_storage.storage_p = [(stor_id, val) for stor_id, val in zip(storage_id, values)]\n",
+    "action_descr = [(stor_id, val) for stor_id, val in zip(storage_id, values)]\n",
+    "# alternatively: action_descr = np.array([-1.7, 2.3], dtype=float)\n",
+    "\n",
+    "# with the dictionnary\n",
+    "action_storage0 = action_space({\"set_storage\": action_descr})\n",
+    "# or with the property\n",
+    "action_storage1 = action_space()\n",
+    "action_storage1.set_storage = action_descr\n",
+    "\n",
     "print(\"The storage action applied is\")\n",
-    "print(action_storage)"
+    "print(action_storage1)"
    ]
   },
   {