Merge pull request #145 from BDonnot/bd_dev

Merge recent developments

CHANGELOG.rst

@@ -3,10 +3,11 @@ Change Log
 
 [TODO]
 --------------------
+- [???] example (and test) on how to corrupt the observation for the agent (without corrupting the environment)
+- [???] use some kind of "env.get_state()" when simulating instead of recoding everything "by hand"
 - [???] use "backend.get_action_to_set()" in simulate
 - [???] use the prod_p_forecasted and co in the "next_chronics" of simulate
-- [???] add the storage power in the backend.get_action_to_set()"
-- [???] add a "_cst_" or something in the `const` member of all the class
+- [???] add a "_cst_" or something in the `const` member of all the classes
 - [???] in deepcopy of env, make tests that the "pointers" are properly propagated in the attributes (for example
   `envcpy._game_rules.legal_action` should not be copied when building `envcpy._helper_action_env`)
 - [???] add multi agent
@@ -29,12 +30,42 @@ Change Log
 - [???] "asynch" multienv
 - [???] properly model interconnecting powerlines
 
+[1.6.5] - 2022-xx-yy
+---------------------
+- [BREAKING] the name of the python files for the "Chronics" module are now lowercase (complient with PEP). If you
+  did things like `from grid2op.Chronics.ChangeNothing import ChangeNothing` you need to change it like
+  `from grid2op.Chronics.changeNothing import ChangeNothing` or even better, and this is the preferred way to include
+  them: `from grid2op.Chronics import ChangeNothing`. It should not affect lots of code (more refactoring of the kind
+  are to be expected in following versions).
+- [BREAKING] same as above for the "Observation" module. It should not affect lots of code (more refactoring of the kind
+  are to be expected in following versions).
+- [FIXED] an issue when copying the environment with the opponent (see issue https://github.com/rte-france/Grid2Op/issues/274)
+- [FIXED] a bug leading to the wrong "backend.get_action_to_set()" when there were storage units on the grid. 
+- [FIXED] a bug in the "BackendConverter" when there are storage  on the grid
+- [FIXED] issue https://github.com/rte-france/Grid2Op/issues/265
+- [FIXED] issue https://github.com/rte-france/Grid2Op/issues/261
+- [ADDED] possibility to "env.set_id" by giving only the folder of the chronics and not the whole path.
+- [ADDED] function "env.chronics_handler.available_chronics()" to return the list of available chronics
+  for a given environment
+- [ADDED] possibility, through the `Parameters` class, to limit the number of possible calls to `obs.simulate(...)` 
+  see `param.MAX_SIMULATE_PER_STEP` and `param.MAX_SIMULATE_PER_EPISODE` (see issue https://github.com/rte-france/Grid2Op/issues/273)
+- [ADDED] a class to generate a "Chronics" readable by grid2op from numpy arrays (see https://github.com/rte-france/Grid2Op/issues/271)
+- [ADDED] an attribute `delta_time` in the observation that tells the time (in minutes) between two consecutive steps.
+- [ADDED] a method of the action space to show a list of actions to get back to the original topology 
+  (see https://github.com/rte-france/Grid2Op/issues/275)
+  `env.action_space.get_back_to_ref_state(obs)`
+- [ADDED] a method of the action to store it in a grid2op independant fashion (using json and dictionaries), see `act.as_serializable_dict()`
+- [ADDED] possibility to generate a gym `DiscreteActSpace` from a given list of actions (see https://github.com/rte-france/Grid2Op/issues/277)
+- [IMPROVED] observation now raises `Grid2OpException` instead of `RuntimeError`
+- [IMRPOVED] docs (and notebooks) for the "split_train_val" https://github.com/rte-france/Grid2Op/issues/269
+- [IMRPOVED] the "split_train_val" function to also generate a test dataset see https://github.com/rte-france/Grid2Op/issues/276
+
 [1.6.4] - 2021-11-08
 ---------------------
-- [BREAKING] the name of the python file for the "agent" module are now lowercase (complient with PEP). If you
+- [BREAKING] the name of the python files for the "agent" module are now lowercase (complient with PEP). If you
   did things like `from grid2op.Agent.BaseAgent import BaseAgent` you need to change it like
   `from grid2op.Agent.baseAgent import BaseAgent` or even better, and this is the preferred way to include
-  them: `from grid2op.Agent import BaseAgent` It should not affect lots of code.
+  them: `from grid2op.Agent import BaseAgent`. It should not affect lots of code.
 - [FIXED] a bug where the shunt had a voltage when disconnected using pandapower backend
 - [FIXED] a bug preventing to print the action space if some "part" of it had no size (empty action space)
 - [FIXED] a bug preventing to copy an action properly (especially for the alarm)

_profiling/utils_benchmark.py

@@ -189,7 +189,7 @@ def run_env(env, max_ts, agent):
     reward = env.reward_range[0]
     nb_ts = 0
     prev_act = None
-    beg_ = time.time()
+    beg_ = time.perf_counter()
     with tqdm(total=nb_rows) as pbar:
         while not done:
             act = agent.act(obs, reward, done)
@@ -206,7 +206,7 @@ def run_env(env, max_ts, agent):
             prev_act = act
             # if done:
             #     print(act)
-    end_ = time.time()
+    end_ = time.perf_counter()
     total_time = end_ - beg_
     return nb_ts, total_time, aor, gen_p, gen_q
 
@@ -222,7 +222,7 @@ def run_env_with_reset(env, max_ts, agent, seed=None):
     done = False
     reward = env.reward_range[0]
     nb_ts = 0
-    beg_ = time.time()
+    beg_ = time.perf_counter()
     reset_count = 0
     with tqdm(total=nb_rows) as pbar:
         while not done:
@@ -242,7 +242,7 @@ def run_env_with_reset(env, max_ts, agent, seed=None):
                 obs = env.reset()
                 reset_count += 1
                 done = False
-    end_ = time.time()
+    end_ = time.perf_counter()
     total_time = end_ - beg_
     return nb_ts, total_time, aor, gen_p, gen_q, reset_count
 

docs/action.rst

@@ -359,6 +359,8 @@ An action can be incorrect because of two main factors:
 
 Ambiguous or Illegal, the action will be replaced by a "do nothing" without any other incidents on the game.
 
+.. _action_powerline_status:
+
 Note on powerline status
 ------------------------
 As of grid2op version 1.2.0, we attempted to clean and rationalize the API concerning the change of

docs/environment.rst

@@ -550,11 +550,12 @@ This can be done with:
 
     # extract 1% of the "chronics" to be used in the validation environment. The other 99% will
     # be used for test
-    nm_env_train, nm_env_val = env.train_val_split_random(pct_val=1.)
+    nm_env_train, nm_env_val, nm_env_test = env.train_val_split_random(pct_val=1., pct_test=1.)
 
     # and now you can use the training set only to train your agent:
     print(f"The name of the training environment is \\"{nm_env_train}\\"")
     print(f"The name of the validation environment is \\"{nm_env_val}\\"")
+    print(f"The name of the test environment is \\"{nm_env_test}\\"")
     env_train = grid2op.make(nm_env_train)
 
 You can then use, in the above case:
@@ -577,67 +578,9 @@ And then, at time of validation:
     env_val = grid2op.make(env_name+"_val") # to only use the "validation chronics"
     # do whatever you want with env_val
 
-
-As of now, grid2op do not support "from the API" the possibility to split with convenient
-names a environment a second times. If you want to do a "train / validation / test" split we recommend you to:
-
-1. make a training / test split (see below)
-2. split again the training set into training / validation (see below)
-3. you will have locally an environment named "trainval" on your computer. This directory will not weight
-   more than a few kilobytes.
-
-The example, not really convenient at the moment, please find a feature request if that is a problem for
-you:
-
-.. code-block:: python
-
-    import grid2op
-    import os
-
-    env_name = "l2rpn_case14_sandbox"  # or any other...
-    env = grid2op.make(env_name)
-
-    # retrieve the names of the chronics:
-    full_path_data = env.chronics_handler.subpaths
-    chron_names = [os.path.split(el)[-1] for el in full_path_data]
-
-    # splitting into training / test, keeping the "last" 10 chronics to the test set
-    nm_env_trainval, nm_env_test = env.train_val_split(val_scen_id=chron_names[-10:],
-                                                       add_for_val="test",
-                                                       add_for_train="trainval")
-
-    # now splitting again the training set into training and validation, keeping the last 10 chronics
-    # of this environment for validation
-    env_trainval = grid2op.make(nm_env_trainval)  # create the "trainval" environment
-    full_path_data = env_trainval.chronics_handler.subpaths
-    chron_names = [os.path.split(el)[-1] for el in full_path_data]
-    nm_env_train, nm_env_val = env_trainval.train_val_split(val_scen_id=chron_names[-10:],
-                                                            remove_from_name="_trainval$")
-
-And later on, you can do, if you followed the names above:
-
-.. code-block:: python
-
-    import grid2op
-    import os
-
-    env_name = "l2rpn_case14_sandbox"  # or any other...
-    env_train = grid2op.make(env_name+"_train")
-    env_val = grid2op.make(env_name+"_val")
+    # and of course
     env_test = grid2op.make(env_name+"_test")
 
-And you can also, if you want, delete the folder "l2rpn_case14_sandbox_trainval" from your machine:
-
-.. code-block:: python
-
-    import grid2op
-    import os
-
-    env_name = "l2rpn_case14_sandbox"  # or any other...
-    env_trainval = grid2op.make(env_name+"_trainval")
-    print(f"You can safely delete, if you want, the folder: \n\t\"{env_trainval.get_path_env()}\" \nnow useless.")
-
-
 Customization
 -------------
 

docs/gym.rst

@@ -103,6 +103,7 @@ For example, an observation space will look like:
 - "v_ex": Box(`env.n_line`,)  [type: float, low: 0, high: inf]
 - "v_or": Box(`env.n_line`,)  [type: flaot, low: 0, high: inf]
 - "year": Discrete(2100)
+- "delta_time": Box(0.0, inf, (1,), float32)
 
 Each keys correspond to an attribute of the observation. In this example `"line_status": MultiBinary(20)`
 represents the attribute `obs.line_status` which is a boolean vector (for each powerline
@@ -320,6 +321,8 @@ Reinforcement learning frameworks
 
 TODO
 
+Any contribution is welcome here
+
 Other frameworks
 **********************
 Any contribution is welcome here

docs/observation.rst

@@ -49,6 +49,7 @@
 .. _attention_budget: ./observation.html#grid2op.Observation.BaseObservation.attention_budget
 .. _max_step: ./observation.html#grid2op.Observation.BaseObservation.max_step
 .. _current_step: ./observation.html#grid2op.Observation.BaseObservation.current_step
+.. _delta_time: ./observation.html#grid2op.Observation.BaseObservation.delta_time
 
 .. _observation_module:
 
@@ -131,6 +132,7 @@ Name(s)
 `last_alarm`_                                                                    int        `dim_alarms`_
 `attention_budget`_                                                              int        1
 `max_step`_ , `current_step`_                                                    int        1
+`delta_time`_                                                                    float      1
 =============================================================================    ========= ============
 
 (*NB* for concision, if a coma ("*,*") is present in the "Name(s)" part of the column, it means multiple attributes

getting_started/03_Action.ipynb

@@ -624,7 +624,7 @@
    "outputs": [],
    "source": [
     "curtail_act2 = action_space()\n",
-    "curtail_act2.curtail = [(gen_id, amount)]\n",
+    "curtail_act2.curtail = [(gen_id, ratio_curtailment)]\n",
     "print(curtail_act2)"
    ]
   },
@@ -1346,7 +1346,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.5"
+   "version": "3.8.10"
   }
  },
  "nbformat": 4,

getting_started/04_TrainingAnAgent.ipynb

@@ -84,6 +84,50 @@
     "res"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 0) Good practice\n",
+    "\n",
+    "As in other machine learning tasks, we highly recommend, before even trying to train an agent, to split the \"chronics\" (ie the episode data) into 3 datasets:\n",
+    "- \"train\" use to train the agent\n",
+    "- \"val\" use to validate the hyper parameters\n",
+    "- \"test\" at which you would look only once to report the agent performance in a scientific paper (for example)\n",
+    "\n",
+    "Grid2op lets you do that with relative ease:\n",
+    "\n",
+    "```python\n",
+    "import grid2op\n",
+    "env_name = \"l2rpn_case14_sandbox\"  # or any other...\n",
+    "env = grid2op.make(env_name)\n",
+    "\n",
+    "# extract 1% of the \"chronics\" to be used in the validation environment. The other 99% will\n",
+    "# be used for test\n",
+    "nm_env_train, nm_env_val, nm_env_test = env.train_val_split_random(pct_val=1., pct_test=1.)\n",
+    "\n",
+    "# and now you can use the training set only to train your agent:\n",
+    "print(f\"The name of the training environment is \\\\\"{nm_env_train}\\\\\"\")\n",
+    "print(f\"The name of the validation environment is \\\\\"{nm_env_val}\\\\\"\")\n",
+    "print(f\"The name of the test environment is \\\\\"{nm_env_test}\\\\\"\")\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "And now, you can use the training environment to train your agent:\n",
+    "\n",
+    "```python\n",
+    "import grid2op\n",
+    "env_name = \"l2rpn_case14_sandbox\"\n",
+    "env = grid2op.make(env_name+\"_train\")\n",
+    "```\n",
+    "\n",
+    "Be carefull, on windows you might run into issues. Don't hesitate to have a look at the documentation of this funciton if this the case (see https://grid2op.readthedocs.io/en/latest/environment.html#grid2op.Environment.Environment.train_val_split and https://grid2op.readthedocs.io/en/latest/environment.html#grid2op.Environment.Environment.train_val_split_random)"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -1004,7 +1048,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.5"
+   "version": "3.8.10"
   }
  },
  "nbformat": 4,