Expanded the first section of doc and added a parameter to mercury

config/mercury_config.toml

@@ -103,6 +103,8 @@ insert_time_stamp = false
 save_all_hotspot_data = false
 hotspot_save_folder = "../results" # Save all regulation information
 
+file_aggregated_results = "results_test.csv"
+
 skip_results = false # will skip computation of the results at the end. Good for testing quickly
 
 # If True, this will skip an iteration with same folder path if it already exists

core/world_builder.py

@@ -748,7 +748,6 @@ def create_airports(self):
 
 		self.airports = {airport.uid: airport for airport in self.airports_per_icao.values()}
 
-
 	def create_AMANs(self):
 		"""
 		Creates all AMANs, including EAMANs.

docs/Basic usage.rst

@@ -8,6 +8,7 @@ get the output from the model.
 
 .. toctree::
     How to run Mercury.rst
+    docker.rst
     Data Input.rst
     Data Output.rst
     CLI.rst

docs/CLI.rst

@@ -10,11 +10,11 @@ following:
 
     ./mercury.py -id -1
 
-This will for instance run one iteration of the basic scenario "-1", reading the default ``config/mercury_config.toml``
-config file.
+This will for instance run one iteration of scenario "-1", reading the default ``config/mercury_config.toml``
+config file. Another path to the config file can be set using the "-psi" option, see below.
 
 The CLI can be used in a very agile way. Indeed, all parameters from the mercury config are accessible through the
-interface, as well as the parameters defined in the scenario (the list of parameter is built at runtime, so even new custom parameters will be accessible). All parameters
+interface. All parameters
 are listed in :ref:`simulation_parameters` but some have shortcuts and the most important ones are explained here:
 
 - "-id": the id of the scenario to run. If "-id X" is run, there must be a "scenario=X" folder in the input folder.
@@ -27,15 +27,17 @@ are listed in :ref:`simulation_parameters` but some have shortcuts and the most
 - "-pc": number of core to use in parallel for multiple iterations (a single run of Mercury is always single-core and
   single thread)
 
-On top of that, oen can use the cli to set and iterate over parameters. Iterable parameters include scenario ids, case
-study ids. For instance, this command performs one iteration on scenario -1 and one iteration on scenarios -2:
+On top of that, one can use the cli to set and iterate over parameters . Iterable parameters
+include scenario ids, case study ids. For instance, this command performs one iteration on scenario -1
+and one iteration on scenarios -2:
 
 .. code:: bash
 
     ./mercury.py -id -1 -2
 
-Any parameter defined in the scenario (or case_study) parameter file can also be fixed or iterated through the CLI.
-For instance, if the price of fuel is defined as:
+Any parameter defined in the scenario (or case_study) parameter file can also be fixed or iterated through the CLI
+(the list of parameter is built at runtime, so even new custom parameters and parameters coming from modules will be
+available). For instance, if the price of fuel is defined as:
 
 .. code:: toml
 
@@ -49,8 +51,8 @@ in the scenario config file, then one can iterate over it like this:
 
     ./mercury.py -id -1 -airlines__fuel_price 0.3 0.5 0.7
 
-Note that by specifying iterations over several parameters, one ends up always with the combination of parameters, for
-instance:
+Note that by specifying iterations over several parameters, one ends up always with the full combination of parameter
+values (the tensorial product), for instance:
 
 .. code:: bash
 

docs/Data Input.rst

@@ -3,16 +3,15 @@
 Data Input
 ==========
 
-Running Mercury requires to input some data in the right format.
-
 Note: a sample of data is provided with Mercury and can be downloaded
 `here <https://zenodo.org/records/11384379/files/Mercury_data_sample.zip?download=1>`_.
 
-The input data is organised in scenarios and case studies. Scenarios can be seen typically as a set of schedules,
+Running Mercury requires to input some data in the right format. The input data is organised in scenarios and case
+studies. Scenarios can be seen typically as a set of schedules,
 passenger itineraries, as well as the definition other agents like the AMAN. A case study is usually represented by a
 subset of flights and/or different operational configuration. When running Mercury, one has to specify as least the id
-of the scenario, and optionally the if of the case study. If no case study is chosen, Mercury will run the case "0",
-coinciding exactly with the data and parameters defined by APIthe scenario itself.
+of the scenario, and optionally the id of a case study. If no case study is chosen, Mercury will run the case "0",
+coinciding exactly with the data and parameters defined by the scenario itself.
 
 The scenarios are read from the input folder, defined in the ``mercury_config.toml`` file, by default ``../input``. The
 input folder should follow the following structure:
@@ -60,18 +59,46 @@ The second part of this file is composed by parameters and their values, for ins
     delay_estimation_lag = 60
 
 Like in the first part, the parameters are organised in different sections and subsections, here for instance "modules"
-and "airlines". This important when using the CLI or the programmatic interface, because the parameters have to be called
-based on their subsections, for instance "airlines__non_ATFM_delay_loc", i.e. the name of the section, two underscores
+and "airlines". This is important when using the CLI or the programmatic interface, because the parameters have to be called
+based on their section, for instance ``airlines__non_ATFM_delay_loc``, i.e. the name of the section, two underscores
 ``__``, then the name of the parameter. A full description of the parameters can be found here:
-:ref:`scenarios_parameters`.
+:ref:`scenario_parameter_file`.
 
 The tables listed in the first part have to be included in the scenario folder, in ``data``, following the same structure
-than the toml file. All tables must be in parquet format.
+than the toml file. All tables must be in parquet format. A detailed description of the all the input tables can be
+found here: :ref:`input_tables`.
+
+When defining a case study for a scenario, one has to follow a similar approach. First, all case studies for a scenario
+should be included in the ``case_studies`` folder inside the scenario directory. Then one has to put a case study
+configuration file called ``case_study_config.toml``1, similar to the scenario's. However, one only needs to put the modification with respect to the
+scenario in this new folder. For instance, this config file:
+
+.. code:: toml
+
+    [info]
+        case_study_id = -1
+        description = "Only flights departing from ED"
+
+    [data]
+        [data.schedules]
+            input_subset = "flight_subset"
+
+    [paras]
+        [paras.airlines]
+        fuel_price = 0.1
+
+        [paras.modules]
+        modules_to_load = ["nostromo_EAMAN"]
+
+    [agents_configuration]
 
-A detailed description of the all the input tables can be found here: :ref:`input_tables`.
+All the four main sections (``info``, ``data``, ``paras``, ``agents_configurations``) have to appear in the files but
+some can be empty. Only parameters that are different need to appear, in this case ``fuel_price`` and ``modules_to_load``.
+ Moreover, one can replace or modify some of the data included in the scenario by including new tables in the ``data``
+folder inside the case study one, and mention it in the toml file (here the ``flight_subset``).
 
 Finally, note that the GUI version of Mercury provides an easy way of exploring the different types of data, modifying
-them, creating scenarios, etc. More information here: :ref:`gui`.
+them, creating scenarios and case studies, etc. More information here: :ref:`gui`.
 
 
 

docs/Data Output.rst

@@ -3,22 +3,25 @@
 Data Output
 ===========
 
-Mercury is an agent-based model with a high number of agents, therefore producing potentially high volume of data
+Mercury is an agent-based model with a high number of agents, therefore producing high volumes of data
 on each agents.
 
+Detailed output
+---------------
+
 The main output of Mercury is a set of tables, stored in the output directory defined by the variable
 ``write_profile.path`` in the mercury config file (by default ``../results``).
 These tables (saved as ``.csv.gz`` files) are stored in a folder whose name is structured as
-``[model version]_[scenario id]_[case study id]_[iteration number]``. One can also insert a timestamp in folder name
-by setting the ``outputs_handling.insert_time_stamp`` parameter to True in the mercury config file.
+``[model version]_[scenario id]_[case study id]_[iteration number]``. One can also insert a timestamp in the folder name
+by setting the ``outputs_handling.insert_time_stamp`` parameter to ``True`` in the mercury config file.
 
 The two most important tables saved by Mercury are the following:
 
 - output_flights: compiles important information the flights, including fuel consumption, delay, etc.
 - output_pax: gathers information related to passengers, in particular their final delay, whether they missed
   connections etc.
 
-Other types of data are also saved at the save time, including:
+Other types of data are also saved at the same time, including:
 
 - output_dci: information related to Dynamic Cost Indexing when used.
 - output_eaman: information related to the optimisation process followed by the EAMAN.
@@ -35,4 +38,25 @@ a script ``unzip_results.py`` is included to easily unzip all the files.
 
 The description of all fields appearing in the output tables can be found here: :ref:`output_tables`.
 
+Aggregated output
+-----------------
+
+On top of the detailed output, Mercury also produces by default a summary of the results, encompassing all
+iterations. This summary is saved to the result folder and is name by the ``outputs_handling.file_aggregated_results``
+parameter, by default ``results.csv`` (note: unless the user manually changes this name in the config file, this file is
+thus overwritten everytime Mercury is run).
+
+These aggregated results represent a subset of the metrics gathered in the full output, computed in average (+std) for
+each iteration over all flights or passengers. The table is structured so that it features the scenario id, the case
+study id, the parameters that have been swept by the user. An example can be seen in the figure below:
+
+.. image:: docs/images/example_aggregated_output.png
+  :width: 600
+  :alt: Example of aggregated output
+
+This aggregated output is the same that is produced by the ``run`` method of the Mercury object (see :ref:`notebook`).
+It is designed to be useful for quick and easy tests, but can also be customised, see :ref:`parametriser_aggregator`.
+
+
+
 

docs/How to run Mercury.rst

@@ -21,15 +21,16 @@ organised as explained in :ref:`data_input` to be readable by Mercury. In partic
 "scenario_config.py", which is compiling all the necessary information to run this particular scenario. It is organised
 in two parts:
 
-- the path too all tables needed to run the scenario, all of them included in the data folder.
+- the path to all tables needed to run the scenario, all of them included in the data folder.
 - the parameters needed to set the simulation up. These parameters are linked to the agent behaviours (e.g.price of fuel).
 
-All scenario parameters are described here: :ref:`scenario_parameter_file`.
+More details about the input data can be found here :ref:`data_intput` and all scenario parameters are described here:
+:ref:`scenario_parameter_file`.
 
-The mercury config also includes the path to where the results will be saved, but default in ``../results``. The structure
+The mercury config also includes the path to where the results will be saved, by default in ``../results``. The structure
 of the results is discussed and explained in :ref:`data_output`.
 
-All parameters both from the mercury_config.toml file and the scenario parameter file can be set at runtime, in the cli
+All parameters both from the ``mercury_config.toml`` file and the scenario parameter file can be set at runtime, in the cli
 version, with the Mercury object, or in the GUI (e.g. using the argument ``--airlines__fuel_price 1.2`` in the CLI).
 
 Finally, the three entry points (CLI, Mercury object, GUI) are available as Docker images, which requires only the

docs/Programmatic use.rst

@@ -25,7 +25,7 @@ It can then be used to run simulation, by passing various arguments. For instanc
     results, results_seq = mercury.run(scenarios=scenarios,
                                           case_studies=case_studies)
 
-will run case study 0 and and -1 from scenario -1, both included in the default input folder (``../input``).
+will run case study 0 and and -1 from scenario -1.
 
 Just like the CLI, one can fix or iterate over any argument included in the Mercury config file or the scenario/case
 study config file, with a slightly different interface than the CLI:
@@ -59,7 +59,7 @@ study config file, with a slightly different interface than the CLI:
 
 In this case we need to load the mercury config file independently beforehand, then modify a parameter inside. There is
 however no need to load the scenario config file, and parameter can be fixed or iterated through the use of two
-dictionary. Note that teh syntax for the parameters are the same than for the CLI, i.e.
+dictionaries, respectively ``paras_sc_fixed`` and ``paras_sc_iterated``. Note that the syntax for the parameters are the same than for the CLI, i.e.
 ``[section of config file]__[name of parameter]``.
 
 The run method returns two dataframes containing the results from the runs. The first one corresponds to the one saved
@@ -69,8 +69,8 @@ the full detailed results (for each iteration), similarly to the CLI, by default
 
 Note that contrary to the CLI version, in general string parameters can be iterated with the object interface.
 
-The Mercury object can also use two important features to help the user, the parmametriser and the aggregator. Both
-are described in detail in the advanced usage, here: :ref:`parametriser_aggregator`
+The Mercury object can also use two important features to help the user, the parametriser and the aggregator. Both
+are described in detail in the advanced usage section, here: :ref:`parametriser_aggregator`
 
 
 

mercury.py

@@ -264,15 +264,15 @@ def manual_bool_cast(string):
 	with generic_connection(profile=profile_write_agg['connection'],
 							typ=profile_write_agg['type'],
 							base_path=profile_write_agg.get('path')) as connection:
-
-		file_name = 'results.csv'
 
-		print('Saving summarised results here: {}'.format((Path(connection['base_path']) / file_name).resolve()))
+		file_name_agg = paras_simulation['outputs_handling__file_aggregated_results'] # 'results.csv'
+
+		print('Saving summarised results here: {}'.format((Path(connection['base_path']) / file_name_agg).resolve()))
 
 		write_data(data=results,
 					fmt=profile_write_agg['fmt'],
 					path=profile_write_agg['path'],
-					file_name=file_name,
+					file_name=file_name_agg,
 					connection=connection,
 					how=profile_write_agg['mode'])
 
@@ -281,7 +281,8 @@ def manual_bool_cast(string):
 			with generic_connection(profile=profile_write_agg['connection'],
 								typ=profile_write_agg['type']) as connection:
 
-				file_name = 'results_seq_{}.csv'.format(stuff)
+				# file_name = 'results_seq_{}.csv'.format(stuff)
+				file_name = '{}_seq_{}.csv'.format(file_name_agg.split('.csv')[0], stuff)
 
 				print('Saving summarised additional results here: {}'.format(
 						(Path(connection['base_path']) / file_name).resolve()))