diff --git a/README.md b/README.md index 262b5ea..a555ff3 100644 --- a/README.md +++ b/README.md @@ -1,28 +1,122 @@ -# Energy-optimal Federated Learning +## 1. Artifact Identification ++ **Article’s title**: Scheduling Algorithms for Federated Learning With Minimal Energy Consumption ++ **Author’s name and affiliation**: Laércio Lima Pilla - University of Bordeaux, CNRS, Bordeaux INP, Inria, LaBRI, UMR 5800, F-33400 Talence, France ++ **DOI** : [10.1109/TPDS.2023.3240833](https://doi.org/10.1109/TPDS.2023.3240833) ++ **Abstract**: Federated Learning (FL) has opened the opportunity for collaboratively training machine learning models on heterogeneous mobile or Edge devices while keeping local data private. With an increase in its adoption, a growing concern is related to its economic and environmental cost (as is also the case for other machine learning techniques). Unfortunately, little work has been done to optimize its energy consumption or emissions of carbon dioxide or equivalents, as energy minimization is usually left as a secondary objective. In this paper, we investigate the problem of minimizing the energy consumption of FL training on heterogeneous devices by controlling the workload distribution. We model this as the Minimal Cost FL Schedule problem, a total cost minimization problem with identical, independent, and atomic tasks that have to be assigned to heterogeneous resources with arbitrary cost functions. We propose a pseudo-polynomial optimal solution to the problem based on the previously unexplored Multiple-Choice Minimum-Cost Maximal Knapsack Packing Problem. We also provide four algorithms for scenarios where cost functions are monotonically increasing and follow the same behavior. This artifact contains implementations of the aforementioned algorithms and the means to compare the quality of their solutions and their execution times. They enable the reproduction of the article’s results, and they can be adapted for use in real systems or new experiments. -This repository contains the prototypes of five different scheduling algorithms able to minimize the energy consumed on heterogeneous devices during a Federated Learning training round. -The algorithms are described in a preprint available in [HAL](https://hal.archives-ouvertes.fr/hal-03775491) and [arXiv](https://arxiv.org/abs/2209.0621). -This repository was built based on a [previous codebase](https://github.com/llpilla/olar-federated-learning) used to simulate scheduling algorithms that try to minimize the duration of training rounds. +## 2. Artifact Dependencies and Requirements +This code has been tested on systems with the following basic characteristics: ++ Hardware resources: an x86_64 processor, few GB of RAM, few MB of storage. The running code never consumed more than 1 GB of memory and its results never occupied more than 10 MB in storage. Initial results indicate that the code also runs on ARM 64 processors (A64FX more specifically). ++ Operating systems: a Linux system such as Ubuntu. The code should be compatible with MacOS systems too. Windows systems with Windows Subsystem for Linux should also be compatible. ++ Software libraries needed: Python3 for the base code and Shell script for some simple scripts. We use numpy in the base code and matplotlib, pandas, seaborn, and scipy for the analysis of the results and figure generation. Jupyter can also be used for analysis but it is not required. ++ Input datasets: all inputs are generated by Python3 scripts already available in the artifact. Two sets of results generated in different experimental campaigns are also available to enable the replication of the analysis without requiring new runs. -The Python 3 and Bash scripts can be used to test scheduling algorithms in different scenarios with variations in the number of tasks, resources, kinds of resources (cost functions), lower and upper limits of tasks per resources, etc. +The specific characteristics of previously employed environments are provided later in Section 4. Reproducibility of Experiments. -If you wish to reproduce all the results in "*Scheduling Algorithms for Federated Learning with Minimal Energy Consumption*", execute `./run_all_total_cost_experiments.sh` to get total cost results and `./run_all_timing_experiments.sh` to get execution time results. -If you want to reproduce the analysis in said manuscript, check [stored\_results](stored\_results) and run `./run_all_analysis.sh`. +## 3. Artifact Installation and Deployment Process +### Installation +If you download this artifact from Zenodo (directly from the website or using `wget`), you will have to decompress the .zip file (for instance, using `unzip [file]`). You can skip this step if you are downloading this artifact from GitHub using `git clone`. +You will find the following organization in the base folder: +``` +├── chameleon_requirements.txt +├── chameleon_results +│ ├── Analysis of execution times.ipynb +│ ├── Analysis of execution times.py +│ ├── Analysis of total costs.ipynb +│ ├── Analysis of total costs.py +│ ├── results_of_timing_with_fixed_resources.csv +│ ├── results_of_timing_with_fixed_tasks.csv +│ ├── results_with_constant_marginal_costs.csv +│ ├── results_with_constant_marginal_costs_no_upper_limit.csv +│ ├── results_with_decreasing_marginal_costs.csv +│ ├── results_with_increasing_marginal_costs.csv +│ ├── results_with_random_costs.csv +│ └── run_all_analysis.sh +├── code +│ ├── devices.py +│ ├── __init__.py +│ ├── schedulers.py +│ └── support.py +├── experiment_with_constant_marginal_costs_no_upper_limit.py +├── experiment_with_constant_marginal_costs.py +├── experiment_with_decreasing_marginal_costs.py +├── experiment_with_increasing_marginal_costs.py +├── experiment_with_random_costs.py +├── LICENSE +├── original_requirements.txt +├── original_results +│ ├── Analysis of execution times.ipynb +│ ├── Analysis of execution times.py +│ ├── Analysis of total costs.ipynb +│ ├── Analysis of total costs.py +│ ├── results_of_timing_with_fixed_resources.csv +│ ├── results_of_timing_with_fixed_tasks.csv +│ ├── results_with_constant_marginal_costs.csv +│ ├── results_with_constant_marginal_costs_no_upper_limit.csv +│ ├── results_with_decreasing_marginal_costs.csv +│ ├── results_with_increasing_marginal_costs.csv +│ ├── results_with_random_costs.csv +│ └── run_all_analysis.sh +├── README.md +├── run_all_timing_experiments.sh +├── run_all_total_cost_experiments.sh +├── run_analysis_on_new_results.sh +├── timing_with_fixed_resources.py +├── timing_with_fixed_tasks.py +└── unitary_tests.py +``` +You can install the necessary Python3 libraries using the command `pip3 install -r [requirements file]`. Use file `original_requirements.txt` for more recent library versions and to reproduce the environment of the original experiments in the article, or use file `chameleon_requirements.txt` for older library versions and to reproduce the results in the Chameleon platform. -## Dependencies +### Deployment +If you want to build your own experiments, please check the `experiment_with_…` files for examples on how to test the algorithms. -We use numpy, matplotlib, pandas, seaborn, and scipy. -Please run `pip3 install -r requirements.txt` to install them, if needed. +You can run codes directly using Python3 (example: `python3 experiment_with_random_costs.py`). -## How to use +You can also use one of the following Shell scripts below to launch multiple experiments: ++ `run_all_total_cost_experiments.sh`: runs all total cost experiments using files starting with the name “experiment”. ++ `run_all_timing_experiments.sh`: runs all timing experiments using files starting with the name “timing”. -If you want to reproduce total cost results only, run `./run_all_total_cost_experiments.py`. It takes about 11 core-hours to run. -Given that the experiments are independent, they could be parallelized with mpi4py. +Running these scripts should take between 10 and 15 hours each on most platforms. More detailed timings are provided in Section 4.3 Experimental platforms. -If you want to reproduce scheduling time results only, run `./run_all_timing_experiments.py`. Beware that they may take several hours to run (about 10 hours in a conventional laptop), and the computer should be left alone while running these experiments to avoid adding unwanted noise to the results. +After running your experiments, you can use the script `run_analysis_on_new_results.sh` to run the analysis scripts that are stored in the `original_results` folder (`Analysis of total costs.py` and `Analysis of execution times.py`) over your new results. The Python3 scripts can also be used to generate the same figures and analysis employed in the article using the provided data. -If you have reproduced these results and want to analyze them, run `./run_analysis_on_new_results.sh`. -The available timing results were obtained on a Latitude 7420 laptop with an 11th Gen Intel(R) Core(TM) i7-1185G7 processor, 32GB of LPDDR4X memory running at 2133Mhz, and a Western Digital PC SN530 NVMe WDC 512GB SSD. -The OS on the machine was Ubuntu 20.04.5 LTS with kernel version 5.14.0-1054-oem. Python3 version 3.8.10 was used to run the experiments. +## 4. Reproducibility of Experiments +A detailed description of the experiments and their results is available in the supplemental material of the article “Scheduling Algorithms for Federated Learning With Minimal Energy Consumption” (DOI: [10.1109/TPDS.2023.3240833/mm1](https://doi.org/10.1109/TPDS.2023.3240833/mm1)). + +Experiments are conducted in two parts: one run to gather total cost results, and one run for timing the algorithms. It is important to not run other software during the timing experiments to avoid adding noise to the performance measurements. These experiments have been first conducted on an ‘original’ machine (whose results are presented in the article and supplemental material) and they have been reproduced in a machine in the Chameleon testbed. Information about both machines are provided later in Section 4.3. + +### 4.1 Total cost experiments +These experiments test the five proposed algorithms ((MC)2MKP, MarIn, MarCo, MarDecUn, and MarDec) and FedAvg with resources following four different cost behaviors (random, marginal increasing, marginal constant, and marginal decreasing). The algorithms have to schedule from 1000 to 5000 tasks (in increments of 100) over 10 or 100 resources. They can be launched with the command `./run_all_total_cost_experiments.sh`. + +The results are organized in five CSV files, namely `results_with_constant_marginal_costs.csv`, `results_with_constant_marginal_costs_no_upper_limit.csv`, `results_with_decreasing_marginal_costs.csv`, `results_with_increasing_marginal_costs.csv`, and `results_with_random_costs.csv`. Each line in the CSV files contains the name of the algorithm employed, the number of tasks, the number of resources, and the total cost found in the scenario. Each file also contains a header with a description of the experiment. + +These results emphasize the known properties of the algorithms: ++ (MC)2MKP always finds optimal solutions; ++ MarIn finds optimal solutions when marginal costs are increasing (or constant); ++ MarCo finds optimal solutions when marginal costs are constant; ++ MarDec and MarDecUn find optimal solutions when marginal costs are decreasing (or constant); ++ FedAvg provides no guarantees of optimality. + +The original results and the results collected in the Chameleon testbed show minor differences for the results with increasing and decreasing marginal costs. The differences are on the low digits of some floating point values. The absolute differences are under 10e-10 and the relative differences (absolute difference divided by the original value) are under 10e-15. We attribute these differences to changes in hardware and software that can lead to different floating point imprecisions. Nonetheless, these minor differences do not change any of the conclusions of the experiments. + +### 4.2 Timing experiments +Our experiments here are split into two. We first fix the number of resources at 100, and vary the tasks from 200 to 2000 in increments of 200, showing us how the number of tasks influences the execution time. Then, we fix the number of tasks at 2000, and vary the number of resources from 20 to 80 in increments of 20. All resources follow linear cost functions, as we assume that their costs should not have a major impact on the performance of the schedulers. For each triple we gather 20 samples. Each sample is composed of 5 runs of a scheduler measured using Python’s timeit module. The order that the samples are collected is randomized to reduce issues with interference and system jitter. These experiments can be launched with the command `./run_all_timing_experiments.sh`. + +The results are organized in two CSV files named `results_of_timing_with_fixed_resources.csv` and `results_of_timing_with_fixed_tasks.csv`. Each line in the CSV files contains the name of the algorithm employed, the number of tasks, the number of resources, and the time taken to run the five repetitions in each sample in seconds. Each file also contains a header with a description of the experiment. + +These results reflect how the different time complexities of the algorithms lead to vastly different execution times. (MC)2MKP (O(T^2n)) is visibly the slowest algorithm, with execution times varying between the hundreds of milliseconds and the tens of seconds. When we compare it to MarDec (O(Tn^2)), we can see that their times are similar when the number of tasks and resources are similar too. However, when we increase the number of tasks by a factor of ten, (MC)2MKP’s time increases by a factor of a hundred, while MarDec’s time only increases by a factor of ten. + +MarDecUn and FedAvg show a difference of about one order of magnitude in their execution times, even though they are both linear in the number of resources. This happens because they require very different operations. MarDecUn requires looping over the resources to assign the lower limits to all resources and to find the one with the smallest marginal cost. Meanwhile, FedAvg can directly assign the same number of tasks to all resources using an optimized numpy operation, leading to a faster execution. + +### 4.3 Experimental platforms ++ **Original** + + Hardware: experiments were executed on a Dell Latitude 7420 notebook with an 11th Gen Intel(R) Core(TM) i7-1185G7 processor, 32GB of LPDDR4X RAM (2133MHz), and a Western Digital PC SN530 NVMe WDC 512GB SSD. The computer was plugged to a power source at all times. + + Software: the computer runs Ubuntu 20.04.5 LTS (kernel version 5.14.0-1054-oem). We used Python 3.8.10 with numpy version 1.22.2 for the experiments. Modules matplotlib (3.5.1), pandas (1.4.3), seaborn (0.11.2), and scipy (1.7.1) were used for the visualization and statistical analysis of results. During scheduling time experiments, no other applications were open besides a terminal. + + Required time: total cost experiments take about 11 hours, while timing experiments take about 10 hours in this machine. Times were measured with `time`. + + Previously generated results: all original results are available in folder `original_results`. Please run `./run_all_analysis.sh` in the folder to repeat the result analysis. ++ **Chameleon** + + Hardware: experiments were conducted on a *compute_skylake* node at TACC through the Chameleon testbed. It contains two Intel(R) Xeon(R) Gold 6126 processors, 192 GB of DDR4 RAM (2666MHz), and 240GB of SSD storage. + + Software: the computer runs an image of CC-Ubuntu18.04 (kernel version 4.15.0-191-generic), which comes with Python 3.6.9. Modules matplotlib (3.3.4), pandas (1.1.5), seaborn (0.11.2), scipy (1.5.4), and numpy (1.19.5) were installed for the experiments. The tool `screen` was used to keep the experiments running while disconnected. + + Required time: total cost experiments take about 13.5 hours, while timing experiments take about 13 hours in this machine. Times were measured with `time`. + + Previously generated results: all original results are available in folder `chameleon_results`. Please run `./run_all_analysis.sh` in the folder to repeat the result analysis. -If you want to build new experiments, check the several `experiment` and `timing` files available. diff --git a/chameleon_requirements.txt b/chameleon_requirements.txt new file mode 100755 index 0000000..f21febb --- /dev/null +++ b/chameleon_requirements.txt @@ -0,0 +1,5 @@ +numpy>=1.19.5 +matplotlib>=3.3.4 +pandas>=1.1.5 +seaborn>=0.11.2 +scipy>=1.5.4 diff --git a/stored_results/Analysis of execution times.ipynb b/chameleon_results/Analysis of execution times.ipynb similarity index 89% rename from stored_results/Analysis of execution times.ipynb rename to chameleon_results/Analysis of execution times.ipynb index 5d22283..876f7d5 100644 --- a/stored_results/Analysis of execution times.ipynb +++ b/chameleon_results/Analysis of execution times.ipynb @@ -59,7 +59,8 @@ "source": [ "# reads the result file\n", "results = pd.read_csv('results_of_timing_with_fixed_resources.csv', comment='#')\n", - "print('- Results with increasing numbers of tasks')\n", + "print('Execution time analysis')\n", + "print('- Results with increasing numbers of tasks (fixed resources)')\n", "results.head(9)" ] }, @@ -93,7 +94,7 @@ "metadata": {}, "outputs": [], "source": [ - "print('-- Generating figures')\n", + "print('-- Generating figure fig-time-fixed-resources.pdf for the results with a fixed number of resources')\n", "\n", "# Sets figure parameters\n", "plt.figure(figsize=(6,5))\n", @@ -140,14 +141,14 @@ "metadata": {}, "outputs": [], "source": [ - "print('- Result description')\n", + "print('- Result description: data distribution for 200 and 2000 tasks with different schedulers')\n", "for sched in schedulers:\n", " for tasks in (200, 2000):\n", " res = results[(results['Scheduler'] == sched) &\n", " (results['Tasks'] == tasks)].avg\n", " print(f'Scheduler {sched} with {tasks} tasks')\n", " print(res.describe())\n", - " print('\\n')" + " print(' ')" ] }, { @@ -164,6 +165,7 @@ "outputs": [], "source": [ "print('-- Checking the distribution of origin for different results')\n", + "print(' Kolmogorov-Smirnov test results with p-values < 0.05 mean that the results do not follow normal distributions')\n", "# Checking all schedulers\n", "np.random.seed(2022)\n", "for sched in schedulers:\n", @@ -183,6 +185,8 @@ "source": [ "# Statistical comparison between MarCo and MarDecUn\n", "# Using Mann-Whitney U test as results do not follow normal distributions some times (p-values < 0.05)\n", + "print(' ')\n", + "print('-- Comparing MarCo and MarDecUn. p-values < 0.05 indicate that the algorithms perform differently')\n", "for tasks in range(200,2001,200):\n", " marco = list(results[(results['Scheduler'] == 'MarCo') &\n", " (results['Tasks'] == tasks)].avg)\n", @@ -214,7 +218,7 @@ "source": [ "# reads the result file\n", "results = pd.read_csv('results_of_timing_with_fixed_tasks.csv', comment='#')\n", - "print('- Results with increasing numbers of resources')\n", + "print('\\n- Results with increasing numbers of resources (fixed tasks)')\n", "results.head(9)" ] }, @@ -248,7 +252,7 @@ "metadata": {}, "outputs": [], "source": [ - "print('-- Generating figures')\n", + "print('-- Generating figure fig-time-fixed-tasks.pdf for the results with a fixed number of tasks')\n", "\n", "# Sets figure parameters\n", "plt.figure(figsize=(6,5))\n", @@ -295,14 +299,14 @@ "metadata": {}, "outputs": [], "source": [ - "print('- Result description')\n", + "print('- Result description: data distribution for 20 and 80 resources with different schedulers')\n", "for sched in schedulers:\n", " for resources in (20, 80):\n", " res = results[(results['Scheduler'] == sched) &\n", " (results['Resources'] == resources)].avg\n", " print(f'Scheduler {sched} with {resources} resources')\n", " print(res.describe())\n", - " print('\\n')" + " print(' ')" ] }, { @@ -319,6 +323,7 @@ "outputs": [], "source": [ "print('-- Checking the distribution of origin for different results')\n", + "print(' Kolmogorov-Smirnov test results with p-values < 0.05 mean that the results do not follow normal distributions')\n", "# Checking all schedulers\n", "np.random.seed(2022)\n", "for sched in schedulers:\n", @@ -338,13 +343,16 @@ "source": [ "# Statistical comparison between MarCo and MarDecUn\n", "# Using Mann-Whitney U test as results do not follow normal distributions some times (p-values < 0.05)\n", + "print(' ')\n", + "print('-- Comparing MarCo and MarDecUn. p-values < 0.05 indicate that the algorithms perform differently')\n", "for resources in range(20,81,20):\n", " marco = list(results[(results['Scheduler'] == 'MarCo') &\n", " (results['Resources'] == resources)].avg)\n", " mardec = list(results[(results['Scheduler'] == 'MarDec') &\n", " (results['Resources'] == resources)].avg)\n", " print(f'Mann-Whitney U test - ({resources} resources).')\n", - " print(stats.mannwhitneyu(marco, mardec, alternative='two-sided'))" + " print(stats.mannwhitneyu(marco, mardec, alternative='two-sided'))\n", + "print('\\n') " ] }, { diff --git a/stored_results/Analysis of execution times.py b/chameleon_results/Analysis of execution times.py similarity index 86% rename from stored_results/Analysis of execution times.py rename to chameleon_results/Analysis of execution times.py index 4cce930..2c2f9a5 100644 --- a/stored_results/Analysis of execution times.py +++ b/chameleon_results/Analysis of execution times.py @@ -41,7 +41,8 @@ # reads the result file results = pd.read_csv('results_of_timing_with_fixed_resources.csv', comment='#') -print('- Results with increasing numbers of tasks') +print('Execution time analysis') +print('- Results with increasing numbers of tasks (fixed resources)') results.head(9) @@ -66,7 +67,7 @@ # In[ ]: -print('-- Generating figures') +print('-- Generating figure fig-time-fixed-resources.pdf for the results with a fixed number of resources') # Sets figure parameters plt.figure(figsize=(6,5)) @@ -105,14 +106,14 @@ # In[ ]: -print('- Result description') +print('- Result description: data distribution for 200 and 2000 tasks with different schedulers') for sched in schedulers: for tasks in (200, 2000): res = results[(results['Scheduler'] == sched) & (results['Tasks'] == tasks)].avg print(f'Scheduler {sched} with {tasks} tasks') print(res.describe()) - print('\n') + print(' ') # ### Distributions of the results @@ -121,6 +122,7 @@ print('-- Checking the distribution of origin for different results') +print(' Kolmogorov-Smirnov test results with p-values < 0.05 mean that the results do not follow normal distributions') # Checking all schedulers np.random.seed(2022) for sched in schedulers: @@ -137,6 +139,8 @@ # Statistical comparison between MarCo and MarDecUn # Using Mann-Whitney U test as results do not follow normal distributions some times (p-values < 0.05) +print(' ') +print('-- Comparing MarCo and MarDecUn. p-values < 0.05 indicate that the algorithms perform differently') for tasks in range(200,2001,200): marco = list(results[(results['Scheduler'] == 'MarCo') & (results['Tasks'] == tasks)].avg) @@ -155,7 +159,7 @@ # reads the result file results = pd.read_csv('results_of_timing_with_fixed_tasks.csv', comment='#') -print('- Results with increasing numbers of resources') +print('\n- Results with increasing numbers of resources (fixed tasks)') results.head(9) @@ -180,7 +184,7 @@ # In[ ]: -print('-- Generating figures') +print('-- Generating figure fig-time-fixed-tasks.pdf for the results with a fixed number of tasks') # Sets figure parameters plt.figure(figsize=(6,5)) @@ -219,14 +223,14 @@ # In[ ]: -print('- Result description') +print('- Result description: data distribution for 20 and 80 resources with different schedulers') for sched in schedulers: for resources in (20, 80): res = results[(results['Scheduler'] == sched) & (results['Resources'] == resources)].avg print(f'Scheduler {sched} with {resources} resources') print(res.describe()) - print('\n') + print(' ') # ### Distributions of the results @@ -235,6 +239,7 @@ print('-- Checking the distribution of origin for different results') +print(' Kolmogorov-Smirnov test results with p-values < 0.05 mean that the results do not follow normal distributions') # Checking all schedulers np.random.seed(2022) for sched in schedulers: @@ -251,6 +256,8 @@ # Statistical comparison between MarCo and MarDecUn # Using Mann-Whitney U test as results do not follow normal distributions some times (p-values < 0.05) +print(' ') +print('-- Comparing MarCo and MarDecUn. p-values < 0.05 indicate that the algorithms perform differently') for resources in range(20,81,20): marco = list(results[(results['Scheduler'] == 'MarCo') & (results['Resources'] == resources)].avg) @@ -258,6 +265,7 @@ (results['Resources'] == resources)].avg) print(f'Mann-Whitney U test - ({resources} resources).') print(stats.mannwhitneyu(marco, mardec, alternative='two-sided')) +print('\n') # MarCo and MarDec perform differently (p-values < 0.05). diff --git a/stored_results/Analysis of total costs.ipynb b/chameleon_results/Analysis of total costs.ipynb similarity index 94% rename from stored_results/Analysis of total costs.ipynb rename to chameleon_results/Analysis of total costs.ipynb index 5555637..24bb77c 100644 --- a/stored_results/Analysis of total costs.ipynb +++ b/chameleon_results/Analysis of total costs.ipynb @@ -57,6 +57,7 @@ "source": [ "# reads the result file\n", "results = pd.read_csv('results_with_random_costs.csv', comment='#')\n", + "print('Total cost analysis')\n", "print('- Results with random costs')\n", "results.head(9)" ] @@ -78,7 +79,7 @@ "metadata": {}, "outputs": [], "source": [ - "print('-- Generating figures')\n", + "print('-- Generating figures fig-random-10.pdf and fig-random-100.pdf')\n", "\n", "# Sets figure parameters\n", "plt.figure(figsize=(6,5))\n", @@ -145,6 +146,7 @@ "outputs": [], "source": [ "# Checking how many times other schedulers meet the performance of (MC)^2MKP\n", + "print('-- Checking how many times other schedulers meet the performance of (MC)2MKP')\n", "mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True)\n", "for scheduler in schedulers[1:]:\n", " other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True)\n", @@ -171,7 +173,7 @@ "source": [ "# reads the result file\n", "results = pd.read_csv('results_with_increasing_marginal_costs.csv', comment='#')\n", - "print('- Results with increasing marginal costs')\n", + "print('\\n- Results with increasing marginal costs')\n", "results.head(9)" ] }, @@ -190,7 +192,7 @@ "metadata": {}, "outputs": [], "source": [ - "print('-- Generating figures')\n", + "print('-- Generating figures fig-increasing-10.pdf and fig-increasing-100.pdf')\n", "\n", "# Sets figure parameters\n", "plt.figure(figsize=(6,5))\n", @@ -259,6 +261,7 @@ "outputs": [], "source": [ "# Checking how many times other schedulers meet the performance of (MC)^2MKP\n", + "print('-- Checking how many times other schedulers meet the performance of (MC)2MKP')\n", "mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True)\n", "for scheduler in schedulers[1:]:\n", " other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True)\n", @@ -285,7 +288,7 @@ "source": [ "# reads the result file\n", "results = pd.read_csv('results_with_constant_marginal_costs.csv', comment='#')\n", - "print('- Results with constant marginal costs')\n", + "print('\\n- Results with constant marginal costs')\n", "results.head(9)" ] }, @@ -304,7 +307,7 @@ "metadata": {}, "outputs": [], "source": [ - "print('-- Generating figures')\n", + "print('-- Generating figures fig-constant-10.pdf and fig-constant-100.pdf')\n", "\n", "# Sets figure parameters\n", "plt.figure(figsize=(6,5))\n", @@ -373,6 +376,7 @@ "outputs": [], "source": [ "# Checking how many times other schedulers meet the performance of (MC)^2MKP\n", + "print('-- Checking how many times other schedulers meet the performance of (MC)2MKP')\n", "mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True)\n", "for scheduler in schedulers[1:]:\n", " other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True)\n", @@ -399,7 +403,7 @@ "source": [ "# reads the result file\n", "results = pd.read_csv('results_with_decreasing_marginal_costs.csv', comment='#')\n", - "print('- Results with decreasing marginal costs')\n", + "print('\\n- Results with decreasing marginal costs')\n", "results.head(9)" ] }, @@ -418,7 +422,7 @@ "metadata": {}, "outputs": [], "source": [ - "print('-- Generating figures')\n", + "print('-- Generating figures fig-decreasing-10.pdf, fig-decreasing-100.pdf, and fig-decreasing-100-zoom.pdf')\n", "\n", "# Sets figure parameters\n", "plt.figure(figsize=(6,5))\n", @@ -520,6 +524,7 @@ "outputs": [], "source": [ "# Checking how many times other schedulers meet the performance of (MC)^2MKP\n", + "print('-- Checking how many times other schedulers meet the performance of (MC)2MKP')\n", "mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True)\n", "for scheduler in schedulers[1:]:\n", " other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True)\n", @@ -546,7 +551,7 @@ "source": [ "# reads the result file\n", "results = pd.read_csv('results_with_constant_marginal_costs_no_upper_limit.csv', comment='#')\n", - "print('- Results with constant marginal costs and no upper limits')\n", + "print('\\n- Results with constant marginal costs and no upper limits')\n", "results.head(9)" ] }, @@ -565,7 +570,7 @@ "metadata": {}, "outputs": [], "source": [ - "print('-- Generating figures')\n", + "print('-- Generating figures fig-constant-10-no-u.pdf and fig-constant-100-no-u.pdf')\n", "\n", "# Sets figure parameters\n", "plt.figure(figsize=(6,5))\n", @@ -634,6 +639,7 @@ "outputs": [], "source": [ "# Checking how many times other schedulers meet the performance of (MC)^2MKP\n", + "print('-- Checking how many times other schedulers meet the performance of (MC)2MKP')\n", "mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True)\n", "for scheduler in ['MarIn', 'MarCo', 'MarDecUn', 'FedAvg']:\n", " other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True)\n", diff --git a/stored_results/Analysis of total costs.py b/chameleon_results/Analysis of total costs.py similarity index 93% rename from stored_results/Analysis of total costs.py rename to chameleon_results/Analysis of total costs.py index 48fb382..c125b87 100644 --- a/stored_results/Analysis of total costs.py +++ b/chameleon_results/Analysis of total costs.py @@ -39,6 +39,7 @@ # reads the result file results = pd.read_csv('results_with_random_costs.csv', comment='#') +print('Total cost analysis') print('- Results with random costs') results.head(9) @@ -54,7 +55,7 @@ # In[ ]: -print('-- Generating figures') +print('-- Generating figures fig-random-10.pdf and fig-random-100.pdf') # Sets figure parameters plt.figure(figsize=(6,5)) @@ -115,6 +116,7 @@ # Checking how many times other schedulers meet the performance of (MC)^2MKP +print('-- Checking how many times other schedulers meet the performance of (MC)2MKP') mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True) for scheduler in schedulers[1:]: other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True) @@ -133,7 +135,7 @@ # reads the result file results = pd.read_csv('results_with_increasing_marginal_costs.csv', comment='#') -print('- Results with increasing marginal costs') +print('\n- Results with increasing marginal costs') results.head(9) @@ -146,7 +148,7 @@ # In[ ]: -print('-- Generating figures') +print('-- Generating figures fig-increasing-10.pdf and fig-increasing-100.pdf') # Sets figure parameters plt.figure(figsize=(6,5)) @@ -209,6 +211,7 @@ # Checking how many times other schedulers meet the performance of (MC)^2MKP +print('-- Checking how many times other schedulers meet the performance of (MC)2MKP') mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True) for scheduler in schedulers[1:]: other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True) @@ -227,7 +230,7 @@ # reads the result file results = pd.read_csv('results_with_constant_marginal_costs.csv', comment='#') -print('- Results with constant marginal costs') +print('\n- Results with constant marginal costs') results.head(9) @@ -240,7 +243,7 @@ # In[ ]: -print('-- Generating figures') +print('-- Generating figures fig-constant-10.pdf and fig-constant-100.pdf') # Sets figure parameters plt.figure(figsize=(6,5)) @@ -303,6 +306,7 @@ # Checking how many times other schedulers meet the performance of (MC)^2MKP +print('-- Checking how many times other schedulers meet the performance of (MC)2MKP') mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True) for scheduler in schedulers[1:]: other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True) @@ -321,7 +325,7 @@ # reads the result file results = pd.read_csv('results_with_decreasing_marginal_costs.csv', comment='#') -print('- Results with decreasing marginal costs') +print('\n- Results with decreasing marginal costs') results.head(9) @@ -334,7 +338,7 @@ # In[ ]: -print('-- Generating figures') +print('-- Generating figures fig-decreasing-10.pdf, fig-decreasing-100.pdf, and fig-decreasing-100-zoom.pdf') # Sets figure parameters plt.figure(figsize=(6,5)) @@ -427,6 +431,7 @@ # Checking how many times other schedulers meet the performance of (MC)^2MKP +print('-- Checking how many times other schedulers meet the performance of (MC)2MKP') mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True) for scheduler in schedulers[1:]: other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True) @@ -445,7 +450,7 @@ # reads the result file results = pd.read_csv('results_with_constant_marginal_costs_no_upper_limit.csv', comment='#') -print('- Results with constant marginal costs and no upper limits') +print('\n- Results with constant marginal costs and no upper limits') results.head(9) @@ -458,7 +463,7 @@ # In[ ]: -print('-- Generating figures') +print('-- Generating figures fig-constant-10-no-u.pdf and fig-constant-100-no-u.pdf') # Sets figure parameters plt.figure(figsize=(6,5)) @@ -521,6 +526,7 @@ # Checking how many times other schedulers meet the performance of (MC)^2MKP +print('-- Checking how many times other schedulers meet the performance of (MC)2MKP') mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True) for scheduler in ['MarIn', 'MarCo', 'MarDecUn', 'FedAvg']: other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True) diff --git a/chameleon_results/results_of_timing_with_fixed_resources.csv b/chameleon_results/results_of_timing_with_fixed_resources.csv new file mode 100644 index 0000000..95d92b9 --- /dev/null +++ b/chameleon_results/results_of_timing_with_fixed_resources.csv @@ -0,0 +1,1216 @@ +# Description of the experiment: +# +# - We generate the costs to up to 2.000 tasks for 100 resources. +# - All costs follow linear functions (i.e., constant marginal costs) +# with RNG seeds [100..199]. +# - We schedule from 200 to 2.000 tasks in increments of 200. +# - We run (MC)^2MKP, MarIn, MarCo, MarDec, MarDecUn, and FedAvg. +# - All resources have a lower limit of 1. +# - The first half of the resources have no upper limit. +# - The second half has an upper limit of 2*(tasks/resources). +# - Each sample is composed of 5 executions of the schedulers. +# - We get 20 samples for each pair (scheduler, tasks) +# - The order of execution of the different schedulers is +# randomly defined. We set an initial RNG seed = 0 and increase +# it every time we need a new order. +Scheduler,Tasks,Resources,Time +FedAvg,200,100,1.5146999430726282e-05 +MarCo,200,100,0.000594770999668981 +MarIn,200,100,0.0023002489997452358 +MarDecUn,200,100,0.0005269009998301044 +(MC)2MKP,200,100,3.694108846000745 +MarDec,200,100,1.1558015730006446 +MarCo,200,100,0.0006051149994164007 +MarIn,200,100,0.0023028849991533207 +MarDec,200,100,1.1524718330001633 +(MC)2MKP,200,100,3.6925394350000715 +MarDecUn,200,100,0.0005750079999415902 +FedAvg,200,100,1.3830000170855783e-05 +MarDec,200,100,1.176536233000661 +MarIn,200,100,0.002324667999346275 +MarDecUn,200,100,0.00053465699966182 +MarCo,200,100,0.0005962220002402319 +FedAvg,200,100,1.2123999113100581e-05 +(MC)2MKP,200,100,3.7076381639999454 +MarDecUn,200,100,0.000569603000258212 +FedAvg,200,100,1.3529999705497175e-05 +MarDec,200,100,1.164738003999446 +MarIn,200,100,0.002320147001228179 +(MC)2MKP,200,100,3.6504994480001187 +MarCo,200,100,0.0006403670013241936 +MarDec,200,100,1.1566505350001535 +FedAvg,200,100,1.2322998372837901e-05 +MarDecUn,200,100,0.0005281710000417661 +(MC)2MKP,200,100,3.702329509000265 +MarIn,200,100,0.0023433539990946883 +MarCo,200,100,0.0005976630000077421 +FedAvg,200,100,1.2021999282296747e-05 +MarCo,200,100,0.0005942839998169802 +MarDec,200,100,1.149507882000762 +MarIn,200,100,0.00231486100165057 +(MC)2MKP,200,100,3.734016967999196 +MarDecUn,200,100,0.0005232219991739839 +MarDec,200,100,1.1603754479983763 +FedAvg,200,100,1.21770008263411e-05 +(MC)2MKP,200,100,3.669216594000318 +MarDecUn,200,100,0.0005286010000418173 +MarIn,200,100,0.00230338099936489 +MarCo,200,100,0.000603119000516017 +MarDecUn,200,100,0.0005370210001274245 +FedAvg,200,100,1.223599974764511e-05 +(MC)2MKP,200,100,3.69494812099947 +MarCo,200,100,0.00059591100034595 +MarIn,200,100,0.002310462999957963 +MarDec,200,100,1.1632502490010665 +(MC)2MKP,200,100,3.6585892270013574 +MarCo,200,100,0.0005927599995629862 +FedAvg,200,100,1.2230999345774762e-05 +MarIn,200,100,0.002308514000105788 +MarDec,200,100,1.1529918989999715 +MarDecUn,200,100,0.0005261579990474274 +FedAvg,200,100,1.226199856318999e-05 +MarIn,200,100,0.002321541000128491 +MarCo,200,100,0.0005873199988855049 +MarDecUn,200,100,0.0005160740001883823 +(MC)2MKP,200,100,3.682108099999823 +MarDec,200,100,1.155593367999245 +MarCo,200,100,0.0005875919996469747 +FedAvg,200,100,1.2102000255254097e-05 +(MC)2MKP,200,100,3.6469613119988935 +MarDecUn,200,100,0.0005751959997724043 +MarDec,200,100,1.1631991630001721 +MarIn,200,100,0.002309663001142326 +MarDec,200,100,1.1665248799999972 +MarCo,200,100,0.00058885900034511 +FedAvg,200,100,1.2364000212983228e-05 +MarDecUn,200,100,0.0005354240001906874 +(MC)2MKP,200,100,3.6894428770010563 +MarIn,200,100,0.002294889000040712 +(MC)2MKP,200,100,3.662440942998728 +MarDec,200,100,1.1574273879996326 +MarIn,200,100,0.0023404370003845543 +MarCo,200,100,0.0005955420001555467 +FedAvg,200,100,1.1979000191786326e-05 +MarDecUn,200,100,0.0005194759996811626 +MarIn,200,100,0.00230461200044374 +MarDecUn,200,100,0.0005222079998929985 +MarDec,200,100,1.1527519779992872 +FedAvg,200,100,1.2194999726489186e-05 +(MC)2MKP,200,100,3.6850607989999844 +MarCo,200,100,0.0006486959991889307 +FedAvg,200,100,1.4121000276645645e-05 +MarIn,200,100,0.0023377969991997816 +MarCo,200,100,0.0006045860009180615 +MarDec,200,100,1.1573094880004646 +(MC)2MKP,200,100,3.7009041930014064 +MarDecUn,200,100,0.0005319169995345874 +MarDecUn,200,100,0.0005195939993427601 +MarCo,200,100,0.0005836139989696676 +FedAvg,200,100,1.231600072060246e-05 +MarIn,200,100,0.0023033319994283374 +MarDec,200,100,1.1469600229993375 +(MC)2MKP,200,100,3.718217925999852 +(MC)2MKP,200,100,3.684308654999768 +MarCo,200,100,0.0005941210001765285 +MarDecUn,200,100,0.0005320410000422271 +MarDec,200,100,1.1458560969986138 +FedAvg,200,100,1.2199001503176987e-05 +MarIn,200,100,0.002316576999874087 +MarCo,200,100,0.0005943550004303688 +MarDecUn,200,100,0.0005323049990693107 +MarDec,200,100,1.146276518000377 +(MC)2MKP,200,100,3.687502818000212 +FedAvg,200,100,1.3373999536270276e-05 +MarIn,200,100,0.002359994001380983 +FedAvg,200,100,1.227100074174814e-05 +MarDec,200,100,1.1483029820010415 +MarIn,200,100,0.002302491999216727 +(MC)2MKP,200,100,3.71512279000126 +MarDecUn,200,100,0.0005221139999775914 +MarCo,200,100,0.000586472000577487 +MarIn,200,100,0.0023278530006791698 +MarDec,200,100,1.1606750069986447 +MarDecUn,200,100,0.0005284320013743127 +(MC)2MKP,200,100,3.7614808660000563 +MarCo,200,100,0.0005951549992460059 +FedAvg,200,100,1.2313001207076013e-05 +MarIn,400,100,0.00525331100106996 +MarDec,400,100,3.166599228999985 +(MC)2MKP,400,100,14.618489664000663 +FedAvg,400,100,1.3068000043858774e-05 +MarCo,400,100,0.0006355060013447655 +MarDecUn,400,100,0.0005283520004013553 +FedAvg,400,100,1.1954998626606539e-05 +MarCo,400,100,0.0005750939999416005 +MarDecUn,400,100,0.0005206079986237455 +MarDec,400,100,3.1605196989985416 +(MC)2MKP,400,100,14.332728302999385 +MarIn,400,100,0.005312872001013602 +MarIn,400,100,0.005289327000355115 +MarCo,400,100,0.0005716549985663733 +MarDecUn,400,100,0.0005164800004422432 +(MC)2MKP,400,100,14.333711934999883 +MarDec,400,100,3.1568990839987237 +FedAvg,400,100,1.2328000593697652e-05 +FedAvg,400,100,1.2005000826320611e-05 +MarDec,400,100,3.147923652000827 +MarCo,400,100,0.000584530000196537 +MarIn,400,100,0.005277566000586376 +(MC)2MKP,400,100,14.261448789999122 +MarDecUn,400,100,0.0005207880003581522 +MarDec,400,100,3.132321678000153 +FedAvg,400,100,1.226199856318999e-05 +MarIn,400,100,0.005246487999102101 +(MC)2MKP,400,100,14.320519527000215 +MarDecUn,400,100,0.0005666929992003134 +MarCo,400,100,0.0005774050005129538 +FedAvg,400,100,1.4080000255489722e-05 +(MC)2MKP,400,100,14.290950856999189 +MarIn,400,100,0.0052393050009413855 +MarDecUn,400,100,0.0005121240010339534 +MarCo,400,100,0.0005806179997307481 +MarDec,400,100,3.1544863740000437 +MarDec,400,100,3.169458546999522 +MarDecUn,400,100,0.0005200849991524592 +MarCo,400,100,0.0005812050003441982 +MarIn,400,100,0.00529092299984768 +(MC)2MKP,400,100,14.276755962000607 +FedAvg,400,100,1.229800000146497e-05 +MarIn,400,100,0.005217756999627454 +MarCo,400,100,0.0005811450009787222 +FedAvg,400,100,1.229699955729302e-05 +MarDec,400,100,3.156301363000239 +(MC)2MKP,400,100,14.314385075000246 +MarDecUn,400,100,0.0005806169992865762 +MarDecUn,400,100,0.0005247310000413563 +(MC)2MKP,400,100,14.30251693199898 +MarCo,400,100,0.0005931100004090695 +MarDec,400,100,3.1518628250014444 +FedAvg,400,100,1.2352000339888036e-05 +MarIn,400,100,0.005256708998786053 +MarCo,400,100,0.0005881489996681921 +MarDec,400,100,3.183804878999581 +MarIn,400,100,0.005251219999991008 +(MC)2MKP,400,100,14.257341233000261 +MarDecUn,400,100,0.0005208990005485248 +FedAvg,400,100,1.2185000741737895e-05 +MarDecUn,400,100,0.000525372999618412 +MarIn,400,100,0.005281685000227299 +MarCo,400,100,0.0005761610009358265 +(MC)2MKP,400,100,14.463827325000238 +MarDec,400,100,3.1459268469989183 +FedAvg,400,100,1.2407999747665599e-05 +MarDec,400,100,3.1676806329996907 +MarIn,400,100,0.005242645000180346 +MarDecUn,400,100,0.0005135299998073606 +FedAvg,400,100,1.2129999959142879e-05 +(MC)2MKP,400,100,14.308470873998886 +MarCo,400,100,0.0006303589998424286 +FedAvg,400,100,1.3795000995742157e-05 +MarIn,400,100,0.005291849000059301 +(MC)2MKP,400,100,14.30973994300075 +MarCo,400,100,0.0005816319990117336 +MarDec,400,100,3.1337441079995187 +MarDecUn,400,100,0.000530312998307636 +FedAvg,400,100,1.2189000699436292e-05 +MarIn,400,100,0.005279240000163554 +MarDecUn,400,100,0.0005272469988994999 +MarCo,400,100,0.0005834660005348269 +(MC)2MKP,400,100,14.541304657999717 +MarDec,400,100,3.1337128870000015 +MarDecUn,400,100,0.000522584001373616 +(MC)2MKP,400,100,14.230061516000205 +FedAvg,400,100,1.3377000868786126e-05 +MarDec,400,100,3.177277704999142 +MarCo,400,100,0.000580736001211335 +MarIn,400,100,0.005242147000899422 +MarCo,400,100,0.000580715000978671 +(MC)2MKP,400,100,14.3844847879991 +FedAvg,400,100,1.2252001397428103e-05 +MarDecUn,400,100,0.0005110050005896483 +MarDec,400,100,3.128120052999293 +MarIn,400,100,0.005238622001343174 +MarDecUn,400,100,0.0005199790011829464 +MarIn,400,100,0.005249476000244613 +(MC)2MKP,400,100,14.338441230000171 +MarDec,400,100,3.184199234998232 +MarCo,400,100,0.0005853060010849731 +FedAvg,400,100,1.229800000146497e-05 +MarCo,400,100,0.00058283300131734 +MarIn,400,100,0.00525155499963148 +(MC)2MKP,400,100,14.326376257000447 +FedAvg,400,100,1.2616999811143614e-05 +MarDec,400,100,3.156967696999345 +MarDecUn,400,100,0.0005246310010988964 +MarDec,400,100,3.172576492001099 +MarCo,400,100,0.0006017670002620434 +(MC)2MKP,400,100,14.33265233199927 +FedAvg,400,100,1.252999936696142e-05 +MarDecUn,400,100,0.0005219940012466395 +MarIn,400,100,0.005227071000263095 +MarDecUn,400,100,0.000514260998897953 +MarCo,400,100,0.0005727530005970038 +(MC)2MKP,400,100,14.24920637499963 +MarDec,400,100,3.137433350999345 +FedAvg,400,100,1.2471999070839956e-05 +MarIn,400,100,0.005290728999170824 +FedAvg,600,100,1.2084001355106011e-05 +MarIn,600,100,0.008430293000856182 +MarCo,600,100,0.0005856829993717838 +MarDec,600,100,5.229417340000509 +(MC)2MKP,600,100,32.27610593099962 +MarDecUn,600,100,0.000562012000955292 +MarIn,600,100,0.00832374899982824 +MarDec,600,100,5.143288474000656 +MarCo,600,100,0.0005876359991816571 +FedAvg,600,100,1.228099972649943e-05 +MarDecUn,600,100,0.0005301689998304937 +(MC)2MKP,600,100,32.42867459399895 +(MC)2MKP,600,100,32.41729764900083 +MarIn,600,100,0.008374065999305458 +FedAvg,600,100,1.2389000403345563e-05 +MarCo,600,100,0.0005968569985270733 +MarDec,600,100,5.2115916420007125 +MarDecUn,600,100,0.0005265899999358226 +FedAvg,600,100,1.2475999028538354e-05 +MarCo,600,100,0.0005872290003026137 +MarIn,600,100,0.008317732999785221 +MarDecUn,600,100,0.0005206839996390045 +(MC)2MKP,600,100,32.26503279499957 +MarDec,600,100,5.193692261000251 +(MC)2MKP,600,100,32.50877137200041 +MarCo,600,100,0.0005925400000705849 +FedAvg,600,100,1.2259999493835494e-05 +MarIn,600,100,0.008331745000759838 +MarDecUn,600,100,0.0005212610012677033 +MarDec,600,100,5.188542977997713 +MarCo,600,100,0.000583421002374962 +MarIn,600,100,0.008301555000798544 +MarDec,600,100,5.20695945999978 +(MC)2MKP,600,100,32.495336260999466 +FedAvg,600,100,1.3490000128513202e-05 +MarDecUn,600,100,0.0005682989976776298 +MarDecUn,600,100,0.0005248500019661151 +MarIn,600,100,0.008296699001220986 +MarCo,600,100,0.0005825030020787381 +MarDec,600,100,5.219637928999873 +(MC)2MKP,600,100,32.24535052399733 +FedAvg,600,100,1.2249998690094799e-05 +MarCo,600,100,0.0005860300007043406 +MarIn,600,100,0.0083142779985792 +MarDec,600,100,5.195109510001203 +MarDecUn,600,100,0.0005219380000198726 +FedAvg,600,100,1.2464999599615112e-05 +(MC)2MKP,600,100,32.529637697996804 +MarCo,600,100,0.0005896229995414615 +MarDec,600,100,5.213022084997647 +FedAvg,600,100,1.2107000657124445e-05 +MarIn,600,100,0.008278860997961601 +MarDecUn,600,100,0.0005215710007178131 +(MC)2MKP,600,100,32.31819259000258 +MarDecUn,600,100,0.0005561100006161723 +MarDec,600,100,5.240011780999339 +FedAvg,600,100,1.2307002180023119e-05 +MarIn,600,100,0.008242397001595236 +(MC)2MKP,600,100,32.317153309002606 +MarCo,600,100,0.0005898109993722755 +MarDec,600,100,5.217809349000163 +MarCo,600,100,0.0005771639989688993 +MarIn,600,100,0.008399946997087682 +MarDecUn,600,100,0.0005132149999553803 +FedAvg,600,100,1.2010001228190958e-05 +(MC)2MKP,600,100,32.26937758999702 +MarCo,600,100,0.0006379789992934093 +MarDec,600,100,5.189266810000845 +MarDecUn,600,100,0.0005195140001887921 +FedAvg,600,100,1.223499930347316e-05 +(MC)2MKP,600,100,32.443163290001394 +MarIn,600,100,0.008374051998544019 +(MC)2MKP,600,100,32.5815254750014 +MarCo,600,100,0.0005763850022049155 +MarIn,600,100,0.00823384400064242 +MarDecUn,600,100,0.0005156150000402704 +MarDec,600,100,5.162534441999014 +FedAvg,600,100,1.2149001122452319e-05 +MarDec,600,100,5.150028772000951 +(MC)2MKP,600,100,32.52000311499796 +MarCo,600,100,0.0006271279999054968 +FedAvg,600,100,1.4175999240251258e-05 +MarDecUn,600,100,0.0005163339992577676 +MarIn,600,100,0.008236515001044609 +MarDec,600,100,5.217790884998976 +(MC)2MKP,600,100,32.5148608519994 +MarIn,600,100,0.0082891620004375 +MarDecUn,600,100,0.000525696999829961 +MarCo,600,100,0.0005906779988436028 +FedAvg,600,100,1.4014996850164607e-05 +MarDec,600,100,5.181160186002671 +MarIn,600,100,0.008277047996671172 +(MC)2MKP,600,100,32.59545707699726 +MarDecUn,600,100,0.0005209040027693845 +MarCo,600,100,0.0005921449992456473 +FedAvg,600,100,1.2439999409252778e-05 +MarIn,600,100,0.008345420999830822 +MarCo,600,100,0.0005854029986949172 +(MC)2MKP,600,100,32.52911626400237 +MarDecUn,600,100,0.0005693860002793372 +MarDec,600,100,5.191937761999725 +FedAvg,600,100,1.2925000191899016e-05 +MarIn,600,100,0.008371469997655367 +MarDecUn,600,100,0.0005215689998294692 +MarDec,600,100,5.241443237002386 +MarCo,600,100,0.0005859860029886477 +(MC)2MKP,600,100,32.65057747099854 +FedAvg,600,100,1.2289001460885629e-05 +MarCo,600,100,0.000576146998355398 +MarDec,600,100,5.207645440001215 +MarIn,600,100,0.008364345998415956 +FedAvg,600,100,1.2729997251881287e-05 +(MC)2MKP,600,100,32.53236852400005 +MarDecUn,600,100,0.0005616339985863306 +(MC)2MKP,600,100,32.20766971699777 +MarCo,600,100,0.0006385250017046928 +MarDecUn,600,100,0.0005206969981372822 +FedAvg,600,100,1.5679997886763886e-05 +MarDec,600,100,5.184242433999316 +MarIn,600,100,0.008313722999446327 +(MC)2MKP,800,100,57.937547941997764 +MarDec,800,100,7.240412864997779 +MarDecUn,800,100,0.0005175410005904268 +MarCo,800,100,0.0005911839980399236 +MarIn,800,100,0.01129863199821557 +FedAvg,800,100,1.472100120736286e-05 +FedAvg,800,100,1.2016000255243853e-05 +MarIn,800,100,0.011176850999618182 +(MC)2MKP,800,100,57.55704956000045 +MarDec,800,100,7.224374818000797 +MarCo,800,100,0.0005914770008530468 +MarDecUn,800,100,0.0005196079982852098 +MarDec,800,100,7.229716879999614 +FedAvg,800,100,1.2288001016713679e-05 +MarDecUn,800,100,0.0005227010005910415 +MarIn,800,100,0.01132976100052474 +(MC)2MKP,800,100,58.16576265099866 +MarCo,800,100,0.0006454219983424991 +MarIn,800,100,0.011281508999672951 +(MC)2MKP,800,100,58.827741658999 +MarCo,800,100,0.0006268270008149557 +MarDecUn,800,100,0.0005253150011412799 +FedAvg,800,100,1.3714998203795403e-05 +MarDec,800,100,7.19837346499844 +MarCo,800,100,0.0005807350025861524 +MarDecUn,800,100,0.0005223549997026566 +FedAvg,800,100,1.223299841512926e-05 +(MC)2MKP,800,100,57.50367173400082 +MarIn,800,100,0.011332320998917567 +MarDec,800,100,7.238405826999951 +MarCo,800,100,0.0005846320018463302 +MarDecUn,800,100,0.0005128449993208051 +MarDec,800,100,7.218911085001309 +MarIn,800,100,0.011276150999037782 +(MC)2MKP,800,100,57.12755919999836 +FedAvg,800,100,1.3630000466946512e-05 +(MC)2MKP,800,100,57.86393588699866 +MarIn,800,100,0.011279508002189687 +MarCo,800,100,0.000595511999563314 +MarDecUn,800,100,0.0005212020005274098 +FedAvg,800,100,1.24100006360095e-05 +MarDec,800,100,7.2065975360019365 +(MC)2MKP,800,100,57.63413406499967 +MarDec,800,100,7.288081115002569 +FedAvg,800,100,1.2308002624195069e-05 +MarIn,800,100,0.011284250998869538 +MarCo,800,100,0.0005917589987802785 +MarDecUn,800,100,0.0005236899996816646 +MarIn,800,100,0.01127343999905861 +MarDecUn,800,100,0.0005186170019442216 +(MC)2MKP,800,100,58.24725916999887 +FedAvg,800,100,1.3574997865362093e-05 +MarDec,800,100,7.169396594999853 +MarCo,800,100,0.0005899329989915714 +(MC)2MKP,800,100,57.50785185299901 +MarDec,800,100,7.290748588999122 +MarCo,800,100,0.0005895240028621629 +FedAvg,800,100,1.2403997970977798e-05 +MarIn,800,100,0.011339378001139266 +MarDecUn,800,100,0.0005275060029816814 +MarIn,800,100,0.011157801996887429 +FedAvg,800,100,1.2755001080222428e-05 +MarDecUn,800,100,0.000526956999237882 +MarDec,800,100,7.283123495999462 +(MC)2MKP,800,100,57.65316757800247 +MarCo,800,100,0.0006291559984674677 +MarIn,800,100,0.011315758001728682 +MarCo,800,100,0.0005735019985877443 +(MC)2MKP,800,100,57.95260179199977 +MarDec,800,100,7.146017605999077 +FedAvg,800,100,1.2510001397458836e-05 +MarDecUn,800,100,0.0005208660004427657 +FedAvg,800,100,1.2405002053128555e-05 +MarIn,800,100,0.011390817002393305 +MarDec,800,100,7.183801277002203 +MarCo,800,100,0.000581409996811999 +MarDecUn,800,100,0.0005126709984324407 +(MC)2MKP,800,100,57.97586058199886 +MarDec,800,100,7.239538917001482 +MarIn,800,100,0.011302576000161935 +MarDecUn,800,100,0.0005136299987498205 +(MC)2MKP,800,100,57.61571148300209 +FedAvg,800,100,1.3722001312999055e-05 +MarCo,800,100,0.0006423289996746462 +FedAvg,800,100,1.2477001291699708e-05 +(MC)2MKP,800,100,57.66104772100152 +MarCo,800,100,0.0006215449975570664 +MarDec,800,100,7.195762370000011 +MarIn,800,100,0.011290233000181615 +MarDecUn,800,100,0.0005288009997457266 +MarCo,800,100,0.0005751580029027537 +MarDec,800,100,7.229342193000775 +MarIn,800,100,0.011339789001794998 +MarDecUn,800,100,0.0005193010001676157 +FedAvg,800,100,1.2386000889819115e-05 +(MC)2MKP,800,100,58.369285134001984 +MarDec,800,100,7.216980749999493 +FedAvg,800,100,1.2354001228231937e-05 +MarDecUn,800,100,0.0005209449991525616 +MarCo,800,100,0.000590822000958724 +(MC)2MKP,800,100,57.36542567500146 +MarIn,800,100,0.011140358001284767 +MarCo,800,100,0.0005722660025639925 +(MC)2MKP,800,100,58.031149654998444 +MarIn,800,100,0.01127024400193477 +MarDecUn,800,100,0.0005107339966343716 +FedAvg,800,100,1.2407999747665599e-05 +MarDec,800,100,7.205299745997763 +MarDecUn,800,100,0.0005226850007602479 +MarDec,800,100,7.2702043220015184 +MarCo,800,100,0.0005768190021626651 +MarIn,800,100,0.011305915002594702 +(MC)2MKP,800,100,57.41470611399927 +FedAvg,800,100,1.3790999219054356e-05 +MarIn,800,100,0.011285159002 +MarDec,800,100,7.209742602997721 +(MC)2MKP,800,100,57.76476208399981 +MarDecUn,800,100,0.0005563889972108882 +MarCo,800,100,0.0005935680019319989 +FedAvg,800,100,1.4019999071024358e-05 +(MC)2MKP,1000,100,90.31721983899843 +MarIn,1000,100,0.014391860997420736 +FedAvg,1000,100,1.2175998563179746e-05 +MarCo,1000,100,0.0005922199998167343 +MarDecUn,1000,100,0.0005312339999363758 +MarDec,1000,100,9.177698740000778 +FedAvg,1000,100,1.2483000318752602e-05 +MarDec,1000,100,9.212091542998678 +MarCo,1000,100,0.0005792529991595075 +(MC)2MKP,1000,100,90.05837783599782 +MarIn,1000,100,0.01423447300112457 +MarDecUn,1000,100,0.0005289209984766785 +(MC)2MKP,1000,100,89.88839413700043 +MarDec,1000,100,9.181229128000268 +MarIn,1000,100,0.014277753998612752 +FedAvg,1000,100,1.2560001778183505e-05 +MarCo,1000,100,0.0005803900021419395 +MarDecUn,1000,100,0.0005263009988993872 +MarIn,1000,100,0.014298294998297933 +FedAvg,1000,100,1.2485001207096502e-05 +MarDec,1000,100,9.147801493996667 +MarDecUn,1000,100,0.0005217159996391274 +(MC)2MKP,1000,100,90.01682262600298 +MarCo,1000,100,0.0006330090000119526 +FedAvg,1000,100,1.3920001947553828e-05 +MarDecUn,1000,100,0.0005202300017117523 +(MC)2MKP,1000,100,91.05844109999816 +MarDec,1000,100,9.203223753000202 +MarIn,1000,100,0.014209465000021737 +MarCo,1000,100,0.0005819830003019888 +FedAvg,1000,100,1.2677999620791525e-05 +MarDecUn,1000,100,0.0005219090016908012 +MarDec,1000,100,9.108689431002858 +MarIn,1000,100,0.014300122998974985 +(MC)2MKP,1000,100,89.49763382299716 +MarCo,1000,100,0.0006269929981499445 +MarCo,1000,100,0.0005712539968953934 +MarIn,1000,100,0.014269577997765737 +MarDecUn,1000,100,0.0005228970003372524 +FedAvg,1000,100,1.2484000762924552e-05 +(MC)2MKP,1000,100,90.3440330150006 +MarDec,1000,100,9.31426756899964 +MarIn,1000,100,0.014189776000421261 +(MC)2MKP,1000,100,90.54825132599944 +MarCo,1000,100,0.0006291870013228618 +MarDec,1000,100,9.19687977599824 +MarDecUn,1000,100,0.0005177090024517383 +FedAvg,1000,100,1.2423999578459188e-05 +MarCo,1000,100,0.0005820650003443006 +MarDec,1000,100,9.278806726000767 +MarIn,1000,100,0.014251040996896336 +MarDecUn,1000,100,0.0005218449987296481 +FedAvg,1000,100,1.22580022434704e-05 +(MC)2MKP,1000,100,90.4219120710004 +(MC)2MKP,1000,100,91.40830772900154 +FedAvg,1000,100,1.3337001291802153e-05 +MarCo,1000,100,0.0006256970009417273 +MarIn,1000,100,0.01428629400106729 +MarDec,1000,100,9.18828090199895 +MarDecUn,1000,100,0.0005158870008017402 +MarIn,1000,100,0.014316933000372956 +MarDec,1000,100,9.18720677699821 +(MC)2MKP,1000,100,91.07509467999989 +FedAvg,1000,100,1.4957000530557707e-05 +MarCo,1000,100,0.0006361380001180805 +MarDecUn,1000,100,0.0005216799982008524 +MarDecUn,1000,100,0.0005157819978194311 +(MC)2MKP,1000,100,89.6109899990006 +MarIn,1000,100,0.014377826999407262 +MarDec,1000,100,9.222620355998515 +FedAvg,1000,100,1.2680000509135425e-05 +MarCo,1000,100,0.0005749689989897888 +(MC)2MKP,1000,100,89.791898331001 +MarIn,1000,100,0.014338045999465976 +MarDec,1000,100,9.17786126899955 +MarDecUn,1000,100,0.0005308279978635255 +FedAvg,1000,100,1.2612999853445217e-05 +MarCo,1000,100,0.0005906760015932377 +MarCo,1000,100,0.0005813940006191842 +MarIn,1000,100,0.01430853300189483 +MarDec,1000,100,9.202892363999126 +(MC)2MKP,1000,100,90.57282488800047 +MarDecUn,1000,100,0.0005723989997932222 +FedAvg,1000,100,1.5677996998419985e-05 +FedAvg,1000,100,1.2137999874539673e-05 +MarDec,1000,100,9.095834861000185 +MarIn,1000,100,0.01442034299907391 +(MC)2MKP,1000,100,89.95878593199814 +MarCo,1000,100,0.0006310390017461032 +MarDecUn,1000,100,0.0005219720005698036 +(MC)2MKP,1000,100,90.10434526499739 +FedAvg,1000,100,1.3811000826535746e-05 +MarDec,1000,100,9.148190413998236 +MarCo,1000,100,0.0005984670024190564 +MarDecUn,1000,100,0.0005221650026214775 +MarIn,1000,100,0.014542640001309337 +MarIn,1000,100,0.014322679002361838 +MarDecUn,1000,100,0.0005192810021981131 +(MC)2MKP,1000,100,90.91073176899954 +FedAvg,1000,100,1.5649999113520607e-05 +MarCo,1000,100,0.0006341490006889217 +MarDec,1000,100,9.143101853998814 +(MC)2MKP,1000,100,89.50727746399934 +FedAvg,1000,100,1.4461998944170773e-05 +MarDec,1000,100,9.265867771999183 +MarIn,1000,100,0.014319186000648187 +MarDecUn,1000,100,0.0005248100023891311 +MarCo,1000,100,0.0005816799966851249 +MarDecUn,1000,100,0.0005252509981801268 +MarDec,1000,100,9.260145628999453 +MarIn,1000,100,0.014195556999766268 +(MC)2MKP,1000,100,89.81468006400246 +FedAvg,1000,100,1.483699816162698e-05 +MarCo,1000,100,0.0006362650019582361 +MarCo,1000,100,0.0005813080024381634 +MarDec,1000,100,9.282489647997863 +(MC)2MKP,1000,100,89.67561097600264 +FedAvg,1000,100,1.5502999303862453e-05 +MarDecUn,1000,100,0.0005670730024576187 +MarIn,1000,100,0.014430848001211416 +MarIn,1200,100,0.017320634000498103 +MarCo,1200,100,0.000591392999922391 +MarDec,1200,100,11.367296472999442 +MarDecUn,1200,100,0.0005171369994059205 +FedAvg,1200,100,1.2425000022631139e-05 +(MC)2MKP,1200,100,129.97584460399958 +(MC)2MKP,1200,100,128.69841403699684 +FedAvg,1200,100,1.4815999747952446e-05 +MarDec,1200,100,11.251152595999883 +MarCo,1200,100,0.0005907590020797215 +MarIn,1200,100,0.01738903899968136 +MarDecUn,1200,100,0.000535379000211833 +MarIn,1200,100,0.01740884000173537 +FedAvg,1200,100,1.2344000424491242e-05 +MarDec,1200,100,11.170270440998138 +MarCo,1200,100,0.0006276249987422489 +MarDecUn,1200,100,0.0005184639994695317 +(MC)2MKP,1200,100,128.93259356099952 +MarCo,1200,100,0.0006210899991856422 +(MC)2MKP,1200,100,130.6781312210005 +FedAvg,1200,100,1.5648001863155514e-05 +MarDec,1200,100,11.163408058997447 +MarDecUn,1200,100,0.0005222709987720009 +MarIn,1200,100,0.017283307999605313 +MarDecUn,1200,100,0.0005220729981374461 +(MC)2MKP,1200,100,129.4979377139971 +MarCo,1200,100,0.000636495002254378 +MarDec,1200,100,11.222471094999491 +MarIn,1200,100,0.01726184600192937 +FedAvg,1200,100,1.4393001038115472e-05 +MarDec,1200,100,11.192728113001067 +MarCo,1200,100,0.0005920679977862164 +MarDecUn,1200,100,0.0005309129992383532 +FedAvg,1200,100,1.3963999663246796e-05 +MarIn,1200,100,0.017241790003026836 +(MC)2MKP,1200,100,129.42564315300115 +MarDec,1200,100,11.234976517000177 +FedAvg,1200,100,1.3856999430572614e-05 +(MC)2MKP,1200,100,129.73061339000196 +MarDecUn,1200,100,0.0005577949996222742 +MarIn,1200,100,0.017399962998752017 +MarCo,1200,100,0.0005793519994767848 +FedAvg,1200,100,1.4121000276645645e-05 +MarDec,1200,100,11.208071244000166 +MarDecUn,1200,100,0.0005269360008242074 +MarCo,1200,100,0.0005719930013583507 +MarIn,1200,100,0.017236945997865405 +(MC)2MKP,1200,100,130.76048819599964 +MarDec,1200,100,12.343853101003333 +(MC)2MKP,1200,100,130.60433679100242 +MarCo,1200,100,0.0006350410003506113 +FedAvg,1200,100,1.673099905019626e-05 +MarIn,1200,100,0.017321757000900107 +MarDecUn,1200,100,0.0005270789988571778 +MarCo,1200,100,0.0005763059998571407 +MarDec,1200,100,11.26574622300177 +MarIn,1200,100,0.01726259399947594 +MarDecUn,1200,100,0.0005113639999763109 +(MC)2MKP,1200,100,129.44784751200132 +FedAvg,1200,100,1.4340999769046903e-05 +FedAvg,1200,100,1.2142001651227474e-05 +MarCo,1200,100,0.0006232909981918056 +MarDec,1200,100,11.251823121998314 +MarIn,1200,100,0.01741302799928235 +MarDecUn,1200,100,0.0005233429983491078 +(MC)2MKP,1200,100,129.74797187399963 +MarIn,1200,100,0.01748877699719742 +MarCo,1200,100,0.0005918399983784184 +MarDecUn,1200,100,0.0005250660033198074 +(MC)2MKP,1200,100,131.2331974359986 +FedAvg,1200,100,1.6133999451994896e-05 +MarDec,1200,100,11.111059702998318 +FedAvg,1200,100,1.2169999536126852e-05 +MarIn,1200,100,0.0172542969994538 +MarCo,1200,100,0.0005886670005565975 +(MC)2MKP,1200,100,129.65357978500106 +MarDecUn,1200,100,0.0005710590012313332 +MarDec,1200,100,11.195680110999092 +MarIn,1200,100,0.01733226500073215 +(MC)2MKP,1200,100,129.61770254400108 +MarDecUn,1200,100,0.0005607709972537123 +MarCo,1200,100,0.000585560999752488 +MarDec,1200,100,11.22422503299822 +FedAvg,1200,100,1.3699998817173764e-05 +(MC)2MKP,1200,100,129.63075032399865 +MarIn,1200,100,0.017267555002035806 +MarDec,1200,100,11.309698635999666 +FedAvg,1200,100,1.3346001651370898e-05 +MarDecUn,1200,100,0.0005277220006973948 +MarCo,1200,100,0.0005853109978488646 +MarIn,1200,100,0.017267955001443624 +MarDec,1200,100,11.177631346003182 +FedAvg,1200,100,1.2262000382179394e-05 +MarDecUn,1200,100,0.0005341240030247718 +(MC)2MKP,1200,100,129.4821548200016 +MarCo,1200,100,0.0006447320010920521 +MarDec,1200,100,11.295986985998752 +MarIn,1200,100,0.017433007000363432 +MarCo,1200,100,0.0005796570003440138 +(MC)2MKP,1200,100,129.72740595199866 +MarDecUn,1200,100,0.0005714379985874984 +FedAvg,1200,100,1.6763002349762246e-05 +MarDec,1200,100,11.208558157999505 +MarCo,1200,100,0.0005751169992436189 +MarDecUn,1200,100,0.0005268810018606018 +MarIn,1200,100,0.018342149000091013 +FedAvg,1200,100,1.2299999070819467e-05 +(MC)2MKP,1200,100,129.47189223699752 +MarCo,1200,100,0.0006267240023589693 +FedAvg,1200,100,1.560800228617154e-05 +MarDec,1200,100,11.093043526998372 +MarIn,1200,100,0.017506100997707108 +MarDecUn,1200,100,0.0005136340005265083 +(MC)2MKP,1200,100,131.03009239200037 +MarIn,1200,100,0.0173904510011198 +(MC)2MKP,1200,100,131.5650692880008 +MarDecUn,1200,100,0.0005708749995392282 +MarDec,1200,100,11.194855005000136 +FedAvg,1200,100,1.3683998986380175e-05 +MarCo,1200,100,0.000593794000451453 +MarIn,1400,100,0.02029679499901249 +MarDec,1400,100,13.08227615999931 +FedAvg,1400,100,1.4209002983989194e-05 +MarDecUn,1400,100,0.0005246879991318565 +MarCo,1400,100,0.0005758320003224071 +(MC)2MKP,1400,100,178.42642578999948 +FedAvg,1400,100,1.4518998796120286e-05 +MarIn,1400,100,0.02026493700032006 +MarDec,1400,100,13.06873030800125 +MarDecUn,1400,100,0.0005285849983920343 +(MC)2MKP,1400,100,177.7019303499983 +MarCo,1400,100,0.0006391950009856373 +MarIn,1400,100,0.020415764000063064 +MarDecUn,1400,100,0.0005246599976089783 +(MC)2MKP,1400,100,177.79194289400039 +MarDec,1400,100,13.365777358998457 +FedAvg,1400,100,1.3670000043930486e-05 +MarCo,1400,100,0.0005898649978917092 +MarIn,1400,100,0.02018763800151646 +MarDecUn,1400,100,0.0005238660014583729 +MarDec,1400,100,13.259735483999975 +(MC)2MKP,1400,100,179.78170753700033 +MarCo,1400,100,0.0006358829996315762 +FedAvg,1400,100,1.7154998204205185e-05 +(MC)2MKP,1400,100,177.31788408599823 +FedAvg,1400,100,1.5790999896125868e-05 +MarCo,1400,100,0.0006331449985736981 +MarDecUn,1400,100,0.0005196040001465008 +MarIn,1400,100,0.020381083002575906 +MarDec,1400,100,13.292381699000543 +(MC)2MKP,1400,100,177.27707173199815 +MarIn,1400,100,0.02035059099944192 +MarDec,1400,100,13.251452181000786 +MarDecUn,1400,100,0.0005204199987929314 +MarCo,1400,100,0.0005881990000489168 +FedAvg,1400,100,1.436899765394628e-05 +MarDec,1400,100,13.297730222999235 +FedAvg,1400,100,1.204099680762738e-05 +(MC)2MKP,1400,100,178.85870627199984 +MarDecUn,1400,100,0.0005614020010398235 +MarIn,1400,100,0.019779584999923827 +MarCo,1400,100,0.0005830590016557835 +FedAvg,1400,100,1.4173001545714214e-05 +MarDecUn,1400,100,0.0005292999994708225 +(MC)2MKP,1400,100,175.27853121799853 +MarCo,1400,100,0.000630744998488808 +MarDec,1400,100,13.271154548001505 +MarIn,1400,100,0.020427234998351196 +(MC)2MKP,1400,100,177.764776901 +MarDec,1400,100,13.2273529769991 +FedAvg,1400,100,1.3937002222519368e-05 +MarIn,1400,100,0.020082388997252565 +MarDecUn,1400,100,0.000524289996974403 +MarCo,1400,100,0.0005896169968764298 +MarIn,1400,100,0.020492788000410656 +MarDec,1400,100,13.03060643900244 +MarDecUn,1400,100,0.0005222190011409111 +MarCo,1400,100,0.0005788620001112577 +FedAvg,1400,100,1.4228000509319827e-05 +(MC)2MKP,1400,100,176.92211667599986 +(MC)2MKP,1400,100,177.09940898499917 +FedAvg,1400,100,1.5817000530660152e-05 +MarDec,1400,100,13.26574942499792 +MarCo,1400,100,0.0005795200013380963 +MarDecUn,1400,100,0.0005227250003372319 +MarIn,1400,100,0.02112298800057033 +(MC)2MKP,1400,100,175.3459577169997 +MarCo,1400,100,0.0006491879976238124 +MarIn,1400,100,0.02016660600202158 +FedAvg,1400,100,1.5510999219259247e-05 +MarDec,1400,100,13.253793665000558 +MarDecUn,1400,100,0.0005242369988991413 +MarDecUn,1400,100,0.0005183419998502359 +(MC)2MKP,1400,100,176.63157571100237 +MarDec,1400,100,13.176446774999931 +FedAvg,1400,100,1.3540997315431014e-05 +MarCo,1400,100,0.0005894999994779937 +MarIn,1400,100,0.02028889400025946 +MarCo,1400,100,0.0005864209997525904 +MarDecUn,1400,100,0.0005198559993004892 +FedAvg,1400,100,1.2292999599594623e-05 +MarDec,1400,100,13.340165551999235 +(MC)2MKP,1400,100,178.69968716700168 +MarIn,1400,100,0.02017353300107061 +MarIn,1400,100,0.020235675998264924 +MarDec,1400,100,13.246665772003325 +MarCo,1400,100,0.0005936950001341756 +FedAvg,1400,100,1.4533998182741925e-05 +(MC)2MKP,1400,100,176.38004780400297 +MarDecUn,1400,100,0.0005669739985023625 +MarCo,1400,100,0.0005868049993296154 +MarIn,1400,100,0.02027885000279639 +(MC)2MKP,1400,100,177.36766082099712 +MarDec,1400,100,13.103844605000631 +FedAvg,1400,100,1.3896999007556587e-05 +MarDecUn,1400,100,0.0005150779979885556 +MarDec,1400,100,13.199169067000184 +(MC)2MKP,1400,100,179.54282560399952 +MarCo,1400,100,0.0006413570008589886 +MarDecUn,1400,100,0.0005207930007600226 +FedAvg,1400,100,1.6912999853957444e-05 +MarIn,1400,100,0.02046826099831378 +MarDec,1400,100,13.205649234001612 +MarIn,1400,100,0.020272376998036634 +MarCo,1400,100,0.0005862230027560145 +(MC)2MKP,1400,100,175.94669679999788 +MarDecUn,1400,100,0.0005623170000035316 +FedAvg,1400,100,1.6941998183028772e-05 +(MC)2MKP,1400,100,175.60390314899996 +FedAvg,1400,100,1.590000101714395e-05 +MarDecUn,1400,100,0.0005711409976356663 +MarDec,1400,100,13.228833238001243 +MarIn,1400,100,0.020427481998922303 +MarCo,1400,100,0.0005837620010424871 +MarCo,1400,100,0.0005808860005345196 +MarIn,1400,100,0.020377319000544958 +MarDecUn,1400,100,0.0005152289995749015 +FedAvg,1400,100,1.2456999684218317e-05 +(MC)2MKP,1400,100,175.29710946200066 +MarDec,1400,100,13.229564219000167 +MarDecUn,1600,100,0.0005188600007386412 +MarCo,1600,100,0.0005796830009785481 +MarIn,1600,100,0.023418673998094164 +MarDec,1600,100,15.11831717899986 +FedAvg,1600,100,1.357600194751285e-05 +(MC)2MKP,1600,100,228.86156809099703 +MarDec,1600,100,15.12647165999806 +(MC)2MKP,1600,100,232.1331197110012 +MarCo,1600,100,0.0006302619985945057 +MarDecUn,1600,100,0.0005215720011619851 +FedAvg,1600,100,1.700899883871898e-05 +MarIn,1600,100,0.023294129001442343 +MarIn,1600,100,0.023235304000991164 +MarCo,1600,100,0.0005736940001952462 +(MC)2MKP,1600,100,232.46839015099977 +MarDec,1600,100,16.396454644996993 +MarDecUn,1600,100,0.0005835619995195884 +FedAvg,1600,100,1.5507001080550253e-05 +MarDecUn,1600,100,0.0005749890005972702 +(MC)2MKP,1600,100,230.36643889600236 +MarCo,1600,100,0.0006336779988487251 +MarDec,1600,100,15.204558086999896 +FedAvg,1600,100,1.3643002603203058e-05 +MarIn,1600,100,0.023165432998212054 +MarIn,1600,100,0.023168916999566136 +MarDecUn,1600,100,0.0005200799969315995 +FedAvg,1600,100,1.2427000910975039e-05 +(MC)2MKP,1600,100,229.8651501380009 +MarCo,1600,100,0.0006273200015129987 +MarDec,1600,100,15.292885078000836 +(MC)2MKP,1600,100,231.68177640100112 +MarIn,1600,100,0.02376391299912939 +MarDec,1600,100,15.40184754099755 +MarDecUn,1600,100,0.0005249409987300169 +MarCo,1600,100,0.0005944750009803101 +FedAvg,1600,100,1.4318000467028469e-05 +FedAvg,1600,100,1.2229000276420265e-05 +(MC)2MKP,1600,100,228.849695308003 +MarDec,1600,100,15.335014423999382 +MarIn,1600,100,0.023126021998905344 +MarCo,1600,100,0.0005789410024590325 +MarDecUn,1600,100,0.0005126760006533004 +FedAvg,1600,100,1.4230001397663727e-05 +MarDecUn,1600,100,0.0005129199998918921 +MarDec,1600,100,15.173255360998155 +MarCo,1600,100,0.0005684830030077137 +MarIn,1600,100,0.023408491997543024 +(MC)2MKP,1600,100,230.351927574 +(MC)2MKP,1600,100,230.81205642200075 +MarIn,1600,100,0.023200059000373585 +MarCo,1600,100,0.0005844339975737967 +MarDec,1600,100,15.262901016998512 +FedAvg,1600,100,1.3402997865341604e-05 +MarDecUn,1600,100,0.0005126379983266816 +(MC)2MKP,1600,100,229.48867773599704 +MarCo,1600,100,0.0006400920028681867 +MarDecUn,1600,100,0.0005226630019024014 +FedAvg,1600,100,1.8871996871894225e-05 +MarDec,1600,100,15.258451919999061 +MarIn,1600,100,0.023447154002496973 +FedAvg,1600,100,1.2434997188393027e-05 +(MC)2MKP,1600,100,229.28239369100265 +MarIn,1600,100,0.02321086799929617 +MarDecUn,1600,100,0.0005122360016684979 +MarCo,1600,100,0.0005761920001532417 +MarDec,1600,100,15.377450616000715 +MarDec,1600,100,15.191663412999333 +FedAvg,1600,100,1.3630000466946512e-05 +MarDecUn,1600,100,0.0005304529986460693 +MarIn,1600,100,0.023105924999981653 +MarCo,1600,100,0.0005956040004093666 +(MC)2MKP,1600,100,233.51644840399968 +FedAvg,1600,100,1.4713001291966066e-05 +(MC)2MKP,1600,100,229.3422393899964 +MarCo,1600,100,0.0006406510001397692 +MarIn,1600,100,0.023638865000975784 +MarDecUn,1600,100,0.0005282010024529882 +MarDec,1600,100,15.143968699998368 +MarDecUn,1600,100,0.0005217229991103522 +MarCo,1600,100,0.0005845609994139522 +MarIn,1600,100,0.023314352001762018 +FedAvg,1600,100,1.4196997653925791e-05 +(MC)2MKP,1600,100,229.76759446099823 +MarDec,1600,100,15.296646991999296 +(MC)2MKP,1600,100,231.27243996799734 +FedAvg,1600,100,1.562400211696513e-05 +MarCo,1600,100,0.0006272259997786023 +MarDecUn,1600,100,0.0005207940012041945 +MarDec,1600,100,15.22697127200081 +MarIn,1600,100,0.02311821400144254 +MarDecUn,1600,100,0.0005200009982218035 +FedAvg,1600,100,1.2334003258729354e-05 +(MC)2MKP,1600,100,229.76242066900159 +MarCo,1600,100,0.0006349450013658497 +MarIn,1600,100,0.023334221998084104 +MarDec,1600,100,15.21016205399792 +(MC)2MKP,1600,100,230.81942205999803 +MarDecUn,1600,100,0.0005585859980783425 +MarIn,1600,100,0.023440508000931004 +MarCo,1600,100,0.0005807020024803933 +FedAvg,1600,100,1.560500095365569e-05 +MarDec,1600,100,15.354659374999756 +MarIn,1600,100,0.023077455000020564 +FedAvg,1600,100,1.2374002835713327e-05 +MarDec,1600,100,15.174344351002219 +MarCo,1600,100,0.000575112000660738 +MarDecUn,1600,100,0.0005191140007809736 +(MC)2MKP,1600,100,229.3409588079994 +MarDecUn,1600,100,0.0005712740021408536 +MarDec,1600,100,15.278927942999871 +MarIn,1600,100,0.023271670001122402 +MarCo,1600,100,0.0005869829983566888 +(MC)2MKP,1600,100,231.66853535100017 +FedAvg,1600,100,1.4883000403642654e-05 +MarCo,1600,100,0.0006252649982343428 +FedAvg,1600,100,1.2198001059005037e-05 +MarIn,1600,100,0.023345502999291057 +(MC)2MKP,1600,100,230.68765539800006 +MarDec,1600,100,15.203036536000582 +MarDecUn,1600,100,0.0005132609985594172 +MarDec,1800,100,18.286052891999134 +MarIn,1800,100,0.028132437000749633 +FedAvg,1800,100,1.5055997209856287e-05 +MarDecUn,1800,100,0.0005580639990512282 +MarCo,1800,100,0.0006268789984460454 +(MC)2MKP,1800,100,296.5763208169992 +MarCo,1800,100,0.0006315409991657361 +MarIn,1800,100,0.026403767998999683 +(MC)2MKP,1800,100,290.8087495209984 +MarDec,1800,100,17.209355283997866 +FedAvg,1800,100,1.3682001736015081e-05 +MarDecUn,1800,100,0.0005180460029805545 +MarDec,1800,100,17.24545307199878 +(MC)2MKP,1800,100,293.7409593150005 +MarIn,1800,100,0.026544119999016402 +MarCo,1800,100,0.0005806240005767904 +FedAvg,1800,100,1.579300078446977e-05 +MarDecUn,1800,100,0.0005198350008868147 +MarDec,1800,100,17.28950891999557 +FedAvg,1800,100,1.2535005225799978e-05 +(MC)2MKP,1800,100,297.1196114319973 +MarDecUn,1800,100,0.000574233999941498 +MarCo,1800,100,0.0005896190050407313 +MarIn,1800,100,0.02642321399616776 +MarDecUn,1800,100,0.0005198440048843622 +MarDec,1800,100,17.32749790899834 +FedAvg,1800,100,1.3466000382322818e-05 +MarIn,1800,100,0.026500649997615255 +MarCo,1800,100,0.0005907649974687956 +(MC)2MKP,1800,100,290.6187198300031 +MarIn,1800,100,0.026433967999764718 +MarDecUn,1800,100,0.0005255579962977208 +MarDec,1800,100,17.178365885003586 +FedAvg,1800,100,1.3739001587964594e-05 +MarCo,1800,100,0.000597954000113532 +(MC)2MKP,1800,100,294.00722400400264 +(MC)2MKP,1800,100,295.5894244860028 +MarDecUn,1800,100,0.0006446589977713302 +MarCo,1800,100,0.0006460909935412928 +MarDec,1800,100,17.430938594996405 +MarIn,1800,100,0.0265361040001153 +FedAvg,1800,100,1.4275996363721788e-05 +MarDec,1800,100,17.089442217002215 +MarDecUn,1800,100,0.0005158210042282008 +(MC)2MKP,1800,100,295.27620889300306 +FedAvg,1800,100,1.5746998542454094e-05 +MarCo,1800,100,0.0006326649963739328 +MarIn,1800,100,0.0261902190031833 +(MC)2MKP,1800,100,290.012961446002 +MarCo,1800,100,0.0006458420029957779 +MarDecUn,1800,100,0.000533286998688709 +MarDec,1800,100,17.41559530399536 +MarIn,1800,100,0.026169535994995385 +FedAvg,1800,100,1.4206001651473343e-05 +MarCo,1800,100,0.0005853490001754835 +FedAvg,1800,100,1.2190001143608242e-05 +(MC)2MKP,1800,100,292.4384543299966 +MarDec,1800,100,17.406225077000272 +MarIn,1800,100,0.026332373003242537 +MarDecUn,1800,100,0.0005186959970160387 +MarIn,1800,100,0.026370592997409403 +FedAvg,1800,100,1.4458993973676115e-05 +MarDecUn,1800,100,0.0005156889965292066 +(MC)2MKP,1800,100,292.41353766900284 +MarDec,1800,100,17.484400027002266 +MarCo,1800,100,0.0005863639962626621 +MarDecUn,1800,100,0.0005202300017117523 +MarDec,1800,100,18.067849512000976 +MarIn,1800,100,0.026610513996274676 +MarCo,1800,100,0.0005810729999211617 +(MC)2MKP,1800,100,291.7729647659944 +FedAvg,1800,100,1.5983001503627747e-05 +(MC)2MKP,1800,100,290.246525923998 +MarDec,1800,100,17.114580114001 +MarDecUn,1800,100,0.0005196960046305321 +FedAvg,1800,100,1.4211000234354287e-05 +MarIn,1800,100,0.026195138998446055 +MarCo,1800,100,0.0006065589986974373 +MarCo,1800,100,0.0005910289983148687 +(MC)2MKP,1800,100,293.56771340200066 +MarIn,1800,100,0.026715467996837106 +MarDecUn,1800,100,0.0005221969986450858 +MarDec,1800,100,17.21068619399739 +FedAvg,1800,100,1.340000017080456e-05 +MarIn,1800,100,0.02641014499386074 +MarDecUn,1800,100,0.000524629998835735 +(MC)2MKP,1800,100,290.27558879299613 +MarCo,1800,100,0.0006262620008783415 +MarDec,1800,100,17.11015260199929 +FedAvg,1800,100,1.3791999663226306e-05 +(MC)2MKP,1800,100,294.3553522959992 +MarDecUn,1800,100,0.0005630050000036135 +MarIn,1800,100,0.026234873999783304 +MarCo,1800,100,0.0005926020021433942 +FedAvg,1800,100,1.5796998923178762e-05 +MarDec,1800,100,17.08415904499998 +MarIn,1800,100,0.026081153999257367 +MarCo,1800,100,0.0005826060005347244 +MarDecUn,1800,100,0.00052092999976594 +FedAvg,1800,100,1.2315002095419914e-05 +MarDec,1800,100,17.333037176998914 +(MC)2MKP,1800,100,293.3437687869955 +MarDecUn,1800,100,0.0005668339945259504 +FedAvg,1800,100,1.5757002984173596e-05 +(MC)2MKP,1800,100,292.6215674069972 +MarCo,1800,100,0.0006450049986597151 +MarDec,1800,100,17.17152801300108 +MarIn,1800,100,0.026477694998902734 +MarIn,1800,100,0.02627123399724951 +MarCo,1800,100,0.0005870870008948259 +(MC)2MKP,1800,100,292.85265235300176 +MarDec,1800,100,17.269065391003096 +MarDecUn,1800,100,0.0005224069973337464 +FedAvg,1800,100,1.4457000361289829e-05 +FedAvg,1800,100,1.229200279340148e-05 +(MC)2MKP,1800,100,293.85932774100365 +MarCo,1800,100,0.0006412619986804202 +MarDec,1800,100,17.2565957370025 +MarDecUn,1800,100,0.0005242059996817261 +MarIn,1800,100,0.026417859000503086 +(MC)2MKP,2000,100,359.7349002419942 +MarCo,2000,100,0.0006403279985534027 +MarDec,2000,100,19.15350374099944 +FedAvg,2000,100,1.383500057272613e-05 +MarDecUn,2000,100,0.00052466600027401 +MarIn,2000,100,0.029450427005940583 +MarCo,2000,100,0.0005854659975739196 +(MC)2MKP,2000,100,359.023833059 +MarDecUn,2000,100,0.0005667320001521148 +MarDec,2000,100,19.330750654997246 +MarIn,2000,100,0.029382384003838524 +FedAvg,2000,100,1.4628996723331511e-05 +MarDec,2000,100,19.258101800995064 +MarDecUn,2000,100,0.0005267629967420362 +MarCo,2000,100,0.0005852859976585023 +(MC)2MKP,2000,100,362.12848339900665 +MarIn,2000,100,0.029284091004228685 +FedAvg,2000,100,1.6238998796325177e-05 +MarCo,2000,100,0.0005850169982295483 +(MC)2MKP,2000,100,364.45944935499574 +MarDecUn,2000,100,0.0005676300061168149 +MarIn,2000,100,0.0292644260043744 +MarDec,2000,100,19.268239860000904 +FedAvg,2000,100,1.398799940943718e-05 +MarCo,2000,100,0.0005924490033066832 +FedAvg,2000,100,1.2412994692567736e-05 +MarIn,2000,100,0.02963134300080128 +MarDecUn,2000,100,0.0005138430060469545 +(MC)2MKP,2000,100,362.56268288700085 +MarDec,2000,100,19.251661904003413 +MarDecUn,2000,100,0.0005284190046950243 +FedAvg,2000,100,1.4386001566890627e-05 +MarDec,2000,100,19.285550354004954 +MarCo,2000,100,0.000589064002269879 +MarIn,2000,100,0.0293009590022848 +(MC)2MKP,2000,100,362.6106878049977 +MarCo,2000,100,0.0006394410011125728 +(MC)2MKP,2000,100,362.747807558997 +MarIn,2000,100,0.029621993999171536 +MarDecUn,2000,100,0.0005228829977568239 +MarDec,2000,100,19.416452769997704 +FedAvg,2000,100,1.3486998795997351e-05 +MarIn,2000,100,0.02953912000521086 +(MC)2MKP,2000,100,360.9796632379948 +MarDec,2000,100,19.34653481600253 +FedAvg,2000,100,1.3259996194392443e-05 +MarCo,2000,100,0.0005948559992248192 +MarDecUn,2000,100,0.0005332330038072541 +MarDecUn,2000,100,0.0005188309951336123 +(MC)2MKP,2000,100,366.0886834080011 +MarDec,2000,100,19.313600214998587 +MarIn,2000,100,0.02914581700315466 +FedAvg,2000,100,1.45840022014454e-05 +MarCo,2000,100,0.0005837310018250719 +MarCo,2000,100,0.0005781110012321733 +MarIn,2000,100,0.029355398000916466 +FedAvg,2000,100,1.2350996257737279e-05 +MarDec,2000,100,19.300097708997782 +MarDecUn,2000,100,0.0005225330023677088 +(MC)2MKP,2000,100,365.22923215899937 +FedAvg,2000,100,1.4429999282583594e-05 +MarCo,2000,100,0.0006328699964797124 +(MC)2MKP,2000,100,364.07453197499854 +MarIn,2000,100,0.029421341998386197 +MarDecUn,2000,100,0.0005254920033621602 +MarDec,2000,100,19.302869336002914 +MarDec,2000,100,19.19454691899591 +FedAvg,2000,100,1.3913006114307791e-05 +MarCo,2000,100,0.0005929730032221414 +(MC)2MKP,2000,100,362.5125460900017 +MarDecUn,2000,100,0.0005742169960285537 +MarIn,2000,100,0.029255929999635555 +MarDec,2000,100,19.32626031700056 +MarDecUn,2000,100,0.0005250310059636831 +(MC)2MKP,2000,100,360.97739189999993 +FedAvg,2000,100,1.5673002053517848e-05 +MarIn,2000,100,0.029443861996696796 +MarCo,2000,100,0.0005827860004501417 +FedAvg,2000,100,1.232800423167646e-05 +MarCo,2000,100,0.0005827909990330227 +MarDecUn,2000,100,0.0005250739995972253 +(MC)2MKP,2000,100,357.6254524390024 +MarDec,2000,100,19.442560744995717 +MarIn,2000,100,0.029375833000813145 +(MC)2MKP,2000,100,360.4683802899963 +MarDecUn,2000,100,0.0005648229998769239 +FedAvg,2000,100,1.7263002519030124e-05 +MarDec,2000,100,19.11899794600322 +MarCo,2000,100,0.0005908480015932582 +MarIn,2000,100,0.030982176001998596 +(MC)2MKP,2000,100,360.89151867600594 +MarCo,2000,100,0.0006356410012813285 +MarDec,2000,100,19.403555156997754 +FedAvg,2000,100,1.3387005310505629e-05 +MarDecUn,2000,100,0.0005193270044401288 +MarIn,2000,100,0.02943211699312087 +MarDecUn,2000,100,0.0005105449963593856 +MarIn,2000,100,0.02933311800006777 +MarCo,2000,100,0.0005844970000907779 +FedAvg,2000,100,1.2251002772245556e-05 +(MC)2MKP,2000,100,365.65790418999677 +MarDec,2000,100,19.28352831799566 +FedAvg,2000,100,1.3639000826515257e-05 +MarCo,2000,100,0.0005968709956505336 +(MC)2MKP,2000,100,363.36513960199954 +MarDecUn,2000,100,0.0005639869996230118 +MarDec,2000,100,19.165586858005554 +MarIn,2000,100,0.029511619999539107 +MarDecUn,2000,100,0.0005151690056663938 +(MC)2MKP,2000,100,364.7171023519986 +MarDec,2000,100,19.224937112994667 +FedAvg,2000,100,1.3880999176762998e-05 +MarCo,2000,100,0.0005836220007040538 +MarIn,2000,100,0.029482395999366418 +FedAvg,2000,100,1.2492004316300154e-05 +MarIn,2000,100,0.029178502001741435 +MarCo,2000,100,0.0005767719994764775 +MarDecUn,2000,100,0.0005215559940552339 +(MC)2MKP,2000,100,363.43613376600115 +MarDec,2000,100,19.336832724999113 diff --git a/chameleon_results/results_of_timing_with_fixed_tasks.csv b/chameleon_results/results_of_timing_with_fixed_tasks.csv new file mode 100644 index 0000000..bd4da22 --- /dev/null +++ b/chameleon_results/results_of_timing_with_fixed_tasks.csv @@ -0,0 +1,497 @@ +# Description of the experiment: +# +# - We generate the costs to up to 2.000 tasks for 20 to 80 resources +# with steps of 10. +# - All costs follow linear functions (i.e., constant marginal costs) +# with RNG seeds [0..79]. +# - We schedule 2.000 tasks. +# - We run (MC)^2MKP, MarIn, MarCo, MarDec, MarDecUn, and FedAvg. +# - All resources have a lower limit of 1. +# - The first half of the resources have no upper limit. +# - The second half has an upper limit of 2*(tasks/resources). +# - Each sample is composed of 5 executions of the schedulers. +# - We get 20 samples for each pair (scheduler, tasks) +# - The order of execution of the different schedulers is +# randomly defined. We set an initial RNG seed = 1000 and increase +# it every time we need a new order. +Scheduler,Tasks,Resources,Time +MarIn,2000,20,0.020915584000022136 +MarDec,2000,20,0.7687608819999809 +MarCo,2000,20,0.00016691700000137644 +FedAvg,2000,20,1.2620000006791088e-05 +(MC)2MKP,2000,20,81.71238614600003 +MarDecUn,2000,20,0.00020234400000163077 +MarDecUn,2000,20,0.0001382770000191158 +(MC)2MKP,2000,20,82.18119479 +MarCo,2000,20,0.00023674299995946058 +MarDec,2000,20,0.783302655 +MarIn,2000,20,0.021372002000020984 +FedAvg,2000,20,1.2226000023929373e-05 +FedAvg,2000,20,1.1847999985548086e-05 +MarDec,2000,20,0.7798050829999852 +(MC)2MKP,2000,20,82.159466528 +MarCo,2000,20,0.00022166700000525452 +MarDecUn,2000,20,0.0001412670000036087 +MarIn,2000,20,0.020860395999989123 +MarIn,2000,20,0.022575764999942294 +MarDecUn,2000,20,0.00014440200004628423 +MarCo,2000,20,0.00017525400005524716 +MarDec,2000,20,0.7762024710000333 +(MC)2MKP,2000,20,81.53745230999994 +FedAvg,2000,20,1.5590999964842922e-05 +(MC)2MKP,2000,20,80.78178170400008 +FedAvg,2000,20,1.2201000004097295e-05 +MarIn,2000,20,0.020900165000057314 +MarDec,2000,20,0.7864127730000519 +MarDecUn,2000,20,0.00014063800006169913 +MarCo,2000,20,0.00017014400009429664 +(MC)2MKP,2000,20,80.84346081400008 +MarCo,2000,20,0.00021956000000500353 +MarIn,2000,20,0.020849228999963998 +MarDec,2000,20,0.7857268730000442 +FedAvg,2000,20,1.1796999956459331e-05 +MarDecUn,2000,20,0.0001438660000303571 +(MC)2MKP,2000,20,82.10602103899998 +FedAvg,2000,20,1.6427999980805907e-05 +MarCo,2000,20,0.00024060800001279858 +MarDecUn,2000,20,0.00015277400007107644 +MarIn,2000,20,0.023279527999989114 +MarDec,2000,20,0.8483078379999824 +(MC)2MKP,2000,20,81.02573985599997 +MarDec,2000,20,0.7836977959999558 +MarIn,2000,20,0.02100694700004624 +MarDecUn,2000,20,0.0001415070000803098 +FedAvg,2000,20,1.1954000001423992e-05 +MarCo,2000,20,0.0001682469999195746 +MarDecUn,2000,20,0.00013919900004566443 +MarCo,2000,20,0.00016483500007780094 +FedAvg,2000,20,1.2145000027885544e-05 +MarDec,2000,20,0.7781445870000425 +MarIn,2000,20,0.021307460999992145 +(MC)2MKP,2000,20,80.81521395900006 +FedAvg,2000,20,1.236899993273255e-05 +MarDecUn,2000,20,0.00018643900000370195 +MarIn,2000,20,0.021451644000080705 +MarDec,2000,20,0.7796028019999994 +MarCo,2000,20,0.00016841800004385732 +(MC)2MKP,2000,20,80.89757994699994 +FedAvg,2000,20,1.2313000070207636e-05 +MarIn,2000,20,0.02088708799988126 +MarDecUn,2000,20,0.00014209100004336506 +MarCo,2000,20,0.00017008899999382265 +(MC)2MKP,2000,20,80.79885967400014 +MarDec,2000,20,0.7845601170001828 +(MC)2MKP,2000,20,80.77587126000003 +MarCo,2000,20,0.00022495300004266028 +FedAvg,2000,20,1.2350999895716086e-05 +MarIn,2000,20,0.020905118000200673 +MarDec,2000,20,0.7802260139999362 +MarDecUn,2000,20,0.00013938300003246695 +(MC)2MKP,2000,20,80.87868489599987 +MarDec,2000,20,0.7883089720000953 +FedAvg,2000,20,1.2087999948562356e-05 +MarDecUn,2000,20,0.00014541699988512846 +MarIn,2000,20,0.02104497100003755 +MarCo,2000,20,0.0001684050000676507 +MarIn,2000,20,0.02110857900015617 +FedAvg,2000,20,1.2008000112473383e-05 +MarDecUn,2000,20,0.00013681500013262848 +MarCo,2000,20,0.00016573499988226104 +MarDec,2000,20,0.7765294490000088 +(MC)2MKP,2000,20,82.48953131000007 +MarDec,2000,20,0.7862363399999595 +(MC)2MKP,2000,20,80.95829703300001 +MarDecUn,2000,20,0.00018910800008598017 +MarIn,2000,20,0.021130183999957808 +MarCo,2000,20,0.00016942700017352763 +FedAvg,2000,20,1.3924999848313746e-05 +(MC)2MKP,2000,20,80.63448893700001 +FedAvg,2000,20,1.2126000001444481e-05 +MarIn,2000,20,0.021244077999881483 +MarDecUn,2000,20,0.00013807999994241982 +MarCo,2000,20,0.00016684299998814822 +MarDec,2000,20,0.7801674309998816 +MarCo,2000,20,0.00016660799997225695 +(MC)2MKP,2000,20,81.59217465999996 +FedAvg,2000,20,1.2114999890400213e-05 +MarDecUn,2000,20,0.00019027400003324146 +MarIn,2000,20,0.02110787100014022 +MarDec,2000,20,0.7823750929999278 +FedAvg,2000,20,1.2225000091348193e-05 +(MC)2MKP,2000,20,80.70189052199999 +MarDecUn,2000,20,0.0001873250000699045 +MarCo,2000,20,0.00017759899992597639 +MarIn,2000,20,0.021230456000012055 +MarDec,2000,20,0.7797474899998633 +(MC)2MKP,2000,20,81.02195992199995 +MarCo,2000,20,0.00021834000017406652 +MarDec,2000,20,0.7811986609999622 +MarDecUn,2000,20,0.00014427499991143122 +FedAvg,2000,20,1.2130000186516554e-05 +MarIn,2000,20,0.021018756000103167 +MarDec,2000,20,0.7791493790000459 +MarIn,2000,20,0.021166364999999132 +MarCo,2000,20,0.00016782300008344464 +MarDecUn,2000,20,0.00013911699988966575 +(MC)2MKP,2000,20,80.81778215400004 +FedAvg,2000,20,1.218200009134307e-05 +MarCo,2000,40,0.00030790999994678714 +MarDecUn,2000,40,0.000238439999975526 +FedAvg,2000,40,1.2088000175936031e-05 +(MC)2MKP,2000,40,158.9767808869999 +MarDec,2000,40,3.0924086869999883 +MarIn,2000,40,0.02144292500042866 +(MC)2MKP,2000,40,158.85099123300006 +MarCo,2000,40,0.0003053780001209816 +MarDecUn,2000,40,0.0002411539999229717 +MarDec,2000,40,3.08297633400025 +FedAvg,2000,40,1.2315999811107758e-05 +MarIn,2000,40,0.021710658999836596 +MarIn,2000,40,0.021718315000271105 +MarCo,2000,40,0.0002471299999342591 +(MC)2MKP,2000,40,157.8753596219999 +FedAvg,2000,40,1.2333000086073298e-05 +MarDecUn,2000,40,0.00028061500006515416 +MarDec,2000,40,3.0578790950003167 +MarCo,2000,40,0.0002560910002102901 +MarDec,2000,40,3.069277435999993 +MarIn,2000,40,0.021832325000104902 +(MC)2MKP,2000,40,159.36863562999997 +FedAvg,2000,40,1.2213999980303925e-05 +MarDecUn,2000,40,0.0002808740000546095 +MarCo,2000,40,0.00026700600028561894 +FedAvg,2000,40,1.380399999106885e-05 +(MC)2MKP,2000,40,158.2912126770002 +MarDec,2000,40,3.075951668000016 +MarIn,2000,40,0.0216111939998882 +MarDecUn,2000,40,0.0002433149998068984 +MarDec,2000,40,3.0699485949999143 +FedAvg,2000,40,1.2168000012025004e-05 +MarDecUn,2000,40,0.00023332799992203945 +MarIn,2000,40,0.02134064399979252 +(MC)2MKP,2000,40,157.47521065499996 +MarCo,2000,40,0.000301735999983066 +FedAvg,2000,40,1.2162000075477408e-05 +MarDec,2000,40,3.0733274519998304 +(MC)2MKP,2000,40,157.52987639399998 +MarCo,2000,40,0.0003019390001099964 +MarIn,2000,40,0.021847964000244247 +MarDecUn,2000,40,0.00022916900024938514 +(MC)2MKP,2000,40,158.48125497299998 +FedAvg,2000,40,1.5516000075876946e-05 +MarIn,2000,40,0.021778438999717764 +MarDec,2000,40,3.074716608000017 +MarDecUn,2000,40,0.00023778200011292938 +MarCo,2000,40,0.0002487280003151682 +MarCo,2000,40,0.00024730400036787614 +MarDec,2000,40,3.0760228989997813 +FedAvg,2000,40,1.2029000117763644e-05 +(MC)2MKP,2000,40,158.6332088639997 +MarDecUn,2000,40,0.0002824819998750172 +MarIn,2000,40,0.021692200999950728 +MarIn,2000,40,0.02179760399985753 +MarDec,2000,40,3.0626481990002503 +FedAvg,2000,40,1.2182999853393994e-05 +MarDecUn,2000,40,0.00023484499979531392 +(MC)2MKP,2000,40,157.53702620100012 +MarCo,2000,40,0.0003031000001101347 +(MC)2MKP,2000,40,159.3932814210002 +MarCo,2000,40,0.00030613399985668366 +MarDec,2000,40,3.07681299900014 +MarDecUn,2000,40,0.00023683499966864474 +FedAvg,2000,40,1.1962999906245386e-05 +MarIn,2000,40,0.021533952000027057 +MarCo,2000,40,0.0002475269998285512 +MarIn,2000,40,0.02225500800022928 +MarDec,2000,40,3.0815129190000334 +MarDecUn,2000,40,0.00023653499965803348 +(MC)2MKP,2000,40,158.95927546199982 +FedAvg,2000,40,1.657699976931326e-05 +MarDec,2000,40,3.0784577360000185 +FedAvg,2000,40,1.2372000128380023e-05 +MarIn,2000,40,0.02174629899991487 +MarDecUn,2000,40,0.0002293929996994848 +MarCo,2000,40,0.0002450219999445835 +(MC)2MKP,2000,40,158.49955103000002 +(MC)2MKP,2000,40,157.5452439339997 +MarDec,2000,40,3.0660797500004264 +MarIn,2000,40,0.021481763999872783 +MarDecUn,2000,40,0.00023268600034498377 +FedAvg,2000,40,1.409800006513251e-05 +MarCo,2000,40,0.00024765099988144357 +MarDecUn,2000,40,0.00023626200072612846 +(MC)2MKP,2000,40,158.28599173900056 +MarCo,2000,40,0.00030531199990946334 +MarIn,2000,40,0.021629536000546068 +MarDec,2000,40,3.0762186040001325 +FedAvg,2000,40,1.4489999557554256e-05 +MarCo,2000,40,0.0002471409998179297 +MarIn,2000,40,0.021830149999914283 +MarDec,2000,40,3.0662698020005337 +FedAvg,2000,40,1.2303000403335318e-05 +MarDecUn,2000,40,0.0002378959998168284 +(MC)2MKP,2000,40,158.3707956850003 +MarCo,2000,40,0.000303016000543721 +MarIn,2000,40,0.021528307999687968 +FedAvg,2000,40,1.4123000255494844e-05 +MarDec,2000,40,3.090511860999868 +(MC)2MKP,2000,40,159.89351433999946 +MarDecUn,2000,40,0.0002858039997590822 +(MC)2MKP,2000,40,158.89946812800008 +MarIn,2000,40,0.021396400999947218 +MarDecUn,2000,40,0.00023909599985927343 +MarCo,2000,40,0.0002521859996704734 +MarDec,2000,40,3.0919599790004213 +FedAvg,2000,40,1.412900019204244e-05 +MarIn,2000,40,0.02195556900005613 +MarDecUn,2000,40,0.0002337380001335987 +MarCo,2000,40,0.0002457729997331626 +FedAvg,2000,40,1.189800059364643e-05 +MarDec,2000,40,3.069656157999816 +(MC)2MKP,2000,40,158.02353529500033 +MarIn,2000,40,0.021588854000583524 +MarDec,2000,40,3.073991651999677 +FedAvg,2000,40,1.4317999557533767e-05 +MarCo,2000,40,0.0002541429994380451 +(MC)2MKP,2000,40,158.93376424100006 +MarDecUn,2000,40,0.0002784930002235342 +MarCo,2000,60,0.00035832199955621036 +FedAvg,2000,60,3.224100055376766e-05 +MarIn,2000,60,0.021779761000289 +MarDecUn,2000,60,0.0003248310003982624 +MarDec,2000,60,6.898054473999764 +(MC)2MKP,2000,60,221.60453599100038 +MarCo,2000,60,0.0004004679994977778 +MarDec,2000,60,6.895609826000509 +MarDecUn,2000,60,0.00033366600018780446 +(MC)2MKP,2000,60,223.11811662400032 +FedAvg,2000,60,3.6070000533072744e-05 +MarIn,2000,60,0.021476298999914434 +(MC)2MKP,2000,60,222.61004143599985 +MarIn,2000,60,0.021834633000253234 +MarDecUn,2000,60,0.0003252910000810516 +FedAvg,2000,60,3.0027999855519738e-05 +MarCo,2000,60,0.00035004500023205765 +MarDec,2000,60,6.898543949000668 +(MC)2MKP,2000,60,221.67080890200032 +MarDec,2000,60,6.8685188769995875 +MarCo,2000,60,0.0003629709999586339 +FedAvg,2000,60,2.8565000320668332e-05 +MarDecUn,2000,60,0.00033610900027269963 +MarIn,2000,60,0.021702377000110573 +MarDec,2000,60,6.897144507000121 +(MC)2MKP,2000,60,223.49875998700008 +FedAvg,2000,60,2.9040999834251124e-05 +MarDecUn,2000,60,0.0003702889998749015 +MarCo,2000,60,0.0003556439996827976 +MarIn,2000,60,0.021786823000184086 +MarCo,2000,60,0.00034831900029530516 +MarDecUn,2000,60,0.0003334660004838952 +MarDec,2000,60,6.932478556000206 +FedAvg,2000,60,2.731099993980024e-05 +(MC)2MKP,2000,60,222.6301225999996 +MarIn,2000,60,0.021797606999825803 +MarDecUn,2000,60,0.0003314240002509905 +MarCo,2000,60,0.00034586299989314284 +MarDec,2000,60,6.884106958000302 +FedAvg,2000,60,2.7766999664891046e-05 +MarIn,2000,60,0.02187324299939064 +(MC)2MKP,2000,60,225.00464616800036 +MarDecUn,2000,60,0.0003859529997498612 +MarIn,2000,60,0.021429190000162635 +MarDec,2000,60,6.895089534999897 +FedAvg,2000,60,2.8401999770721886e-05 +(MC)2MKP,2000,60,221.7685524160006 +MarCo,2000,60,0.0003977569995186059 +MarDec,2000,60,6.932159816999956 +(MC)2MKP,2000,60,223.05500796499928 +MarDecUn,2000,60,0.0003838379998342134 +FedAvg,2000,60,3.0218000574677717e-05 +MarCo,2000,60,0.0003567039993868093 +MarIn,2000,60,0.021402972999567282 +MarIn,2000,60,0.021783524999591464 +(MC)2MKP,2000,60,222.31215177300055 +FedAvg,2000,60,3.3782999707909767e-05 +MarDecUn,2000,60,0.00037028600036137505 +MarDec,2000,60,6.881965251999645 +MarCo,2000,60,0.00035088200002064696 +FedAvg,2000,60,2.765099998214282e-05 +(MC)2MKP,2000,60,223.74198158699983 +MarIn,2000,60,0.021630541999911657 +MarDecUn,2000,60,0.00033155900018755347 +MarCo,2000,60,0.0003497930001685745 +MarDec,2000,60,6.857071997000276 +MarIn,2000,60,0.02182741399974475 +(MC)2MKP,2000,60,224.20199079299982 +MarDec,2000,60,6.90263726000012 +MarCo,2000,60,0.00035831200057145907 +FedAvg,2000,60,2.7725000109057873e-05 +MarDecUn,2000,60,0.00032925699997576885 +MarDecUn,2000,60,0.0003356539991727914 +MarDec,2000,60,6.889068552000026 +MarIn,2000,60,0.02233341000010114 +FedAvg,2000,60,2.45809997068136e-05 +MarCo,2000,60,0.00034873799995693844 +(MC)2MKP,2000,60,223.20876122900063 +(MC)2MKP,2000,60,222.62276750100045 +MarIn,2000,60,0.02178955700037477 +MarDecUn,2000,60,0.00032322100014425814 +FedAvg,2000,60,2.971199864987284e-05 +MarCo,2000,60,0.00035115900027449243 +MarDec,2000,60,6.867394263999813 +MarIn,2000,60,0.02197352900111582 +(MC)2MKP,2000,60,223.58944242600046 +FedAvg,2000,60,3.8889000279596075e-05 +MarDec,2000,60,6.8688003749994095 +MarDecUn,2000,60,0.00032164099866349716 +MarCo,2000,60,0.0003523280010995222 +MarIn,2000,60,0.02180293000128586 +MarDecUn,2000,60,0.0003319300012663007 +(MC)2MKP,2000,60,224.8921169109999 +MarDec,2000,60,6.887717634999717 +FedAvg,2000,60,2.801099981297739e-05 +MarCo,2000,60,0.0003547160013113171 +FedAvg,2000,60,2.363599924137816e-05 +MarDec,2000,60,6.863059152999995 +MarIn,2000,60,0.022021748000042862 +MarCo,2000,60,0.00034917300035886 +MarDecUn,2000,60,0.0003245799998694565 +(MC)2MKP,2000,60,222.5965631830004 +MarDec,2000,60,6.8769500279995555 +MarDecUn,2000,60,0.0003277750001871027 +MarCo,2000,60,0.00035408299845585134 +FedAvg,2000,60,3.0164001145749353e-05 +MarIn,2000,60,0.02202948600097443 +(MC)2MKP,2000,60,223.19327255400094 +FedAvg,2000,60,3.42030016327044e-05 +MarDecUn,2000,60,0.00037934500142000616 +MarCo,2000,60,0.00035765499887929764 +MarDec,2000,60,6.865024837001329 +(MC)2MKP,2000,60,223.1830130340004 +MarIn,2000,60,0.02154753299873846 +FedAvg,2000,60,2.758699883997906e-05 +MarDecUn,2000,60,0.00033279700073762797 +MarCo,2000,60,0.00034351499925833195 +MarDec,2000,60,6.873585420000381 +MarIn,2000,60,0.021726042999944184 +(MC)2MKP,2000,60,222.26837071699993 +MarIn,2000,80,0.022424937998948735 +FedAvg,2000,80,1.4171000657370314e-05 +MarCo,2000,80,0.00044238800001039635 +MarDecUn,2000,80,0.0004156280010647606 +(MC)2MKP,2000,80,286.35892306500136 +MarDec,2000,80,12.095398436000323 +MarDec,2000,80,12.154633972000738 +MarCo,2000,80,0.00045271500130183995 +FedAvg,2000,80,1.4583998563466594e-05 +(MC)2MKP,2000,80,286.6713297749993 +MarIn,2000,80,0.0222844880008779 +MarDecUn,2000,80,0.0004287629999453202 +(MC)2MKP,2000,80,287.0308216760004 +MarDec,2000,80,12.18272471399905 +MarIn,2000,80,0.022262082999077393 +MarCo,2000,80,0.000450904999524937 +FedAvg,2000,80,1.444000008632429e-05 +MarDecUn,2000,80,0.00041493200114928186 +MarDecUn,2000,80,0.0004178439994575456 +MarIn,2000,80,0.02191015099924698 +MarCo,2000,80,0.00043641699994623195 +MarDec,2000,80,12.12974181500067 +(MC)2MKP,2000,80,289.3306533619998 +FedAvg,2000,80,1.5768000594107434e-05 +FedAvg,2000,80,1.2382999557303265e-05 +MarDecUn,2000,80,0.0004703819995484082 +MarIn,2000,80,0.022420033001253614 +(MC)2MKP,2000,80,288.42174585499924 +MarCo,2000,80,0.0004939039990858873 +MarDec,2000,80,12.17819652399885 +MarDec,2000,80,12.116051349999907 +FedAvg,2000,80,1.3704999219044112e-05 +MarIn,2000,80,0.022041913000066415 +MarDecUn,2000,80,0.0004249100002198247 +(MC)2MKP,2000,80,286.5382300209985 +MarCo,2000,80,0.000499311998282792 +(MC)2MKP,2000,80,287.92479421499957 +MarCo,2000,80,0.0005097360008221585 +FedAvg,2000,80,1.5720999726909213e-05 +MarDec,2000,80,12.157853549999345 +MarDecUn,2000,80,0.0004139680004300317 +MarIn,2000,80,0.022978789000262623 +MarDecUn,2000,80,0.00042651300100260414 +FedAvg,2000,80,1.2333999620750546e-05 +MarCo,2000,80,0.00044620100015890785 +MarDec,2000,80,12.12773102599931 +(MC)2MKP,2000,80,285.8929153270001 +MarIn,2000,80,0.02262809700005164 +(MC)2MKP,2000,80,285.8613463769998 +FedAvg,2000,80,1.5689000065322034e-05 +MarDecUn,2000,80,0.00047137200090219267 +MarIn,2000,80,0.022781887999371975 +MarDec,2000,80,12.152703697998732 +MarCo,2000,80,0.00044389300092007034 +MarCo,2000,80,0.0004463259992917301 +(MC)2MKP,2000,80,287.759777932999 +MarDecUn,2000,80,0.00046069000018178485 +FedAvg,2000,80,1.6072999642346986e-05 +MarIn,2000,80,0.02224903699971037 +MarDec,2000,80,12.158478199999081 +(MC)2MKP,2000,80,287.42971926300015 +MarDecUn,2000,80,0.00047682399963377975 +MarCo,2000,80,0.00044353299927024636 +MarDec,2000,80,12.14163630999974 +MarIn,2000,80,0.02233894200071518 +FedAvg,2000,80,1.4533001376548782e-05 +MarCo,2000,80,0.0004362149993539788 +FedAvg,2000,80,1.1950000043725595e-05 +MarIn,2000,80,0.022570520000954275 +MarDec,2000,80,12.182939506999901 +(MC)2MKP,2000,80,285.41969082000105 +MarDecUn,2000,80,0.00047524600086035207 +FedAvg,2000,80,1.444300141884014e-05 +MarIn,2000,80,0.0225564709999162 +MarCo,2000,80,0.00044328799958748277 +MarDecUn,2000,80,0.00041424699884373695 +(MC)2MKP,2000,80,287.26329343900034 +MarDec,2000,80,12.185785861000113 +MarDecUn,2000,80,0.0004299650008761091 +MarDec,2000,80,12.168005430001358 +(MC)2MKP,2000,80,286.48613164699964 +MarIn,2000,80,0.02279668099981791 +FedAvg,2000,80,1.5814999642316252e-05 +MarCo,2000,80,0.0004463120003492804 +MarDec,2000,80,12.122918295999625 +(MC)2MKP,2000,80,285.2546195349987 +FedAvg,2000,80,1.5871000869083218e-05 +MarCo,2000,80,0.0004961269987688866 +MarIn,2000,80,0.021932835999905365 +MarDecUn,2000,80,0.0004265509996912442 +MarCo,2000,80,0.00044215500020072795 +FedAvg,2000,80,1.2228001651237719e-05 +MarDec,2000,80,12.131971013999646 +MarIn,2000,80,0.02244390200030466 +MarDecUn,2000,80,0.0004269519995432347 +(MC)2MKP,2000,80,287.5966905999994 +(MC)2MKP,2000,80,287.4782626860015 +MarDec,2000,80,12.171872524999344 +FedAvg,2000,80,1.3801000022795051e-05 +MarIn,2000,80,0.022386152999388287 +MarDecUn,2000,80,0.00041714400140335783 +MarCo,2000,80,0.00043681599890987854 +MarCo,2000,80,0.000438912000390701 +MarDecUn,2000,80,0.00042121300066355616 +MarDec,2000,80,12.117934278001485 +FedAvg,2000,80,1.2301999959163368e-05 +MarIn,2000,80,0.022076678000303218 +(MC)2MKP,2000,80,287.3904541539996 +MarDec,2000,80,12.112907217999236 +(MC)2MKP,2000,80,285.5190110040003 +MarDecUn,2000,80,0.00047109400111366995 +FedAvg,2000,80,1.575900023453869e-05 +MarIn,2000,80,0.02258373400036362 +MarCo,2000,80,0.0004416329993546242 +MarCo,2000,80,0.00043951900079264306 +(MC)2MKP,2000,80,287.05078377000063 +FedAvg,2000,80,1.5939000149955973e-05 +MarDec,2000,80,12.086873851998462 +MarIn,2000,80,0.02212020700062567 +MarDecUn,2000,80,0.00043026999992434867 diff --git a/stored_results/results_with_constant_marginal_costs.csv b/chameleon_results/results_with_constant_marginal_costs.csv similarity index 100% rename from stored_results/results_with_constant_marginal_costs.csv rename to chameleon_results/results_with_constant_marginal_costs.csv diff --git a/stored_results/results_with_constant_marginal_costs_no_upper_limit.csv b/chameleon_results/results_with_constant_marginal_costs_no_upper_limit.csv similarity index 100% rename from stored_results/results_with_constant_marginal_costs_no_upper_limit.csv rename to chameleon_results/results_with_constant_marginal_costs_no_upper_limit.csv diff --git a/chameleon_results/results_with_decreasing_marginal_costs.csv b/chameleon_results/results_with_decreasing_marginal_costs.csv new file mode 100644 index 0000000..e736862 --- /dev/null +++ b/chameleon_results/results_with_decreasing_marginal_costs.csv @@ -0,0 +1,422 @@ +# Description of the experiment: +# +# - We generate the costs to up to 5.000 tasks for 10 and 100 resources. +# - All costs follow log n functions (i.e., decreasing marginal costs) +# with RNG seeds [300..399]. +# - We schedule from 1.000 to 5.000 tasks in increments of 100. +# - We run (MC)^2MKP, MarIn, MarCo, MarDec, and FedAvg. +# - All resources have a lower limit of 5. +# - The first half of the resources have no upper limit. +# - The second half has an upper limit of 2*(tasks/resources). +# - Every result is verified and logged to a CSV file. +Scheduler,Tasks,Resources,Total Cost +(MC)2MKP,1000,10,191.83661231269497 +MarIn,1000,10,191.83661231269497 +MarCo,1000,10,191.83661231269497 +MarDec,1000,10,191.83661231269497 +FedAvg,1000,10,369.4074983962643 +(MC)2MKP,1100,10,192.13399462937522 +MarIn,1100,10,192.13399462937522 +MarCo,1100,10,192.13399462937522 +MarDec,1100,10,192.13399462937522 +FedAvg,1100,10,375.8521206421594 +(MC)2MKP,1200,10,192.4044503236575 +MarIn,1200,10,192.4044503236575 +MarCo,1200,10,192.4044503236575 +MarDec,1200,10,192.4044503236575 +FedAvg,1200,10,381.74045017983724 +(MC)2MKP,1300,10,192.65245327025255 +MarIn,1300,10,192.65245327025255 +MarCo,1300,10,192.65245327025255 +MarDec,1300,10,192.65245327025255 +FedAvg,1300,10,387.16093948971314 +(MC)2MKP,1400,10,192.8814473601422 +MarIn,1400,10,192.8814473601422 +MarCo,1400,10,192.8814473601422 +MarDec,1400,10,192.8814473601422 +FedAvg,1400,10,392.1824895515547 +(MC)2MKP,1500,10,193.09414025682062 +MarIn,1500,10,193.09414025682062 +MarCo,1500,10,193.09414025682062 +MarDec,1500,10,193.09414025682062 +FedAvg,1500,10,396.8598187214149 +(MC)2MKP,1600,10,193.29269936106056 +MarIn,1600,10,193.29269936106056 +MarCo,1600,10,193.29269936106056 +MarDec,1600,10,193.29269936106056 +FedAvg,1600,10,401.23710652895505 +(MC)2MKP,1700,10,193.4788866230593 +MarIn,1700,10,193.4788866230593 +MarCo,1700,10,193.4788866230593 +MarDec,1700,10,193.4788866230593 +FedAvg,1700,10,405.35053756469597 +(MC)2MKP,1800,10,193.65415376272068 +MarIn,1800,10,193.65415376272068 +MarCo,1800,10,193.65415376272068 +MarDec,1800,10,193.65415376272068 +FedAvg,1800,10,409.2301212940795 +(MC)2MKP,1900,10,193.8197110802335 +MarIn,1900,10,193.8197110802335 +MarCo,1900,10,193.8197110802335 +MarDec,1900,10,193.8197110802335 +FedAvg,1900,10,412.90102184775986 +(MC)2MKP,2000,10,193.976578186693 +MarIn,2000,10,193.976578186693 +MarCo,2000,10,193.976578186693 +MarDec,2000,10,193.976578186693 +FedAvg,2000,10,416.3845480683133 +(MC)2MKP,2100,10,194.12562207187463 +MarIn,2100,10,194.12562207187463 +MarCo,2100,10,194.12562207187463 +MarDec,2100,10,194.12562207187463 +FedAvg,2100,10,419.6989029291027 +(MC)2MKP,2200,10,194.2675861224115 +MarIn,2200,10,194.2675861224115 +MarCo,2200,10,194.2675861224115 +MarDec,2200,10,194.2675861224115 +FedAvg,2200,10,422.85975926455035 +(MC)2MKP,2300,10,194.40311255518654 +MarIn,2300,10,194.40311255518654 +MarCo,2300,10,194.40311255518654 +MarDec,2300,10,194.40311255518654 +FedAvg,2300,10,425.88070798519897 +(MC)2MKP,2400,10,194.532759981423 +MarIn,2400,10,194.532759981423 +MarCo,2400,10,194.532759981423 +MarDec,2400,10,194.532759981423 +FedAvg,2400,10,428.7736112359513 +(MC)2MKP,2500,10,194.65701731717306 +MarIn,2500,10,194.65701731717306 +MarCo,2500,10,194.65701731717306 +MarDec,2500,10,194.65701731717306 +FedAvg,2500,10,431.548883707753 +(MC)2MKP,2600,10,194.7763149159005 +MarIn,2600,10,194.7763149159005 +MarCo,2600,10,194.7763149159005 +MarDec,2600,10,194.7763149159005 +FedAvg,2600,10,434.21571895886626 +(MC)2MKP,2700,10,194.89103356336236 +MarIn,2700,10,194.89103356336236 +MarCo,2700,10,194.89103356336236 +MarDec,2700,10,194.89103356336236 +FedAvg,2700,10,436.7822731611959 +(MC)2MKP,2800,10,195.0015118092081 +MarIn,2800,10,195.0015118092081 +MarCo,2800,10,195.0015118092081 +MarDec,2800,10,195.0015118092081 +FedAvg,2800,10,439.2558155350968 +(MC)2MKP,2900,10,195.1080519912494 +MarIn,2900,10,195.1080519912494 +MarCo,2900,10,195.1080519912494 +MarDec,2900,10,195.1080519912494 +FedAvg,2900,10,441.6428524666942 +(MC)2MKP,3000,10,195.2109252225329 +MarIn,3000,10,195.2109252225329 +MarCo,3000,10,195.2109252225329 +MarDec,3000,10,195.2109252225329 +FedAvg,3000,10,443.94923064631746 +(MC)2MKP,3100,10,195.31037554839023 +MarIn,3100,10,195.31037554839023 +MarCo,3100,10,195.31037554839023 +MarDec,3100,10,195.31037554839023 +FedAvg,3100,10,446.18022334441264 +(MC)2MKP,3200,10,195.40662343391315 +MarIn,3200,10,195.40662343391315 +MarCo,3200,10,195.40662343391315 +MarDec,3200,10,195.40662343391315 +FedAvg,3200,10,448.3406030287519 +(MC)2MKP,3300,10,195.4998687072422 +MarIn,3300,10,195.4998687072422 +MarCo,3300,10,195.4998687072422 +MarDec,3300,10,195.4998687072422 +FedAvg,3300,10,450.4347028383412 +(MC)2MKP,3400,10,195.59029305749107 +MarIn,3400,10,195.59029305749107 +MarCo,3400,10,195.59029305749107 +MarDec,3400,10,195.59029305749107 +FedAvg,3400,10,452.4664689050331 +(MC)2MKP,3500,10,195.678062165805 +MarIn,3500,10,195.678062165805 +MarCo,3500,10,195.678062165805 +MarDec,3500,10,195.678062165805 +FedAvg,3500,10,454.4395051108001 +(MC)2MKP,3600,10,195.7633275323699 +MarIn,3600,10,195.7633275323699 +MarCo,3600,10,195.7633275323699 +MarDec,3600,10,195.7633275323699 +FedAvg,3600,10,456.3571115561984 +(MC)2MKP,3700,10,195.84622804998753 +MarIn,3700,10,195.84622804998753 +MarCo,3700,10,195.84622804998753 +MarDec,3700,10,195.84622804998753 +FedAvg,3700,10,458.2223177714665 +(MC)2MKP,3800,10,195.92689136526627 +MarIn,3800,10,195.92689136526627 +MarCo,3800,10,195.92689136526627 +MarDec,3800,10,195.92689136526627 +FedAvg,3800,10,460.03791150957727 +(MC)2MKP,3900,10,196.00543506092438 +MarIn,3900,10,196.00543506092438 +MarCo,3900,10,196.00543506092438 +MarDec,3900,10,196.00543506092438 +FedAvg,3900,10,461.80646380827784 +(MC)2MKP,4000,10,196.0819676866952 +MarIn,4000,10,196.0819676866952 +MarCo,4000,10,196.0819676866952 +MarDec,4000,10,196.0819676866952 +FedAvg,4000,10,463.5303508866524 +(MC)2MKP,4100,10,196.15658966151983 +MarIn,4100,10,196.15658966151983 +MarCo,4100,10,196.15658966151983 +MarDec,4100,10,196.15658966151983 +FedAvg,4100,10,465.2117733441897 +(MC)2MKP,4200,10,196.22939406584183 +MarIn,4200,10,196.22939406584183 +MarCo,4200,10,196.22939406584183 +MarDec,4200,10,196.22939406584183 +FedAvg,4200,10,466.8527730515593 +(MC)2MKP,4300,10,196.30046733968783 +MarIn,4300,10,196.30046733968783 +MarCo,4300,10,196.30046733968783 +MarDec,4300,10,196.30046733968783 +FedAvg,4300,10,468.45524805832173 +(MC)2MKP,4400,10,196.36988989966528 +MarIn,4400,10,196.36988989966528 +MarCo,4400,10,196.36988989966528 +MarDec,4400,10,196.36988989966528 +FedAvg,4400,10,470.0209657905608 +(MC)2MKP,4500,10,196.43773668592368 +MarIn,4500,10,196.43773668592368 +MarCo,4500,10,196.43773668592368 +MarDec,4500,10,196.43773668592368 +FedAvg,4500,10,471.5515747685654 +(MC)2MKP,4600,10,196.504077648407 +MarIn,4600,10,196.504077648407 +MarCo,4600,10,196.504077648407 +MarDec,4600,10,196.504077648407 +FedAvg,4600,10,473.04861503935103 +(MC)2MKP,4700,10,196.56897818030848 +MarIn,4700,10,196.56897818030848 +MarCo,4700,10,196.56897818030848 +MarDec,4700,10,196.56897818030848 +FedAvg,4700,10,474.5135274895396 +(MC)2MKP,4800,10,196.63249950546097 +MarIn,4800,10,196.63249950546097 +MarCo,4800,10,196.63249950546097 +MarDec,4800,10,196.63249950546097 +FedAvg,4800,10,475.94766217976616 +(MC)2MKP,4900,10,196.6946990254152 +MarIn,4900,10,196.6946990254152 +MarCo,4900,10,196.6946990254152 +MarDec,4900,10,196.6946990254152 +FedAvg,4900,10,477.35228582144583 +(MC)2MKP,5000,10,196.75563063113748 +MarIn,5000,10,196.75563063113748 +MarCo,5000,10,196.75563063113748 +MarDec,5000,10,196.75563063113748 +FedAvg,5000,10,478.7285884996727 +(MC)2MKP,1000,100,1465.9860316625259 +MarIn,1000,100,1467.207093993081 +MarCo,1000,100,1467.207093993081 +MarDec,1000,100,1465.9860316625259 +FedAvg,1000,100,1777.7447082287256 +(MC)2MKP,1100,100,1466.181148756585 +MarIn,1100,100,1467.4950180898784 +MarCo,1100,100,1467.4950180898784 +MarDec,1100,100,1466.181148756585 +FedAvg,1100,100,1823.186734133962 +(MC)2MKP,1200,100,1466.3463961244945 +MarIn,1200,100,1467.7449730512383 +MarCo,1200,100,1467.7449730512383 +MarDec,1200,100,1466.3463961244945 +FedAvg,1200,100,1864.9893457172684 +(MC)2MKP,1300,100,1466.4897164881975 +MarIn,1300,100,1467.966260195746 +MarCo,1300,100,1467.966260195746 +MarDec,1300,100,1466.4897164881975 +FedAvg,1300,100,1903.692518875824 +(MC)2MKP,1400,100,1466.6162530857184 +MarIn,1400,100,1468.1650467567813 +MarCo,1400,100,1468.1650467567813 +MarDec,1400,100,1466.6162530857184 +FedAvg,1400,100,1939.72431106043 +(MC)2MKP,1500,100,1466.7295280247838 +MarIn,1500,100,1468.3456529042226 +MarCo,1500,100,1468.3456529042226 +MarDec,1500,100,1466.7295280247838 +FedAvg,1500,100,1973.4298016627529 +(MC)2MKP,1600,100,1466.8320592646844 +MarIn,1600,100,1468.511233786902 +MarCo,1600,100,1468.511233786902 +MarDec,1600,100,1466.8320592646844 +FedAvg,1600,100,2005.0912433085505 +(MC)2MKP,1700,100,1466.925709414487 +MarIn,1700,100,1468.6641703557355 +MarCo,1700,100,1468.6641703557355 +MarDec,1700,100,1466.925709414487 +FedAvg,1700,100,2034.9424445224643 +(MC)2MKP,1800,100,1467.0118951120182 +MarIn,1800,100,1468.8063073439666 +MarCo,1800,100,1468.8063073439666 +MarDec,1800,100,1467.0118951120182 +FedAvg,1800,100,2063.179258541404 +(MC)2MKP,1900,100,1467.0917189342865 +MarIn,1900,100,1468.9391053997535 +MarCo,1900,100,1468.9391053997535 +MarDec,1900,100,1467.0917189342865 +FedAvg,1900,100,2089.96737858922 +(MC)2MKP,2000,100,1467.1660558772633 +MarIn,2000,100,1469.063742288856 +MarCo,2000,100,1469.063742288856 +MarDec,2000,100,1467.1660558772633 +FedAvg,2000,100,2115.448229264326 +(MC)2MKP,2100,100,1467.2356119769258 +MarIn,2100,100,1469.1811824891217 +MarCo,2100,100,1469.1811824891217 +MarDec,2100,100,1467.2356119769258 +FedAvg,2100,100,2139.7434861460183 +(MC)2MKP,2200,100,1467.3009651921086 +MarIn,2200,100,1469.2922264122828 +MarCo,2200,100,1469.2922264122828 +MarDec,2200,100,1467.3009651921086 +FedAvg,2200,100,2162.9585899193794 +(MC)2MKP,2300,100,1467.362594624146 +MarIn,2300,100,1469.3975460655286 +MarCo,2300,100,1469.3975460655286 +MarDec,2300,100,1467.362594624146 +FedAvg,2300,100,2185.1855120512546 +(MC)2MKP,2400,100,1467.4209018514828 +MarIn,2400,100,1469.4977114337764 +MarCo,2400,100,1469.4977114337764 +MarDec,2400,100,1467.4209018514828 +FedAvg,2400,100,2206.504955515688 +(MC)2MKP,2500,100,1467.4762268023949 +MarIn,2500,100,1469.5932103571743 +MarCo,2500,100,1469.5932103571743 +MarDec,2500,100,1467.4762268023949 +FedAvg,2500,100,2226.9881236345623 +(MC)2MKP,2600,100,1467.528859762365 +MarIn,2600,100,1469.6844637512438 +MarCo,2600,100,1469.6844637512438 +MarDec,2600,100,1467.528859762365 +FedAvg,2600,100,2246.698154910966 +(MC)2MKP,2700,100,1467.5790505934012 +MarIn,2700,100,1469.7718374293293 +MarCo,2700,100,1469.7718374293293 +MarDec,2700,100,1467.5790505934012 +FedAvg,2700,100,2265.6912967931175 +(MC)2MKP,2800,100,1467.6270159077915 +MarIn,2800,100,1469.8556514045758 +MarCo,2800,100,1469.8556514045758 +MarDec,2800,100,1467.6270159077915 +FedAvg,2800,100,2284.0178733751973 +(MC)2MKP,2900,100,1467.6729447178698 +MarIn,2900,100,1469.9361872939564 +MarCo,2900,100,1469.9361872939564 +MarDec,2900,100,1467.6729447178698 +FedAvg,2900,100,2301.723088977723 +(MC)2MKP,3000,100,1467.7170029345186 +MarIn,3000,100,1470.0136942736642 +MarCo,3000,100,1470.0136942736642 +MarDec,3000,100,1467.7170029345186 +FedAvg,3000,100,2318.847699915992 +(MC)2MKP,3100,100,1467.759336984929 +MarIn,3100,100,1470.0883939151793 +MarCo,3100,100,1470.0883939151793 +MarDec,3100,100,1467.759336984929 +FedAvg,3100,100,2335.428579580046 +(MC)2MKP,3200,100,1467.800076748774 +MarIn,3200,100,1470.1604841467486 +MarCo,3200,100,1470.1604841467486 +MarDec,3200,100,1467.800076748774 +FedAvg,3200,100,2351.4991965345203 +(MC)2MKP,3300,100,1467.8393379613055 +MarIn,3300,100,1470.2301425244875 +MarCo,3300,100,1470.2301425244875 +MarDec,3300,100,1467.8393379613055 +FedAvg,3300,100,2367.090021225843 +(MC)2MKP,3400,100,1467.8772241954568 +MarIn,3400,100,1470.2975289533713 +MarCo,3400,100,1470.2975289533713 +MarDec,3400,100,1467.8772241954568 +FedAvg,3400,100,2382.2288737195845 +(MC)2MKP,3500,100,1467.9138285084675 +MarIn,3500,100,1470.362787966082 +MarCo,3500,100,1470.362787966082 +MarDec,3500,100,1467.9138285084675 +FedAvg,3500,100,2396.9412224397574 +(MC)2MKP,3600,100,1467.9492348189422 +MarIn,3600,100,1470.426050643626 +MarCo,3600,100,1470.426050643626 +MarDec,3600,100,1467.9492348189422 +FedAvg,3600,100,2411.25044196839 +(MC)2MKP,3700,100,1467.9835190656243 +MarIn,3700,100,1470.4874362435555 +MarCo,3700,100,1470.4874362435555 +MarDec,3700,100,1467.9835190656243 +FedAvg,3700,100,2425.178036458697 +(MC)2MKP,3800,100,1468.0167501881344 +MarIn,3800,100,1470.5470535878687 +MarCo,3800,100,1470.5470535878687 +MarDec,3800,100,1468.0167501881344 +FedAvg,3800,100,2438.743834023064 +(MC)2MKP,3900,100,1468.0489909615214 +MarIn,3900,100,1470.6050022521174 +MarCo,3900,100,1470.6050022521174 +MarDec,3900,100,1468.0489909615214 +FedAvg,3900,100,2451.966156506513 +(MC)2MKP,4000,100,1468.0802987100194 +MarIn,4000,100,1470.6613735890799 +MarCo,4000,100,1470.6613735890799 +MarDec,4000,100,1468.0802987100194 +FedAvg,4000,100,2464.8619682936705 +(MC)2MKP,4100,100,1468.110725920402 +MarIn,4100,100,1470.7162516139815 +MarCo,4100,100,1470.7162516139815 +MarDec,4100,100,1468.110725920402 +FedAvg,4100,100,2477.44700718162 +(MC)2MKP,4200,100,1468.1403207714204 +MarIn,4200,100,1470.7697137732232 +MarCo,4200,100,1470.7697137732232 +MarDec,4200,100,1468.1403207714204 +FedAvg,4200,100,2489.73589985142 +(MC)2MKP,4300,100,1468.169127592733 +MarIn,4300,100,1470.8218316146165 +MarCo,4300,100,1470.8218316146165 +MarDec,4300,100,1468.169127592733 +FedAvg,4300,100,2501.742264063312 +(MC)2MKP,4400,100,1468.1971872643105 +MarIn,4400,100,1470.8726713739384 +MarCo,4400,100,1470.8726713739384 +MarDec,4400,100,1468.1971872643105 +FedAvg,4400,100,2513.4787993662244 +(MC)2MKP,4500,100,1468.2245375653363 +MarIn,4500,100,1470.9222944900837 +MarCo,4500,100,1470.9222944900837 +MarDec,4500,100,1468.2245375653363 +FedAvg,4500,100,2524.9573678366714 +(MC)2MKP,4600,100,1468.251213480088 +MarIn,4600,100,1470.9707580590273 +MarCo,4600,100,1470.9707580590273 +MarDec,4600,100,1468.251213480088 +FedAvg,4600,100,2536.1890661339057 +(MC)2MKP,4700,100,1468.2772474670037 +MarIn,4700,100,1471.0181152351508 +MarCo,4700,100,1471.0181152351508 +MarDec,4700,100,1468.2772474670037 +FedAvg,4700,100,2547.184289968549 +(MC)2MKP,4800,100,1468.3026696961308 +MarIn,4800,100,1471.0644155871053 +MarCo,4800,100,1471.0644155871053 +MarDec,4800,100,1468.3026696961308 +FedAvg,4800,100,2557.952791923481 +(MC)2MKP,4900,100,1468.327508259312 +MarIn,4900,100,1471.1097054142758 +MarCo,4900,100,1471.1097054142758 +MarDec,4900,100,1468.327508259312 +FedAvg,4900,100,2568.5037334329813 +(MC)2MKP,5000,100,1468.3517893567785 +MarIn,5000,100,1471.1540280289853 +MarCo,5000,100,1471.1540280289853 +MarDec,5000,100,1468.3517893567785 +FedAvg,5000,100,2578.845731614345 diff --git a/chameleon_results/results_with_increasing_marginal_costs.csv b/chameleon_results/results_with_increasing_marginal_costs.csv new file mode 100644 index 0000000..5fb3495 --- /dev/null +++ b/chameleon_results/results_with_increasing_marginal_costs.csv @@ -0,0 +1,422 @@ +# Description of the experiment: +# +# - We generate the costs to up to 5.000 tasks for 10 and 100 resources. +# - All costs follow n log n functions (i.e., increasing marginal costs) +# with RNG seeds [200..299]. +# - We schedule from 1.000 to 5.000 tasks in increments of 100. +# - We run (MC)^2MKP, MarIn, MarCo, MarDec, and FedAvg. +# - All resources have a lower limit of 5. +# - The first half of the resources have no upper limit. +# - The second half has an upper limit of 2*(tasks/resources). +# - Every result is verified and logged to a CSV file. +Scheduler,Tasks,Resources,Total Cost +(MC)2MKP,1000,10,16504.237737296917 +MarIn,1000,10,16504.237737296917 +MarCo,1000,10,18179.191933591006 +MarDec,1000,10,18179.191933591006 +FedAvg,1000,10,22815.31622271911 +(MC)2MKP,1100,10,18471.945480597515 +MarIn,1100,10,18471.945480597515 +MarCo,1100,10,20315.873945022646 +MarDec,1100,10,20315.873945022646 +FedAvg,1100,10,25602.50432856914 +(MC)2MKP,1200,10,20467.215932494015 +MarIn,1200,10,20467.215932494015 +MarCo,1200,10,22478.191043909515 +MarDec,1200,10,22478.191043909515 +FedAvg,1200,10,28434.56808128068 +(MC)2MKP,1300,10,22487.68930817973 +MarIn,1300,10,22487.68930817973 +MarCo,1300,10,24663.917907229737 +MarDec,1300,10,24663.917907229737 +FedAvg,1300,10,31307.75928639905 +(MC)2MKP,1400,10,24531.298840775886 +MarIn,1400,10,24531.298840775886 +MarCo,1400,10,26871.18508124705 +MarDec,1400,10,26871.18508124705 +FedAvg,1400,10,34218.90815255527 +(MC)2MKP,1500,10,26596.531911117963 +MarIn,1500,10,26596.531911117963 +MarCo,1500,10,29098.399864823044 +MarDec,1500,10,29098.399864823044 +FedAvg,1500,10,37165.29889700973 +(MC)2MKP,1600,10,28681.723487364692 +MarIn,1600,10,28681.723487364692 +MarCo,1600,10,31344.189044183055 +MarDec,1600,10,31344.189044183055 +FedAvg,1600,10,40144.57865902697 +(MC)2MKP,1700,10,30785.511032825765 +MarIn,1700,10,30785.511032825765 +MarCo,1700,10,33607.35641749887 +MarDec,1700,10,33607.35641749887 +FedAvg,1700,10,43154.689265955014 +(MC)2MKP,1800,10,32906.962719693685 +MarIn,1800,10,32906.962719693685 +MarCo,1800,10,35886.85062145915 +MarDec,1800,10,35886.85062145915 +FedAvg,1800,10,46193.81510443521 +(MC)2MKP,1900,10,35044.907005694055 +MarIn,1900,10,35044.907005694055 +MarCo,1900,10,38181.74031348053 +MarDec,1900,10,38181.74031348053 +FedAvg,1900,10,49260.34260773272 +(MC)2MKP,2000,10,37198.62500113359 +MarIn,2000,10,37198.62500113359 +MarCo,2000,10,40491.19472014526 +MarDec,2000,10,40491.19472014526 +FedAvg,2000,10,52352.82829299111 +(MC)2MKP,2100,10,39367.27840156272 +MarIn,2100,10,39367.27840156272 +MarCo,2100,10,42814.468175255795 +MarDec,2100,10,42814.468175255795 +FedAvg,2100,10,55469.97320510124 +(MC)2MKP,2200,10,41550.08773970032 +MarIn,2200,10,41550.08773970032 +MarCo,2200,10,45150.887674148194 +MarDec,2200,10,45150.887674148194 +FedAvg,2200,10,58610.60223805842 +(MC)2MKP,2300,10,43746.41364662511 +MarIn,2300,10,43746.41364662511 +MarCo,2300,10,47499.84274271336 +MarDec,2300,10,47499.84274271336 +FedAvg,2300,10,61773.647222828105 +(MC)2MKP,2400,10,45955.691654790025 +MarIn,2400,10,45955.691654790025 +MarCo,2400,10,49860.77710674106 +MarDec,2400,10,49860.77710674106 +FedAvg,2400,10,64958.13296126284 +(MC)2MKP,2500,10,48177.348496976876 +MarIn,2500,10,48177.348496976876 +MarCo,2500,10,52233.18177857489 +MarDec,2500,10,52233.18177857489 +FedAvg,2500,10,68163.16559117998 +(MC)2MKP,2600,10,50410.70779252184 +MarIn,2600,10,50410.70779252184 +MarCo,2600,10,54616.5892718827 +MarDec,2600,10,54616.5892718827 +FedAvg,2600,10,71387.92281558138 +(MC)2MKP,2700,10,52655.48140088425 +MarIn,2700,10,52655.48140088425 +MarCo,2700,10,57010.568723398625 +MarDec,2700,10,57010.568723398625 +FedAvg,2700,10,74631.64563696955 +(MC)2MKP,2800,10,54911.16228729791 +MarIn,2800,10,54911.16228729791 +MarCo,2800,10,59414.72175056331 +MarDec,2800,10,59414.72175056331 +FedAvg,2800,10,77893.63131763964 +(MC)2MKP,2900,10,57177.38419301994 +MarIn,2900,10,57177.38419301994 +MarCo,2900,10,61828.67891131049 +MarDec,2900,10,61828.67891131049 +FedAvg,2900,10,81173.22734673227 +(MC)2MKP,3000,10,59453.80110912254 +MarIn,3000,10,59453.80110912254 +MarCo,3000,10,64252.09666040293 +MarDec,3000,10,64252.09666040293 +FedAvg,3000,10,84469.82624025406 +(MC)2MKP,3100,10,61740.11373076246 +MarIn,3100,10,61740.11373076246 +MarCo,3100,10,66684.65471819336 +MarDec,3100,10,66684.65471819336 +FedAvg,3100,10,87782.86103507865 +(MC)2MKP,3200,10,64035.95946086075 +MarIn,3200,10,64035.95946086075 +MarCo,3200,10,69126.0537842315 +MarDec,3200,10,69126.0537842315 +FedAvg,3200,10,91111.80136487592 +(MC)2MKP,3300,10,66340.99735871254 +MarIn,3300,10,66340.99735871254 +MarCo,3300,10,71576.01354100624 +MarDec,3300,10,71576.01354100624 +FedAvg,3300,10,94456.1500269521 +(MC)2MKP,3400,10,68654.97420972404 +MarIn,3400,10,68654.97420972404 +MarCo,3400,10,74034.2709032102 +MarDec,3400,10,74034.2709032102 +FedAvg,3400,10,97815.43996555067 +(MC)2MKP,3500,10,70977.64066208758 +MarIn,3500,10,70977.64066208758 +MarCo,3500,10,76500.57847590004 +MarDec,3500,10,76500.57847590004 +FedAvg,3500,10,101189.23161031531 +(MC)2MKP,3600,10,73308.78135174589 +MarIn,3600,10,73308.78135174589 +MarCo,3600,10,78974.70319129508 +MarDec,3600,10,78974.70319129508 +FedAvg,3600,10,104577.11051913587 +(MC)2MKP,3700,10,75647.93236700342 +MarIn,3700,10,75647.93236700342 +MarCo,3700,10,81456.42509906722 +MarDec,3700,10,81456.42509906722 +FedAvg,3700,10,107978.6852830684 +(MC)2MKP,3800,10,77995.08365774644 +MarIn,3800,10,77995.08365774644 +MarCo,3800,10,83945.53628910806 +MarDec,3800,10,83945.53628910806 +FedAvg,3800,10,111393.58565788779 +(MC)2MKP,3900,10,80349.96706728196 +MarIn,3900,10,80349.96706728196 +MarCo,3900,10,86441.8399291197 +MarDec,3900,10,86441.8399291197 +FedAvg,3900,10,114821.46089243225 +(MC)2MKP,4000,10,82712.41800102999 +MarIn,4000,10,82712.41800102999 +MarCo,4000,10,88945.14940212542 +MarDec,4000,10,88945.14940212542 +FedAvg,4000,10,118261.9782284928 +(MC)2MKP,4100,10,85082.21077067916 +MarIn,4100,10,85082.21077067916 +MarCo,4100,10,91455.28753125989 +MarDec,4100,10,91455.28753125989 +FedAvg,4100,10,121714.82155079152 +(MC)2MKP,4200,10,87459.12810471092 +MarIn,4200,10,87459.12810471092 +MarCo,4200,10,93972.08588106943 +MarDec,4200,10,93972.08588106943 +FedAvg,4200,10,125179.69016873375 +(MC)2MKP,4300,10,89843.04772255223 +MarIn,4300,10,89843.04772255223 +MarCo,4300,10,96495.38412611026 +MarDec,4300,10,96495.38412611026 +FedAvg,4300,10,128656.29771423843 +(MC)2MKP,4400,10,92233.73158810534 +MarIn,4400,10,92233.73158810534 +MarCo,4400,10,99025.02947893138 +MarDec,4400,10,99025.02947893138 +FedAvg,4400,10,132144.3711421412 +(MC)2MKP,4500,10,94631.13906971613 +MarIn,4500,10,94631.13906971613 +MarCo,4500,10,101560.87617062083 +MarDec,4500,10,101560.87617062083 +FedAvg,4500,10,135643.6498215088 +(MC)2MKP,4600,10,97035.01278149338 +MarIn,4600,10,97035.01278149338 +MarCo,4600,10,104102.78497801229 +MarDec,4600,10,104102.78497801229 +FedAvg,4600,10,139153.8847077582 +(MC)2MKP,4700,10,99445.266008627 +MarIn,4700,10,99445.266008627 +MarCo,4700,10,106650.62279242794 +MarDec,4700,10,106650.62279242794 +FedAvg,4700,10,142674.83758679338 +(MC)2MKP,4800,10,101861.78256723234 +MarIn,4800,10,101861.78256723234 +MarCo,4800,10,109204.26222549376 +MarDec,4800,10,109204.26222549376 +FedAvg,4800,10,146206.28038349465 +(MC)2MKP,4900,10,104284.3897334436 +MarIn,4900,10,104284.3897334436 +MarCo,4900,10,111763.58124812706 +MarDec,4900,10,111763.58124812706 +FedAvg,4900,10,149747.99452785455 +(MC)2MKP,5000,10,106712.99848550976 +MarIn,5000,10,106712.99848550976 +MarCo,5000,10,114328.46285927868 +MarDec,5000,10,114328.46285927868 +FedAvg,5000,10,153299.77037287492 +(MC)2MKP,1000,100,9097.141514777664 +MarIn,1000,100,9097.141514777664 +MarCo,1000,100,9838.565451531293 +MarDec,1000,100,9359.223771822657 +FedAvg,1000,100,13468.384693664135 +(MC)2MKP,1100,100,10007.305944663503 +MarIn,1100,100,10007.305944663503 +MarCo,1100,100,10901.738816377327 +MarDec,1100,100,10335.05850555785 +FedAvg,1100,100,15284.29152493378 +(MC)2MKP,1200,100,10943.38412132376 +MarIn,1200,100,10943.38412132376 +MarCo,1200,100,11989.64005511217 +MarDec,1200,100,11336.537816637381 +FedAvg,1200,100,17149.1267192598 +(MC)2MKP,1300,100,11901.851695150308 +MarIn,1300,100,11901.851695150308 +MarCo,1300,100,13098.711125871021 +MarDec,1300,100,12359.873277269784 +FedAvg,1300,100,19058.849911999503 +(MC)2MKP,1400,100,12879.933885148126 +MarIn,1400,100,12879.933885148126 +MarCo,1400,100,14226.293891378531 +MarDec,1400,100,13401.977853869683 +FedAvg,1400,100,21010.03535562356 +(MC)2MKP,1500,100,13875.771067277758 +MarIn,1500,100,13875.771067277758 +MarCo,1500,100,15370.32598884825 +MarDec,1500,100,14460.998824719114 +FedAvg,1500,100,22999.741868777106 +(MC)2MKP,1600,100,14887.356703465493 +MarIn,1600,100,14887.356703465493 +MarCo,1600,100,16529.16016814853 +MarDec,1600,100,15535.443854928451 +FedAvg,1600,100,25025.41700313279 +(MC)2MKP,1700,100,15913.306526314984 +MarIn,1700,100,15913.306526314984 +MarCo,1700,100,17701.450112728406 +MarDec,1700,100,16623.917304829094 +FedAvg,1700,100,27084.82488215448 +(MC)2MKP,1800,100,16952.506622331995 +MarIn,1800,100,16952.506622331995 +MarCo,1800,100,18886.074692632228 +MarDec,1800,100,17725.26028381005 +FedAvg,1800,100,29175.99083603472 +(MC)2MKP,1900,100,18004.000891558833 +MarIn,1900,100,18004.000891558833 +MarCo,1900,100,20082.085712656346 +MarDec,1900,100,18838.494320197344 +FedAvg,1900,100,31297.158219883757 +(MC)2MKP,2000,100,19067.118172721624 +MarIn,2000,100,19067.118172721624 +MarCo,2000,100,21288.670690167513 +MarDec,2000,100,19962.399659181232 +FedAvg,2000,100,33446.754242062685 +(MC)2MKP,2100,100,20140.705884572668 +MarIn,2100,100,20140.705884572668 +MarCo,2100,100,22505.12561492489 +MarDec,2100,100,21096.36754286566 +FedAvg,2100,100,35623.36257135817 +(MC)2MKP,2200,100,21224.31030629666 +MarIn,2200,100,21224.31030629666 +MarCo,2200,100,23730.834551377968 +MarDec,2200,100,22239.896933871 +FedAvg,2200,100,37825.70112296941 +(MC)2MKP,2300,100,22317.765565733338 +MarIn,2300,100,22317.765565733338 +MarCo,2300,100,24965.254059745446 +MarDec,2300,100,23392.433412509487 +FedAvg,2300,100,40052.6038557042 +(MC)2MKP,2400,100,23419.963485523982 +MarIn,2400,100,23419.963485523982 +MarCo,2400,100,26207.901090877633 +MarDec,2400,100,24553.48401837954 +FedAvg,2400,100,42303.00571482348 +(MC)2MKP,2500,100,24530.75257017652 +MarIn,2500,100,24530.75257017652 +MarCo,2500,100,27458.34343692607 +MarDec,2500,100,25722.607485332308 +FedAvg,2500,100,44575.93006968099 +(MC)2MKP,2600,100,25649.66554610624 +MarIn,2600,100,25649.66554610624 +MarCo,2600,100,28716.19209654376 +MarDec,2600,100,26899.40645361847 +FedAvg,2600,100,46870.47815037857 +(MC)2MKP,2700,100,26776.195238602926 +MarIn,2700,100,26776.195238602926 +MarCo,2700,100,29981.09509732982 +MarDec,2700,100,28083.521180624797 +FedAvg,2700,100,49185.82010129569 +(MC)2MKP,2800,100,27910.05140084935 +MarIn,2800,100,27910.05140084935 +MarCo,2800,100,31252.732443410685 +MarDec,2800,100,29274.624404688777 +FedAvg,2800,100,51521.18735373449 +(MC)2MKP,2900,100,29051.072108367687 +MarIn,2900,100,29051.072108367687 +MarCo,2900,100,32530.81194297302 +MarDec,2900,100,30472.41710818411 +FedAvg,2900,100,53875.86608334962 +(MC)2MKP,3000,100,30198.94929278067 +MarIn,3000,100,30198.94929278067 +MarCo,3000,100,33815.06573203968 +MarDec,3000,100,31676.624990505898 +FedAvg,3000,100,56249.19156624421 +(MC)2MKP,3100,100,31353.274346152197 +MarIn,3100,100,31353.274346152197 +MarCo,3100,100,35105.24735498545 +MarDec,3100,100,32886.99550765985 +FedAvg,3100,100,58640.54328464192 +(MC)2MKP,3200,100,32513.859058971797 +MarIn,3200,100,32513.859058971797 +MarCo,3200,100,36401.12929455727 +MarDec,3200,100,34102.74739659325 +FedAvg,3200,100,61049.340661758804 +(MC)2MKP,3300,100,33680.636971165935 +MarIn,3300,100,33680.636971165935 +MarCo,3300,100,37702.50086804315 +MarDec,3300,100,35323.414493821685 +FedAvg,3300,100,63475.03932796383 +(MC)2MKP,3400,100,34853.22205941196 +MarIn,3400,100,34853.22205941196 +MarCo,3400,100,39009.1664241286 +MarDec,3400,100,36549.511527009876 +FedAvg,3400,100,65917.12783804059 +(MC)2MKP,3500,100,36031.38323283188 +MarIn,3500,100,36031.38323283188 +MarCo,3500,100,40320.94378854578 +MarDec,3500,100,37780.85361775099 +FedAvg,3500,100,68375.1247734541 +(MC)2MKP,3600,100,37215.06108924295 +MarIn,3600,100,37215.06108924295 +MarCo,3600,100,41637.66291701408 +MarDec,3600,100,39017.268116245585 +FedAvg,3600,100,70848.57617481236 +(MC)2MKP,3700,100,38403.95676584752 +MarIn,3700,100,38403.95676584752 +MarCo,3700,100,42959.164722014066 +MarDec,3700,100,40258.59342251077 +FedAvg,3700,100,73337.0532588136 +(MC)2MKP,3800,100,39597.9935073622 +MarIn,3800,100,39597.9935073622 +MarCo,3800,100,44285.30004621704 +MarDec,3800,100,41504.67795483846 +FedAvg,3800,100,75840.15038135454 +(MC)2MKP,3900,100,40797.052615510336 +MarIn,3900,100,40797.052615510336 +MarCo,3900,100,45615.92876033901 +MarDec,3900,100,42755.379243148236 +FedAvg,3900,100,78357.48321450774 +(MC)2MKP,4000,100,42000.94535721057 +MarIn,4000,100,42000.94535721057 +MarCo,4000,100,46950.91896711418 +MarDec,4000,100,44010.563128841844 +FedAvg,4000,100,80888.68711002526 +(MC)2MKP,4100,100,43209.53238442003 +MarIn,4100,100,43209.53238442003 +MarCo,4100,100,48290.14629622371 +MarDec,4100,100,45270.10305593047 +FedAvg,4100,100,83433.41562611719 +(MC)2MKP,4200,100,44422.67071231483 +MarIn,4200,100,44422.67071231483 +MarCo,4200,100,49633.49327754445 +MarDec,4200,100,46533.87944075133 +FedAvg,4200,100,85991.33919764402 +(MC)2MKP,4300,100,45640.29446179644 +MarIn,4300,100,45640.29446179644 +MarCo,4300,100,50980.84878213225 +MarDec,4300,100,47801.77910965272 +FedAvg,4300,100,88562.1439326925 +(MC)2MKP,4400,100,46862.207027018994 +MarIn,4400,100,46862.207027018994 +MarCo,4400,100,52332.10752202717 +MarDec,4400,100,49073.69479570639 +FedAvg,4400,100,91145.53052087389 +(MC)2MKP,4500,100,48088.27893976314 +MarIn,4500,100,48088.27893976314 +MarCo,4500,100,53687.169601338916 +MarDec,4500,100,50349.524686884084 +FedAvg,4500,100,93741.21324067748 +(MC)2MKP,4600,100,49318.532450681494 +MarIn,4600,100,49318.532450681494 +MarCo,4600,100,55045.94011220326 +MarDec,4600,100,51629.1720192713 +FedAvg,4600,100,96348.91905489708 +(MC)2MKP,4700,100,50552.77260272408 +MarIn,4700,100,50552.77260272408 +MarCo,4700,100,56408.32877013744 +MarDec,4700,100,52912.544709832175 +FedAvg,4700,100,98968.38678457789 +(MC)2MKP,4800,100,51790.93047886026 +MarIn,4800,100,51790.93047886026 +MarCo,4800,100,57774.24958410542 +MarDec,4800,100,54199.55502402403 +FedAvg,4800,100,101599.36635314618 +(MC)2MKP,4900,100,53032.91272317893 +MarIn,4900,100,53032.91272317893 +MarCo,4900,100,59143.62055725754 +MarDec,4900,100,55490.11927421583 +FedAvg,4900,100,104241.61809342628 +(MC)2MKP,5000,100,54278.69028364971 +MarIn,5000,100,54278.69028364971 +MarCo,5000,100,60516.36341485953 +MarDec,5000,100,56784.15754541682 +FedAvg,5000,100,106894.91211113888 diff --git a/stored_results/results_with_random_costs.csv b/chameleon_results/results_with_random_costs.csv similarity index 100% rename from stored_results/results_with_random_costs.csv rename to chameleon_results/results_with_random_costs.csv diff --git a/chameleon_results/run_all_analysis.sh b/chameleon_results/run_all_analysis.sh new file mode 100755 index 0000000..1e0f2ce --- /dev/null +++ b/chameleon_results/run_all_analysis.sh @@ -0,0 +1,10 @@ +#!/usr/bin/env bash + +set -x + +find . -name "Analysis*.py" | while read line; do + echo "$line" >> result_analysis.txt + python3 "$line" >> result_analysis.txt +done + +echo "Please check file result_analysis.txt for the textual results and the pdf files for the figures." diff --git a/original_requirements.txt b/original_requirements.txt new file mode 100755 index 0000000..61ff4c1 --- /dev/null +++ b/original_requirements.txt @@ -0,0 +1,5 @@ +numpy>=1.22.2 +matplotlib>=3.5.1 +pandas>=1.4.3 +seaborn>=0.11.2 +scipy>=1.7.1 diff --git a/original_results/Analysis of execution times.ipynb b/original_results/Analysis of execution times.ipynb new file mode 100644 index 0000000..876f7d5 --- /dev/null +++ b/original_results/Analysis of execution times.ipynb @@ -0,0 +1,387 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Analysis of the experiments with different schedulers for Federated Learning - Execution time" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# modules for the analysis\n", + "import warnings\n", + "warnings.filterwarnings('ignore')\n", + "import pandas as pd\n", + "import seaborn as sns\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "from scipy import stats\n", + "\n", + "sns.set_theme(style=\"whitegrid\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# setting colors\n", + "c_mardec = [sns.color_palette(\"GnBu\", 10)[8]]\n", + "c_mardecun = [sns.color_palette(\"GnBu\", 10)[4]]\n", + "c_fedavg = [sns.color_palette(\"YlOrBr\", 10)[8]]\n", + "c_marin = [sns.color_palette(\"colorblind\", 10)[4]]\n", + "c_marco = [sns.color_palette(\"PuOr\", 10)[8]]\n", + "c_mc2mkp = [sns.color_palette(\"BuGn\", 10)[8]]\n", + "\n", + "sns.set_palette(c_mc2mkp + c_marin + c_marco + c_mardecun + c_mardec + c_fedavg)\n", + "\n", + "schedulers = ['(MC)2MKP', 'MarIn', 'MarCo', 'MarDecUn', 'MarDec', 'FedAvg']" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Results with increasing numbers of tasks" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# reads the result file\n", + "results = pd.read_csv('results_of_timing_with_fixed_resources.csv', comment='#')\n", + "print('Execution time analysis')\n", + "print('- Results with increasing numbers of tasks (fixed resources)')\n", + "results.head(9)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# checking the number of results versus the expected number of results\n", + "expected_number = 10*6*1*20 # 10 numbers of tasks, 6 schedulers, 1 number of resources, 20 samples\n", + "print(f'-- Number of results: {len(results)} (expected: {expected_number})')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Transforming the values of all samples\n", + "# Each sample contains the time of 5 repetitions\n", + "# This transformation gives the average time of the 5 repetitions\n", + "# and converts from seconds to us\n", + "results['avg'] = results['Time']*1000000/5" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print('-- Generating figure fig-time-fixed-resources.pdf for the results with a fixed number of resources')\n", + "\n", + "# Sets figure parameters\n", + "plt.figure(figsize=(6,5))\n", + "plt.rcParams['axes.titlesize'] = 13\n", + "plt.rcParams['axes.labelsize'] = 13\n", + "plt.rcParams['xtick.labelsize'] = 13\n", + "plt.rcParams['ytick.labelsize'] = 13\n", + "plt.rcParams['legend.fontsize'] = 12\n", + "plt.xlabel('Number of tasks (T)', fontsize=13)\n", + "plt.ylabel('Execution time (us, log scale)', fontsize=13)\n", + "plt.xticks(range(200,2001,200))\n", + "plt.xticks(rotation=15)\n", + "\n", + "ax = sns.lineplot(data=results[results.Resources == 100],\n", + " x='Tasks',\n", + " y='avg',\n", + " hue='Scheduler',\n", + " style='Scheduler',\n", + " dashes=False,\n", + " markers=True,\n", + " linewidth=2,\n", + " markersize=8,\n", + " hue_order=schedulers)\n", + "\n", + "plt.ylim(1, 100000000)\n", + "plt.yticks([1, 10, 100, 1000, 10000, 100000, 1000000, 100000000])\n", + "plt.yscale('log')\n", + "\n", + "sns.move_legend(ax, \"lower center\", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True)\n", + "\n", + "plt.savefig(\"fig-time-fixed-resources.pdf\", bbox_inches='tight')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Description of results" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print('- Result description: data distribution for 200 and 2000 tasks with different schedulers')\n", + "for sched in schedulers:\n", + " for tasks in (200, 2000):\n", + " res = results[(results['Scheduler'] == sched) &\n", + " (results['Tasks'] == tasks)].avg\n", + " print(f'Scheduler {sched} with {tasks} tasks')\n", + " print(res.describe())\n", + " print(' ')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Distributions of the results" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print('-- Checking the distribution of origin for different results')\n", + "print(' Kolmogorov-Smirnov test results with p-values < 0.05 mean that the results do not follow normal distributions')\n", + "# Checking all schedulers\n", + "np.random.seed(2022)\n", + "for sched in schedulers:\n", + " print(f'\\nResuls for scheduler {sched}:')\n", + " for tasks in range(200,2001,200):\n", + " res = list(results[(results['Scheduler'] == sched) &\n", + " (results['Tasks'] == tasks)].avg)\n", + " print(f'- {tasks} tasks')\n", + " print(stats.kstest(res, 'norm', args=(np.mean(res), np.std(res))))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Statistical comparison between MarCo and MarDecUn\n", + "# Using Mann-Whitney U test as results do not follow normal distributions some times (p-values < 0.05)\n", + "print(' ')\n", + "print('-- Comparing MarCo and MarDecUn. p-values < 0.05 indicate that the algorithms perform differently')\n", + "for tasks in range(200,2001,200):\n", + " marco = list(results[(results['Scheduler'] == 'MarCo') &\n", + " (results['Tasks'] == tasks)].avg)\n", + " mardec = list(results[(results['Scheduler'] == 'MarDec') &\n", + " (results['Tasks'] == tasks)].avg)\n", + " print(f'Mann-Whitney U test - ({tasks} tasks).')\n", + " print(stats.mannwhitneyu(marco, mardec, alternative='two-sided'))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "MarCo and MarDec perform differently (p-values < 0.05)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Results with increasing numbers of resources" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# reads the result file\n", + "results = pd.read_csv('results_of_timing_with_fixed_tasks.csv', comment='#')\n", + "print('\\n- Results with increasing numbers of resources (fixed tasks)')\n", + "results.head(9)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# checking the number of results versus the expected number of results\n", + "expected_number = 1*6*4*20 # 1 number of tasks, 6 schedulers, 4 numbers of resources, 20 samples\n", + "print(f'-- Number of results: {len(results)} (expected: {expected_number})')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Transforming the values of all samples\n", + "# Each sample contains the time of 5 repetitions\n", + "# This transformation gives the average time of the 5 repetitions\n", + "# and converts from seconds to us\n", + "results['avg'] = results['Time']*1000000/5" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print('-- Generating figure fig-time-fixed-tasks.pdf for the results with a fixed number of tasks')\n", + "\n", + "# Sets figure parameters\n", + "plt.figure(figsize=(6,5))\n", + "plt.rcParams['axes.titlesize'] = 13\n", + "plt.rcParams['axes.labelsize'] = 13\n", + "plt.rcParams['xtick.labelsize'] = 13\n", + "plt.rcParams['ytick.labelsize'] = 13\n", + "plt.rcParams['legend.fontsize'] = 12\n", + "plt.xlabel('Number of resources (n)', fontsize=13)\n", + "plt.ylabel('Execution time (us, log scale)', fontsize=13)\n", + "plt.xticks(range(20,81,20))\n", + "plt.xticks(rotation=15)\n", + "\n", + "ax = sns.lineplot(data=results[results.Tasks == 2000],\n", + " x='Resources',\n", + " y='avg',\n", + " hue='Scheduler',\n", + " style='Scheduler',\n", + " dashes=False,\n", + " markers=True,\n", + " linewidth=2,\n", + " markersize=8,\n", + " hue_order=schedulers)\n", + "\n", + "plt.ylim(1, 100000000)\n", + "plt.yticks([1, 10, 100, 1000, 10000, 100000, 1000000, 100000000])\n", + "plt.yscale('log')\n", + "\n", + "sns.move_legend(ax, \"lower center\", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True)\n", + "\n", + "plt.savefig(\"fig-time-fixed-tasks.pdf\", bbox_inches='tight')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Description of results" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print('- Result description: data distribution for 20 and 80 resources with different schedulers')\n", + "for sched in schedulers:\n", + " for resources in (20, 80):\n", + " res = results[(results['Scheduler'] == sched) &\n", + " (results['Resources'] == resources)].avg\n", + " print(f'Scheduler {sched} with {resources} resources')\n", + " print(res.describe())\n", + " print(' ')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Distributions of the results" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print('-- Checking the distribution of origin for different results')\n", + "print(' Kolmogorov-Smirnov test results with p-values < 0.05 mean that the results do not follow normal distributions')\n", + "# Checking all schedulers\n", + "np.random.seed(2022)\n", + "for sched in schedulers:\n", + " print(f'\\nResuls for scheduler {sched}:')\n", + " for resources in range(20,81,20):\n", + " res = list(results[(results['Scheduler'] == sched) &\n", + " (results['Resources'] == resources)].avg)\n", + " print(f'- {resources} resources')\n", + " print(stats.kstest(res, 'norm', args=(np.mean(res), np.std(res))))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Statistical comparison between MarCo and MarDecUn\n", + "# Using Mann-Whitney U test as results do not follow normal distributions some times (p-values < 0.05)\n", + "print(' ')\n", + "print('-- Comparing MarCo and MarDecUn. p-values < 0.05 indicate that the algorithms perform differently')\n", + "for resources in range(20,81,20):\n", + " marco = list(results[(results['Scheduler'] == 'MarCo') &\n", + " (results['Resources'] == resources)].avg)\n", + " mardec = list(results[(results['Scheduler'] == 'MarDec') &\n", + " (results['Resources'] == resources)].avg)\n", + " print(f'Mann-Whitney U test - ({resources} resources).')\n", + " print(stats.mannwhitneyu(marco, mardec, alternative='two-sided'))\n", + "print('\\n') " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "MarCo and MarDec perform differently (p-values < 0.05)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.10" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/original_results/Analysis of execution times.py b/original_results/Analysis of execution times.py new file mode 100644 index 0000000..2c2f9a5 --- /dev/null +++ b/original_results/Analysis of execution times.py @@ -0,0 +1,271 @@ +#!/usr/bin/env python +# coding: utf-8 + +# # Analysis of the experiments with different schedulers for Federated Learning - Execution time + +# In[ ]: + + +# modules for the analysis +import warnings +warnings.filterwarnings('ignore') +import pandas as pd +import seaborn as sns +import matplotlib.pyplot as plt +import numpy as np +from scipy import stats + +sns.set_theme(style="whitegrid") + + +# In[ ]: + + +# setting colors +c_mardec = [sns.color_palette("GnBu", 10)[8]] +c_mardecun = [sns.color_palette("GnBu", 10)[4]] +c_fedavg = [sns.color_palette("YlOrBr", 10)[8]] +c_marin = [sns.color_palette("colorblind", 10)[4]] +c_marco = [sns.color_palette("PuOr", 10)[8]] +c_mc2mkp = [sns.color_palette("BuGn", 10)[8]] + +sns.set_palette(c_mc2mkp + c_marin + c_marco + c_mardecun + c_mardec + c_fedavg) + +schedulers = ['(MC)2MKP', 'MarIn', 'MarCo', 'MarDecUn', 'MarDec', 'FedAvg'] + + +# ## Results with increasing numbers of tasks + +# In[ ]: + + +# reads the result file +results = pd.read_csv('results_of_timing_with_fixed_resources.csv', comment='#') +print('Execution time analysis') +print('- Results with increasing numbers of tasks (fixed resources)') +results.head(9) + + +# In[ ]: + + +# checking the number of results versus the expected number of results +expected_number = 10*6*1*20 # 10 numbers of tasks, 6 schedulers, 1 number of resources, 20 samples +print(f'-- Number of results: {len(results)} (expected: {expected_number})') + + +# In[ ]: + + +# Transforming the values of all samples +# Each sample contains the time of 5 repetitions +# This transformation gives the average time of the 5 repetitions +# and converts from seconds to us +results['avg'] = results['Time']*1000000/5 + + +# In[ ]: + + +print('-- Generating figure fig-time-fixed-resources.pdf for the results with a fixed number of resources') + +# Sets figure parameters +plt.figure(figsize=(6,5)) +plt.rcParams['axes.titlesize'] = 13 +plt.rcParams['axes.labelsize'] = 13 +plt.rcParams['xtick.labelsize'] = 13 +plt.rcParams['ytick.labelsize'] = 13 +plt.rcParams['legend.fontsize'] = 12 +plt.xlabel('Number of tasks (T)', fontsize=13) +plt.ylabel('Execution time (us, log scale)', fontsize=13) +plt.xticks(range(200,2001,200)) +plt.xticks(rotation=15) + +ax = sns.lineplot(data=results[results.Resources == 100], + x='Tasks', + y='avg', + hue='Scheduler', + style='Scheduler', + dashes=False, + markers=True, + linewidth=2, + markersize=8, + hue_order=schedulers) + +plt.ylim(1, 100000000) +plt.yticks([1, 10, 100, 1000, 10000, 100000, 1000000, 100000000]) +plt.yscale('log') + +sns.move_legend(ax, "lower center", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True) + +plt.savefig("fig-time-fixed-resources.pdf", bbox_inches='tight') + + +# ### Description of results + +# In[ ]: + + +print('- Result description: data distribution for 200 and 2000 tasks with different schedulers') +for sched in schedulers: + for tasks in (200, 2000): + res = results[(results['Scheduler'] == sched) & + (results['Tasks'] == tasks)].avg + print(f'Scheduler {sched} with {tasks} tasks') + print(res.describe()) + print(' ') + + +# ### Distributions of the results + +# In[ ]: + + +print('-- Checking the distribution of origin for different results') +print(' Kolmogorov-Smirnov test results with p-values < 0.05 mean that the results do not follow normal distributions') +# Checking all schedulers +np.random.seed(2022) +for sched in schedulers: + print(f'\nResuls for scheduler {sched}:') + for tasks in range(200,2001,200): + res = list(results[(results['Scheduler'] == sched) & + (results['Tasks'] == tasks)].avg) + print(f'- {tasks} tasks') + print(stats.kstest(res, 'norm', args=(np.mean(res), np.std(res)))) + + +# In[ ]: + + +# Statistical comparison between MarCo and MarDecUn +# Using Mann-Whitney U test as results do not follow normal distributions some times (p-values < 0.05) +print(' ') +print('-- Comparing MarCo and MarDecUn. p-values < 0.05 indicate that the algorithms perform differently') +for tasks in range(200,2001,200): + marco = list(results[(results['Scheduler'] == 'MarCo') & + (results['Tasks'] == tasks)].avg) + mardec = list(results[(results['Scheduler'] == 'MarDec') & + (results['Tasks'] == tasks)].avg) + print(f'Mann-Whitney U test - ({tasks} tasks).') + print(stats.mannwhitneyu(marco, mardec, alternative='two-sided')) + + +# MarCo and MarDec perform differently (p-values < 0.05). + +# ## Results with increasing numbers of resources + +# In[ ]: + + +# reads the result file +results = pd.read_csv('results_of_timing_with_fixed_tasks.csv', comment='#') +print('\n- Results with increasing numbers of resources (fixed tasks)') +results.head(9) + + +# In[ ]: + + +# checking the number of results versus the expected number of results +expected_number = 1*6*4*20 # 1 number of tasks, 6 schedulers, 4 numbers of resources, 20 samples +print(f'-- Number of results: {len(results)} (expected: {expected_number})') + + +# In[ ]: + + +# Transforming the values of all samples +# Each sample contains the time of 5 repetitions +# This transformation gives the average time of the 5 repetitions +# and converts from seconds to us +results['avg'] = results['Time']*1000000/5 + + +# In[ ]: + + +print('-- Generating figure fig-time-fixed-tasks.pdf for the results with a fixed number of tasks') + +# Sets figure parameters +plt.figure(figsize=(6,5)) +plt.rcParams['axes.titlesize'] = 13 +plt.rcParams['axes.labelsize'] = 13 +plt.rcParams['xtick.labelsize'] = 13 +plt.rcParams['ytick.labelsize'] = 13 +plt.rcParams['legend.fontsize'] = 12 +plt.xlabel('Number of resources (n)', fontsize=13) +plt.ylabel('Execution time (us, log scale)', fontsize=13) +plt.xticks(range(20,81,20)) +plt.xticks(rotation=15) + +ax = sns.lineplot(data=results[results.Tasks == 2000], + x='Resources', + y='avg', + hue='Scheduler', + style='Scheduler', + dashes=False, + markers=True, + linewidth=2, + markersize=8, + hue_order=schedulers) + +plt.ylim(1, 100000000) +plt.yticks([1, 10, 100, 1000, 10000, 100000, 1000000, 100000000]) +plt.yscale('log') + +sns.move_legend(ax, "lower center", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True) + +plt.savefig("fig-time-fixed-tasks.pdf", bbox_inches='tight') + + +# ### Description of results + +# In[ ]: + + +print('- Result description: data distribution for 20 and 80 resources with different schedulers') +for sched in schedulers: + for resources in (20, 80): + res = results[(results['Scheduler'] == sched) & + (results['Resources'] == resources)].avg + print(f'Scheduler {sched} with {resources} resources') + print(res.describe()) + print(' ') + + +# ### Distributions of the results + +# In[ ]: + + +print('-- Checking the distribution of origin for different results') +print(' Kolmogorov-Smirnov test results with p-values < 0.05 mean that the results do not follow normal distributions') +# Checking all schedulers +np.random.seed(2022) +for sched in schedulers: + print(f'\nResuls for scheduler {sched}:') + for resources in range(20,81,20): + res = list(results[(results['Scheduler'] == sched) & + (results['Resources'] == resources)].avg) + print(f'- {resources} resources') + print(stats.kstest(res, 'norm', args=(np.mean(res), np.std(res)))) + + +# In[ ]: + + +# Statistical comparison between MarCo and MarDecUn +# Using Mann-Whitney U test as results do not follow normal distributions some times (p-values < 0.05) +print(' ') +print('-- Comparing MarCo and MarDecUn. p-values < 0.05 indicate that the algorithms perform differently') +for resources in range(20,81,20): + marco = list(results[(results['Scheduler'] == 'MarCo') & + (results['Resources'] == resources)].avg) + mardec = list(results[(results['Scheduler'] == 'MarDec') & + (results['Resources'] == resources)].avg) + print(f'Mann-Whitney U test - ({resources} resources).') + print(stats.mannwhitneyu(marco, mardec, alternative='two-sided')) +print('\n') + + +# MarCo and MarDec perform differently (p-values < 0.05). diff --git a/original_results/Analysis of total costs.ipynb b/original_results/Analysis of total costs.ipynb new file mode 100644 index 0000000..24bb77c --- /dev/null +++ b/original_results/Analysis of total costs.ipynb @@ -0,0 +1,675 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Analysis of the experiments with different schedulers for Federated Learning - Total cost" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# modules for the analysis\n", + "import warnings\n", + "warnings.filterwarnings('ignore')\n", + "import pandas as pd\n", + "import seaborn as sns\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "\n", + "sns.set_theme(style=\"whitegrid\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# setting colors\n", + "c_mardec = [sns.color_palette(\"GnBu\", 10)[8]]\n", + "c_fedavg = [sns.color_palette(\"YlOrBr\", 10)[8]]\n", + "c_marin = [sns.color_palette(\"colorblind\", 10)[4]]\n", + "c_marco = [sns.color_palette(\"PuOr\", 10)[8]]\n", + "c_mc2mkp = [sns.color_palette(\"BuGn\", 10)[8]]\n", + "\n", + "sns.set_palette(c_mc2mkp + c_marin + c_marco + c_mardec + c_fedavg)\n", + "\n", + "schedulers = ['(MC)2MKP', 'MarIn', 'MarCo', 'MarDec', 'FedAvg']" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Results with random costs" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# reads the result file\n", + "results = pd.read_csv('results_with_random_costs.csv', comment='#')\n", + "print('Total cost analysis')\n", + "print('- Results with random costs')\n", + "results.head(9)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# checking the number of results versus the expected number of results\n", + "expected_number = 41*5*2 # 41 numbers of tasks, 5 schedulers, 2 numbers of resources\n", + "print(f'-- Number of results: {len(results)} (expected: {expected_number})')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print('-- Generating figures fig-random-10.pdf and fig-random-100.pdf')\n", + "\n", + "# Sets figure parameters\n", + "plt.figure(figsize=(6,5))\n", + "plt.rcParams['axes.titlesize'] = 13\n", + "plt.rcParams['axes.labelsize'] = 13\n", + "plt.rcParams['xtick.labelsize'] = 13\n", + "plt.rcParams['ytick.labelsize'] = 13\n", + "plt.rcParams['legend.fontsize'] = 12\n", + "plt.xlabel('Number of tasks (T)', fontsize=13)\n", + "plt.ylabel('Total cost (a.u.)', fontsize=13)\n", + "plt.xticks(rotation=15)\n", + "\n", + "ax = sns.lineplot(data=results[results.Resources == 10],\n", + " x='Tasks',\n", + " y='Total Cost',\n", + " hue='Scheduler',\n", + " style='Scheduler',\n", + " dashes=False,\n", + " markers=True,\n", + " linewidth=2,\n", + " markersize=8)\n", + "\n", + "sns.move_legend(ax, \"lower center\", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True)\n", + "\n", + "plt.savefig(\"fig-random-10.pdf\", bbox_inches='tight')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Sets figure parameters\n", + "plt.figure(figsize=(6,5))\n", + "plt.rcParams['axes.titlesize'] = 13\n", + "plt.rcParams['axes.labelsize'] = 13\n", + "plt.rcParams['xtick.labelsize'] = 13\n", + "plt.rcParams['ytick.labelsize'] = 13\n", + "plt.rcParams['legend.fontsize'] = 12\n", + "plt.xlabel('Number of tasks (T)', fontsize=13)\n", + "plt.ylabel('Total cost (a.u.)', fontsize=13)\n", + "plt.xticks(rotation=15)\n", + "\n", + "ax = sns.lineplot(data=results[results.Resources == 100],\n", + " x='Tasks',\n", + " y='Total Cost',\n", + " hue='Scheduler',\n", + " style='Scheduler',\n", + " dashes=False,\n", + " markers=True,\n", + " linewidth=2,\n", + " markersize=8)\n", + "\n", + "sns.move_legend(ax, \"lower center\", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True)\n", + "\n", + "plt.savefig(\"fig-random-100.pdf\", bbox_inches='tight')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Checking how many times other schedulers meet the performance of (MC)^2MKP\n", + "print('-- Checking how many times other schedulers meet the performance of (MC)2MKP')\n", + "mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True)\n", + "for scheduler in schedulers[1:]:\n", + " other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True)\n", + " greater = np.sum(other_cost > mc2mkp_cost)\n", + " equal = np.sum(other_cost == mc2mkp_cost)\n", + " less = np.sum(other_cost < mc2mkp_cost)\n", + " print(f'Number of times {scheduler} provides a Total Cost that is greater, equal, or smaller than (MC)^2MKP: ' +\n", + " f'{greater}, {equal}, {less}.')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "## Results with increasing marginal costs" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# reads the result file\n", + "results = pd.read_csv('results_with_increasing_marginal_costs.csv', comment='#')\n", + "print('\\n- Results with increasing marginal costs')\n", + "results.head(9)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print(f'-- Number of results: {len(results)} (expected: {expected_number})')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print('-- Generating figures fig-increasing-10.pdf and fig-increasing-100.pdf')\n", + "\n", + "# Sets figure parameters\n", + "plt.figure(figsize=(6,5))\n", + "plt.rcParams['axes.titlesize'] = 13\n", + "plt.rcParams['axes.labelsize'] = 13\n", + "plt.rcParams['xtick.labelsize'] = 13\n", + "plt.rcParams['ytick.labelsize'] = 13\n", + "plt.rcParams['legend.fontsize'] = 12\n", + "plt.xlabel('Number of tasks (T)', fontsize=13)\n", + "plt.ylabel('Total cost (a.u.)', fontsize=13)\n", + "plt.xticks(rotation=15)\n", + "plt.ylim(0, 160000)\n", + "\n", + "ax = sns.lineplot(data=results[results.Resources == 10],\n", + " x='Tasks',\n", + " y='Total Cost',\n", + " hue='Scheduler',\n", + " style='Scheduler',\n", + " dashes=False,\n", + " markers=True,\n", + " linewidth=2,\n", + " markersize=8)\n", + "\n", + "sns.move_legend(ax, \"lower center\", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True)\n", + "\n", + "plt.savefig(\"fig-increasing-10.pdf\", bbox_inches='tight')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Sets figure parameters\n", + "plt.figure(figsize=(6,5))\n", + "plt.rcParams['axes.titlesize'] = 13\n", + "plt.rcParams['axes.labelsize'] = 13\n", + "plt.rcParams['xtick.labelsize'] = 13\n", + "plt.rcParams['ytick.labelsize'] = 13\n", + "plt.rcParams['legend.fontsize'] = 12\n", + "plt.xlabel('Number of tasks (T)', fontsize=13)\n", + "plt.ylabel('Total cost (a.u.)', fontsize=13)\n", + "plt.xticks(rotation=15)\n", + "plt.ylim(0, 120000)\n", + "\n", + "ax = sns.lineplot(data=results[results.Resources == 100],\n", + " x='Tasks',\n", + " y='Total Cost',\n", + " hue='Scheduler',\n", + " style='Scheduler',\n", + " dashes=False,\n", + " markers=True,\n", + " linewidth=2,\n", + " markersize=8)\n", + "\n", + "sns.move_legend(ax, \"lower center\", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True)\n", + "\n", + "plt.savefig(\"fig-increasing-100.pdf\", bbox_inches='tight')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Checking how many times other schedulers meet the performance of (MC)^2MKP\n", + "print('-- Checking how many times other schedulers meet the performance of (MC)2MKP')\n", + "mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True)\n", + "for scheduler in schedulers[1:]:\n", + " other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True)\n", + " greater = np.sum(other_cost > mc2mkp_cost)\n", + " equal = np.sum(other_cost == mc2mkp_cost)\n", + " less = np.sum(other_cost < mc2mkp_cost)\n", + " print(f'Number of times {scheduler} provides a Total Cost that is greater, equal, or smaller than (MC)^2MKP: ' +\n", + " f'{greater}, {equal}, {less}.')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "## Results with constant marginal costs" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# reads the result file\n", + "results = pd.read_csv('results_with_constant_marginal_costs.csv', comment='#')\n", + "print('\\n- Results with constant marginal costs')\n", + "results.head(9)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print(f'-- Number of results: {len(results)} (expected: {expected_number})')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print('-- Generating figures fig-constant-10.pdf and fig-constant-100.pdf')\n", + "\n", + "# Sets figure parameters\n", + "plt.figure(figsize=(6,5))\n", + "plt.rcParams['axes.titlesize'] = 13\n", + "plt.rcParams['axes.labelsize'] = 13\n", + "plt.rcParams['xtick.labelsize'] = 13\n", + "plt.rcParams['ytick.labelsize'] = 13\n", + "plt.rcParams['legend.fontsize'] = 12\n", + "plt.xlabel('Number of tasks (T)', fontsize=13)\n", + "plt.ylabel('Total cost (a.u.)', fontsize=13)\n", + "plt.xticks(rotation=15)\n", + "plt.ylim(0, 30000)\n", + "\n", + "ax = sns.lineplot(data=results[results.Resources == 10],\n", + " x='Tasks',\n", + " y='Total Cost',\n", + " hue='Scheduler',\n", + " style='Scheduler',\n", + " dashes=False,\n", + " markers=True,\n", + " linewidth=2,\n", + " markersize=8)\n", + "\n", + "sns.move_legend(ax, \"lower center\", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True)\n", + "\n", + "plt.savefig(\"fig-constant-10.pdf\", bbox_inches='tight')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Sets figure parameters\n", + "plt.figure(figsize=(6,5))\n", + "plt.rcParams['axes.titlesize'] = 13\n", + "plt.rcParams['axes.labelsize'] = 13\n", + "plt.rcParams['xtick.labelsize'] = 13\n", + "plt.rcParams['ytick.labelsize'] = 13\n", + "plt.rcParams['legend.fontsize'] = 12\n", + "plt.xlabel('Number of tasks (T)', fontsize=13)\n", + "plt.ylabel('Total cost (a.u.)', fontsize=13)\n", + "plt.xticks(rotation=15)\n", + "plt.ylim(0, 30000)\n", + "\n", + "ax = sns.lineplot(data=results[results.Resources == 100],\n", + " x='Tasks',\n", + " y='Total Cost',\n", + " hue='Scheduler',\n", + " style='Scheduler',\n", + " dashes=False,\n", + " markers=True,\n", + " linewidth=2,\n", + " markersize=8)\n", + "\n", + "sns.move_legend(ax, \"lower center\", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True)\n", + "\n", + "plt.savefig(\"fig-constant-100.pdf\", bbox_inches='tight')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Checking how many times other schedulers meet the performance of (MC)^2MKP\n", + "print('-- Checking how many times other schedulers meet the performance of (MC)2MKP')\n", + "mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True)\n", + "for scheduler in schedulers[1:]:\n", + " other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True)\n", + " greater = np.sum(other_cost > mc2mkp_cost)\n", + " equal = np.sum(other_cost == mc2mkp_cost)\n", + " less = np.sum(other_cost < mc2mkp_cost)\n", + " print(f'Number of times {scheduler} provides a Total Cost that is greater, equal, or smaller than (MC)^2MKP: ' +\n", + " f'{greater}, {equal}, {less}.')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "## Results with decreasing marginal costs" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# reads the result file\n", + "results = pd.read_csv('results_with_decreasing_marginal_costs.csv', comment='#')\n", + "print('\\n- Results with decreasing marginal costs')\n", + "results.head(9)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print(f'-- Number of results: {len(results)} (expected: {expected_number})')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print('-- Generating figures fig-decreasing-10.pdf, fig-decreasing-100.pdf, and fig-decreasing-100-zoom.pdf')\n", + "\n", + "# Sets figure parameters\n", + "plt.figure(figsize=(6,5))\n", + "plt.rcParams['axes.titlesize'] = 13\n", + "plt.rcParams['axes.labelsize'] = 13\n", + "plt.rcParams['xtick.labelsize'] = 13\n", + "plt.rcParams['ytick.labelsize'] = 13\n", + "plt.rcParams['legend.fontsize'] = 12\n", + "plt.xlabel('Number of tasks (T)', fontsize=13)\n", + "plt.ylabel('Total cost (a.u.)', fontsize=13)\n", + "plt.xticks(rotation=15)\n", + "plt.ylim(0, 500)\n", + "\n", + "ax = sns.lineplot(data=results[results.Resources == 10],\n", + " x='Tasks',\n", + " y='Total Cost',\n", + " hue='Scheduler',\n", + " style='Scheduler',\n", + " dashes=False,\n", + " markers=True,\n", + " linewidth=2,\n", + " markersize=8)\n", + "\n", + "sns.move_legend(ax, \"lower center\", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True)\n", + "\n", + "plt.savefig(\"fig-decreasing-10.pdf\", bbox_inches='tight')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Sets figure parameters\n", + "plt.figure(figsize=(6,5))\n", + "plt.rcParams['axes.titlesize'] = 13\n", + "plt.rcParams['axes.labelsize'] = 13\n", + "plt.rcParams['xtick.labelsize'] = 13\n", + "plt.rcParams['ytick.labelsize'] = 13\n", + "plt.rcParams['legend.fontsize'] = 12\n", + "plt.xlabel('Number of tasks (T)', fontsize=13)\n", + "plt.ylabel('Total cost (a.u.)', fontsize=13)\n", + "plt.xticks(rotation=15)\n", + "plt.ylim(0, 3000)\n", + "\n", + "ax = sns.lineplot(data=results[results.Resources == 100],\n", + " x='Tasks',\n", + " y='Total Cost',\n", + " hue='Scheduler',\n", + " style='Scheduler',\n", + " dashes=False,\n", + " markers=True,\n", + " linewidth=2,\n", + " markersize=8)\n", + "\n", + "sns.move_legend(ax, \"lower center\", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True)\n", + "\n", + "plt.savefig(\"fig-decreasing-100.pdf\", bbox_inches='tight')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Sets figure parameters\n", + "plt.figure(figsize=(6,5))\n", + "plt.rcParams['axes.titlesize'] = 13\n", + "plt.rcParams['axes.labelsize'] = 13\n", + "plt.rcParams['xtick.labelsize'] = 13\n", + "plt.rcParams['ytick.labelsize'] = 13\n", + "plt.rcParams['legend.fontsize'] = 12\n", + "plt.xlabel('Number of tasks (T)', fontsize=13)\n", + "plt.ylabel('Total cost (a.u.)', fontsize=13)\n", + "plt.xticks(rotation=15)\n", + "plt.ylim(1465, 1472)\n", + "\n", + "ax = sns.lineplot(data=results[results.Resources == 100],\n", + " x='Tasks',\n", + " y='Total Cost',\n", + " hue='Scheduler',\n", + " style='Scheduler',\n", + " dashes=False,\n", + " markers=True,\n", + " linewidth=2,\n", + " markersize=8)\n", + "\n", + "sns.move_legend(ax, \"lower center\", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True)\n", + "\n", + "plt.savefig(\"fig-decreasing-100-zoom.pdf\", bbox_inches='tight')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Checking how many times other schedulers meet the performance of (MC)^2MKP\n", + "print('-- Checking how many times other schedulers meet the performance of (MC)2MKP')\n", + "mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True)\n", + "for scheduler in schedulers[1:]:\n", + " other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True)\n", + " greater = np.sum(other_cost > mc2mkp_cost)\n", + " equal = np.sum(other_cost == mc2mkp_cost)\n", + " less = np.sum(other_cost < mc2mkp_cost)\n", + " print(f'Number of times {scheduler} provides a Total Cost that is greater, equal, or smaller than (MC)^2MKP: ' +\n", + " f'{greater}, {equal}, {less}.')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "## Results with constant marginal costs and no upper limits" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# reads the result file\n", + "results = pd.read_csv('results_with_constant_marginal_costs_no_upper_limit.csv', comment='#')\n", + "print('\\n- Results with constant marginal costs and no upper limits')\n", + "results.head(9)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print(f'-- Number of results: {len(results)} (expected: {expected_number})')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print('-- Generating figures fig-constant-10-no-u.pdf and fig-constant-100-no-u.pdf')\n", + "\n", + "# Sets figure parameters\n", + "plt.figure(figsize=(6,5))\n", + "plt.rcParams['axes.titlesize'] = 13\n", + "plt.rcParams['axes.labelsize'] = 13\n", + "plt.rcParams['xtick.labelsize'] = 13\n", + "plt.rcParams['ytick.labelsize'] = 13\n", + "plt.rcParams['legend.fontsize'] = 12\n", + "plt.xlabel('Number of tasks (T)', fontsize=13)\n", + "plt.ylabel('Total cost (a.u.)', fontsize=13)\n", + "plt.xticks(rotation=15)\n", + "plt.ylim(0, 30000)\n", + "\n", + "ax = sns.lineplot(data=results[results.Resources == 10],\n", + " x='Tasks',\n", + " y='Total Cost',\n", + " hue='Scheduler',\n", + " style='Scheduler',\n", + " dashes=False,\n", + " markers=True,\n", + " linewidth=2,\n", + " markersize=8)\n", + "\n", + "sns.move_legend(ax, \"lower center\", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True)\n", + "\n", + "plt.savefig(\"fig-constant-10-no-u.pdf\", bbox_inches='tight')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Sets figure parameters\n", + "plt.figure(figsize=(6,5))\n", + "plt.rcParams['axes.titlesize'] = 13\n", + "plt.rcParams['axes.labelsize'] = 13\n", + "plt.rcParams['xtick.labelsize'] = 13\n", + "plt.rcParams['ytick.labelsize'] = 13\n", + "plt.rcParams['legend.fontsize'] = 12\n", + "plt.xlabel('Number of tasks (T)', fontsize=13)\n", + "plt.ylabel('Total cost (a.u.)', fontsize=13)\n", + "plt.xticks(rotation=15)\n", + "plt.ylim(0, 30000)\n", + "\n", + "ax = sns.lineplot(data=results[results.Resources == 100],\n", + " x='Tasks',\n", + " y='Total Cost',\n", + " hue='Scheduler',\n", + " style='Scheduler',\n", + " dashes=False,\n", + " markers=True,\n", + " linewidth=2,\n", + " markersize=8)\n", + "\n", + "sns.move_legend(ax, \"lower center\", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True)\n", + "\n", + "plt.savefig(\"fig-constant-100-no-u.pdf\", bbox_inches='tight')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Checking how many times other schedulers meet the performance of (MC)^2MKP\n", + "print('-- Checking how many times other schedulers meet the performance of (MC)2MKP')\n", + "mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True)\n", + "for scheduler in ['MarIn', 'MarCo', 'MarDecUn', 'FedAvg']:\n", + " other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True)\n", + " greater = np.sum(other_cost > mc2mkp_cost)\n", + " equal = np.sum(other_cost == mc2mkp_cost)\n", + " less = np.sum(other_cost < mc2mkp_cost)\n", + " print(f'Number of times {scheduler} provides a Total Cost that is greater, equal, or smaller than (MC)^2MKP: ' +\n", + " f'{greater}, {equal}, {less}.')" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.10" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/original_results/Analysis of total costs.py b/original_results/Analysis of total costs.py new file mode 100644 index 0000000..c125b87 --- /dev/null +++ b/original_results/Analysis of total costs.py @@ -0,0 +1,538 @@ +#!/usr/bin/env python +# coding: utf-8 + +# # Analysis of the experiments with different schedulers for Federated Learning - Total cost + +# In[ ]: + + +# modules for the analysis +import warnings +warnings.filterwarnings('ignore') +import pandas as pd +import seaborn as sns +import matplotlib.pyplot as plt +import numpy as np + +sns.set_theme(style="whitegrid") + + +# In[ ]: + + +# setting colors +c_mardec = [sns.color_palette("GnBu", 10)[8]] +c_fedavg = [sns.color_palette("YlOrBr", 10)[8]] +c_marin = [sns.color_palette("colorblind", 10)[4]] +c_marco = [sns.color_palette("PuOr", 10)[8]] +c_mc2mkp = [sns.color_palette("BuGn", 10)[8]] + +sns.set_palette(c_mc2mkp + c_marin + c_marco + c_mardec + c_fedavg) + +schedulers = ['(MC)2MKP', 'MarIn', 'MarCo', 'MarDec', 'FedAvg'] + + +# ## Results with random costs + +# In[ ]: + + +# reads the result file +results = pd.read_csv('results_with_random_costs.csv', comment='#') +print('Total cost analysis') +print('- Results with random costs') +results.head(9) + + +# In[ ]: + + +# checking the number of results versus the expected number of results +expected_number = 41*5*2 # 41 numbers of tasks, 5 schedulers, 2 numbers of resources +print(f'-- Number of results: {len(results)} (expected: {expected_number})') + + +# In[ ]: + + +print('-- Generating figures fig-random-10.pdf and fig-random-100.pdf') + +# Sets figure parameters +plt.figure(figsize=(6,5)) +plt.rcParams['axes.titlesize'] = 13 +plt.rcParams['axes.labelsize'] = 13 +plt.rcParams['xtick.labelsize'] = 13 +plt.rcParams['ytick.labelsize'] = 13 +plt.rcParams['legend.fontsize'] = 12 +plt.xlabel('Number of tasks (T)', fontsize=13) +plt.ylabel('Total cost (a.u.)', fontsize=13) +plt.xticks(rotation=15) + +ax = sns.lineplot(data=results[results.Resources == 10], + x='Tasks', + y='Total Cost', + hue='Scheduler', + style='Scheduler', + dashes=False, + markers=True, + linewidth=2, + markersize=8) + +sns.move_legend(ax, "lower center", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True) + +plt.savefig("fig-random-10.pdf", bbox_inches='tight') + + +# In[ ]: + + +# Sets figure parameters +plt.figure(figsize=(6,5)) +plt.rcParams['axes.titlesize'] = 13 +plt.rcParams['axes.labelsize'] = 13 +plt.rcParams['xtick.labelsize'] = 13 +plt.rcParams['ytick.labelsize'] = 13 +plt.rcParams['legend.fontsize'] = 12 +plt.xlabel('Number of tasks (T)', fontsize=13) +plt.ylabel('Total cost (a.u.)', fontsize=13) +plt.xticks(rotation=15) + +ax = sns.lineplot(data=results[results.Resources == 100], + x='Tasks', + y='Total Cost', + hue='Scheduler', + style='Scheduler', + dashes=False, + markers=True, + linewidth=2, + markersize=8) + +sns.move_legend(ax, "lower center", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True) + +plt.savefig("fig-random-100.pdf", bbox_inches='tight') + + +# In[ ]: + + +# Checking how many times other schedulers meet the performance of (MC)^2MKP +print('-- Checking how many times other schedulers meet the performance of (MC)2MKP') +mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True) +for scheduler in schedulers[1:]: + other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True) + greater = np.sum(other_cost > mc2mkp_cost) + equal = np.sum(other_cost == mc2mkp_cost) + less = np.sum(other_cost < mc2mkp_cost) + print(f'Number of times {scheduler} provides a Total Cost that is greater, equal, or smaller than (MC)^2MKP: ' + + f'{greater}, {equal}, {less}.') + + +# --- +# ## Results with increasing marginal costs + +# In[ ]: + + +# reads the result file +results = pd.read_csv('results_with_increasing_marginal_costs.csv', comment='#') +print('\n- Results with increasing marginal costs') +results.head(9) + + +# In[ ]: + + +print(f'-- Number of results: {len(results)} (expected: {expected_number})') + + +# In[ ]: + + +print('-- Generating figures fig-increasing-10.pdf and fig-increasing-100.pdf') + +# Sets figure parameters +plt.figure(figsize=(6,5)) +plt.rcParams['axes.titlesize'] = 13 +plt.rcParams['axes.labelsize'] = 13 +plt.rcParams['xtick.labelsize'] = 13 +plt.rcParams['ytick.labelsize'] = 13 +plt.rcParams['legend.fontsize'] = 12 +plt.xlabel('Number of tasks (T)', fontsize=13) +plt.ylabel('Total cost (a.u.)', fontsize=13) +plt.xticks(rotation=15) +plt.ylim(0, 160000) + +ax = sns.lineplot(data=results[results.Resources == 10], + x='Tasks', + y='Total Cost', + hue='Scheduler', + style='Scheduler', + dashes=False, + markers=True, + linewidth=2, + markersize=8) + +sns.move_legend(ax, "lower center", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True) + +plt.savefig("fig-increasing-10.pdf", bbox_inches='tight') + + +# In[ ]: + + +# Sets figure parameters +plt.figure(figsize=(6,5)) +plt.rcParams['axes.titlesize'] = 13 +plt.rcParams['axes.labelsize'] = 13 +plt.rcParams['xtick.labelsize'] = 13 +plt.rcParams['ytick.labelsize'] = 13 +plt.rcParams['legend.fontsize'] = 12 +plt.xlabel('Number of tasks (T)', fontsize=13) +plt.ylabel('Total cost (a.u.)', fontsize=13) +plt.xticks(rotation=15) +plt.ylim(0, 120000) + +ax = sns.lineplot(data=results[results.Resources == 100], + x='Tasks', + y='Total Cost', + hue='Scheduler', + style='Scheduler', + dashes=False, + markers=True, + linewidth=2, + markersize=8) + +sns.move_legend(ax, "lower center", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True) + +plt.savefig("fig-increasing-100.pdf", bbox_inches='tight') + + +# In[ ]: + + +# Checking how many times other schedulers meet the performance of (MC)^2MKP +print('-- Checking how many times other schedulers meet the performance of (MC)2MKP') +mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True) +for scheduler in schedulers[1:]: + other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True) + greater = np.sum(other_cost > mc2mkp_cost) + equal = np.sum(other_cost == mc2mkp_cost) + less = np.sum(other_cost < mc2mkp_cost) + print(f'Number of times {scheduler} provides a Total Cost that is greater, equal, or smaller than (MC)^2MKP: ' + + f'{greater}, {equal}, {less}.') + + +# --- +# ## Results with constant marginal costs + +# In[ ]: + + +# reads the result file +results = pd.read_csv('results_with_constant_marginal_costs.csv', comment='#') +print('\n- Results with constant marginal costs') +results.head(9) + + +# In[ ]: + + +print(f'-- Number of results: {len(results)} (expected: {expected_number})') + + +# In[ ]: + + +print('-- Generating figures fig-constant-10.pdf and fig-constant-100.pdf') + +# Sets figure parameters +plt.figure(figsize=(6,5)) +plt.rcParams['axes.titlesize'] = 13 +plt.rcParams['axes.labelsize'] = 13 +plt.rcParams['xtick.labelsize'] = 13 +plt.rcParams['ytick.labelsize'] = 13 +plt.rcParams['legend.fontsize'] = 12 +plt.xlabel('Number of tasks (T)', fontsize=13) +plt.ylabel('Total cost (a.u.)', fontsize=13) +plt.xticks(rotation=15) +plt.ylim(0, 30000) + +ax = sns.lineplot(data=results[results.Resources == 10], + x='Tasks', + y='Total Cost', + hue='Scheduler', + style='Scheduler', + dashes=False, + markers=True, + linewidth=2, + markersize=8) + +sns.move_legend(ax, "lower center", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True) + +plt.savefig("fig-constant-10.pdf", bbox_inches='tight') + + +# In[ ]: + + +# Sets figure parameters +plt.figure(figsize=(6,5)) +plt.rcParams['axes.titlesize'] = 13 +plt.rcParams['axes.labelsize'] = 13 +plt.rcParams['xtick.labelsize'] = 13 +plt.rcParams['ytick.labelsize'] = 13 +plt.rcParams['legend.fontsize'] = 12 +plt.xlabel('Number of tasks (T)', fontsize=13) +plt.ylabel('Total cost (a.u.)', fontsize=13) +plt.xticks(rotation=15) +plt.ylim(0, 30000) + +ax = sns.lineplot(data=results[results.Resources == 100], + x='Tasks', + y='Total Cost', + hue='Scheduler', + style='Scheduler', + dashes=False, + markers=True, + linewidth=2, + markersize=8) + +sns.move_legend(ax, "lower center", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True) + +plt.savefig("fig-constant-100.pdf", bbox_inches='tight') + + +# In[ ]: + + +# Checking how many times other schedulers meet the performance of (MC)^2MKP +print('-- Checking how many times other schedulers meet the performance of (MC)2MKP') +mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True) +for scheduler in schedulers[1:]: + other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True) + greater = np.sum(other_cost > mc2mkp_cost) + equal = np.sum(other_cost == mc2mkp_cost) + less = np.sum(other_cost < mc2mkp_cost) + print(f'Number of times {scheduler} provides a Total Cost that is greater, equal, or smaller than (MC)^2MKP: ' + + f'{greater}, {equal}, {less}.') + + +# --- +# ## Results with decreasing marginal costs + +# In[ ]: + + +# reads the result file +results = pd.read_csv('results_with_decreasing_marginal_costs.csv', comment='#') +print('\n- Results with decreasing marginal costs') +results.head(9) + + +# In[ ]: + + +print(f'-- Number of results: {len(results)} (expected: {expected_number})') + + +# In[ ]: + + +print('-- Generating figures fig-decreasing-10.pdf, fig-decreasing-100.pdf, and fig-decreasing-100-zoom.pdf') + +# Sets figure parameters +plt.figure(figsize=(6,5)) +plt.rcParams['axes.titlesize'] = 13 +plt.rcParams['axes.labelsize'] = 13 +plt.rcParams['xtick.labelsize'] = 13 +plt.rcParams['ytick.labelsize'] = 13 +plt.rcParams['legend.fontsize'] = 12 +plt.xlabel('Number of tasks (T)', fontsize=13) +plt.ylabel('Total cost (a.u.)', fontsize=13) +plt.xticks(rotation=15) +plt.ylim(0, 500) + +ax = sns.lineplot(data=results[results.Resources == 10], + x='Tasks', + y='Total Cost', + hue='Scheduler', + style='Scheduler', + dashes=False, + markers=True, + linewidth=2, + markersize=8) + +sns.move_legend(ax, "lower center", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True) + +plt.savefig("fig-decreasing-10.pdf", bbox_inches='tight') + + +# In[ ]: + + +# Sets figure parameters +plt.figure(figsize=(6,5)) +plt.rcParams['axes.titlesize'] = 13 +plt.rcParams['axes.labelsize'] = 13 +plt.rcParams['xtick.labelsize'] = 13 +plt.rcParams['ytick.labelsize'] = 13 +plt.rcParams['legend.fontsize'] = 12 +plt.xlabel('Number of tasks (T)', fontsize=13) +plt.ylabel('Total cost (a.u.)', fontsize=13) +plt.xticks(rotation=15) +plt.ylim(0, 3000) + +ax = sns.lineplot(data=results[results.Resources == 100], + x='Tasks', + y='Total Cost', + hue='Scheduler', + style='Scheduler', + dashes=False, + markers=True, + linewidth=2, + markersize=8) + +sns.move_legend(ax, "lower center", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True) + +plt.savefig("fig-decreasing-100.pdf", bbox_inches='tight') + + +# In[ ]: + + +# Sets figure parameters +plt.figure(figsize=(6,5)) +plt.rcParams['axes.titlesize'] = 13 +plt.rcParams['axes.labelsize'] = 13 +plt.rcParams['xtick.labelsize'] = 13 +plt.rcParams['ytick.labelsize'] = 13 +plt.rcParams['legend.fontsize'] = 12 +plt.xlabel('Number of tasks (T)', fontsize=13) +plt.ylabel('Total cost (a.u.)', fontsize=13) +plt.xticks(rotation=15) +plt.ylim(1465, 1472) + +ax = sns.lineplot(data=results[results.Resources == 100], + x='Tasks', + y='Total Cost', + hue='Scheduler', + style='Scheduler', + dashes=False, + markers=True, + linewidth=2, + markersize=8) + +sns.move_legend(ax, "lower center", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True) + +plt.savefig("fig-decreasing-100-zoom.pdf", bbox_inches='tight') + + +# In[ ]: + + +# Checking how many times other schedulers meet the performance of (MC)^2MKP +print('-- Checking how many times other schedulers meet the performance of (MC)2MKP') +mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True) +for scheduler in schedulers[1:]: + other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True) + greater = np.sum(other_cost > mc2mkp_cost) + equal = np.sum(other_cost == mc2mkp_cost) + less = np.sum(other_cost < mc2mkp_cost) + print(f'Number of times {scheduler} provides a Total Cost that is greater, equal, or smaller than (MC)^2MKP: ' + + f'{greater}, {equal}, {less}.') + + +# --- +# ## Results with constant marginal costs and no upper limits + +# In[ ]: + + +# reads the result file +results = pd.read_csv('results_with_constant_marginal_costs_no_upper_limit.csv', comment='#') +print('\n- Results with constant marginal costs and no upper limits') +results.head(9) + + +# In[ ]: + + +print(f'-- Number of results: {len(results)} (expected: {expected_number})') + + +# In[ ]: + + +print('-- Generating figures fig-constant-10-no-u.pdf and fig-constant-100-no-u.pdf') + +# Sets figure parameters +plt.figure(figsize=(6,5)) +plt.rcParams['axes.titlesize'] = 13 +plt.rcParams['axes.labelsize'] = 13 +plt.rcParams['xtick.labelsize'] = 13 +plt.rcParams['ytick.labelsize'] = 13 +plt.rcParams['legend.fontsize'] = 12 +plt.xlabel('Number of tasks (T)', fontsize=13) +plt.ylabel('Total cost (a.u.)', fontsize=13) +plt.xticks(rotation=15) +plt.ylim(0, 30000) + +ax = sns.lineplot(data=results[results.Resources == 10], + x='Tasks', + y='Total Cost', + hue='Scheduler', + style='Scheduler', + dashes=False, + markers=True, + linewidth=2, + markersize=8) + +sns.move_legend(ax, "lower center", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True) + +plt.savefig("fig-constant-10-no-u.pdf", bbox_inches='tight') + + +# In[ ]: + + +# Sets figure parameters +plt.figure(figsize=(6,5)) +plt.rcParams['axes.titlesize'] = 13 +plt.rcParams['axes.labelsize'] = 13 +plt.rcParams['xtick.labelsize'] = 13 +plt.rcParams['ytick.labelsize'] = 13 +plt.rcParams['legend.fontsize'] = 12 +plt.xlabel('Number of tasks (T)', fontsize=13) +plt.ylabel('Total cost (a.u.)', fontsize=13) +plt.xticks(rotation=15) +plt.ylim(0, 30000) + +ax = sns.lineplot(data=results[results.Resources == 100], + x='Tasks', + y='Total Cost', + hue='Scheduler', + style='Scheduler', + dashes=False, + markers=True, + linewidth=2, + markersize=8) + +sns.move_legend(ax, "lower center", bbox_to_anchor=(.5, 1), ncol=3, title=None, frameon=True) + +plt.savefig("fig-constant-100-no-u.pdf", bbox_inches='tight') + + +# In[ ]: + + +# Checking how many times other schedulers meet the performance of (MC)^2MKP +print('-- Checking how many times other schedulers meet the performance of (MC)2MKP') +mc2mkp_cost = results[results['Scheduler'] == '(MC)2MKP']['Total Cost'].reset_index(drop=True) +for scheduler in ['MarIn', 'MarCo', 'MarDecUn', 'FedAvg']: + other_cost = results[results['Scheduler'] == scheduler]['Total Cost'].reset_index(drop=True) + greater = np.sum(other_cost > mc2mkp_cost) + equal = np.sum(other_cost == mc2mkp_cost) + less = np.sum(other_cost < mc2mkp_cost) + print(f'Number of times {scheduler} provides a Total Cost that is greater, equal, or smaller than (MC)^2MKP: ' + + f'{greater}, {equal}, {less}.') + diff --git a/stored_results/results_of_timing_with_fixed_resources.csv b/original_results/results_of_timing_with_fixed_resources.csv similarity index 100% rename from stored_results/results_of_timing_with_fixed_resources.csv rename to original_results/results_of_timing_with_fixed_resources.csv diff --git a/stored_results/results_of_timing_with_fixed_tasks.csv b/original_results/results_of_timing_with_fixed_tasks.csv similarity index 100% rename from stored_results/results_of_timing_with_fixed_tasks.csv rename to original_results/results_of_timing_with_fixed_tasks.csv diff --git a/original_results/results_with_constant_marginal_costs.csv b/original_results/results_with_constant_marginal_costs.csv new file mode 100644 index 0000000..22386e4 --- /dev/null +++ b/original_results/results_with_constant_marginal_costs.csv @@ -0,0 +1,422 @@ +# Description of the experiment: +# +# - We generate the costs to up to 5.000 tasks for 10 and 100 resources. +# - All costs follow linear functions (i.e., constant marginal costs) +# with RNG seeds [100..199]. +# - We schedule from 1.000 to 5.000 tasks in increments of 100. +# - We run (MC)^2MKP, MarIn, MarCo, MarDec, and FedAvg. +# - All resources have a lower limit of 5. +# - The first half of the resources have no upper limit. +# - The second half has an upper limit of 2*(tasks/resources). +# - Every result is verified and logged to a CSV file. +Scheduler,Tasks,Resources,Total Cost +(MC)2MKP,1000,10,2486.498916533192 +MarIn,1000,10,2486.498916533192 +MarCo,1000,10,2486.498916533192 +MarDec,1000,10,2486.498916533192 +FedAvg,1000,10,5086.389214578499 +(MC)2MKP,1100,10,2718.1236780149343 +MarIn,1100,10,2718.1236780149343 +MarCo,1100,10,2718.1236780149343 +MarDec,1100,10,2718.1236780149343 +FedAvg,1100,10,5591.024297554557 +(MC)2MKP,1200,10,2949.7484394966764 +MarIn,1200,10,2949.7484394966764 +MarCo,1200,10,2949.7484394966764 +MarDec,1200,10,2949.7484394966764 +FedAvg,1200,10,6095.659380530616 +(MC)2MKP,1300,10,3181.373200978418 +MarIn,1300,10,3181.373200978418 +MarCo,1300,10,3181.373200978418 +MarDec,1300,10,3181.373200978418 +FedAvg,1300,10,6600.294463506673 +(MC)2MKP,1400,10,3412.99796246016 +MarIn,1400,10,3412.99796246016 +MarCo,1400,10,3412.99796246016 +MarDec,1400,10,3412.99796246016 +FedAvg,1400,10,7104.929546482732 +(MC)2MKP,1500,10,3644.6227239419018 +MarIn,1500,10,3644.6227239419018 +MarCo,1500,10,3644.6227239419018 +MarDec,1500,10,3644.6227239419018 +FedAvg,1500,10,7609.56462945879 +(MC)2MKP,1600,10,3876.2474854236434 +MarIn,1600,10,3876.2474854236434 +MarCo,1600,10,3876.2474854236434 +MarDec,1600,10,3876.2474854236434 +FedAvg,1600,10,8114.199712434848 +(MC)2MKP,1700,10,4107.8722469053855 +MarIn,1700,10,4107.8722469053855 +MarCo,1700,10,4107.8722469053855 +MarDec,1700,10,4107.8722469053855 +FedAvg,1700,10,8618.834795410907 +(MC)2MKP,1800,10,4339.497008387128 +MarIn,1800,10,4339.497008387128 +MarCo,1800,10,4339.497008387128 +MarDec,1800,10,4339.497008387128 +FedAvg,1800,10,9123.469878386964 +(MC)2MKP,1900,10,4571.12176986887 +MarIn,1900,10,4571.12176986887 +MarCo,1900,10,4571.12176986887 +MarDec,1900,10,4571.12176986887 +FedAvg,1900,10,9628.104961363024 +(MC)2MKP,2000,10,4802.746531350611 +MarIn,2000,10,4802.746531350611 +MarCo,2000,10,4802.746531350611 +MarDec,2000,10,4802.746531350611 +FedAvg,2000,10,10132.74004433908 +(MC)2MKP,2100,10,5034.371292832352 +MarIn,2100,10,5034.371292832352 +MarCo,2100,10,5034.371292832352 +MarDec,2100,10,5034.371292832352 +FedAvg,2100,10,10637.37512731514 +(MC)2MKP,2200,10,5265.996054314094 +MarIn,2200,10,5265.996054314094 +MarCo,2200,10,5265.996054314094 +MarDec,2200,10,5265.996054314094 +FedAvg,2200,10,11142.010210291197 +(MC)2MKP,2300,10,5497.620815795835 +MarIn,2300,10,5497.620815795835 +MarCo,2300,10,5497.620815795835 +MarDec,2300,10,5497.620815795835 +FedAvg,2300,10,11646.645293267255 +(MC)2MKP,2400,10,5729.245577277577 +MarIn,2400,10,5729.245577277577 +MarCo,2400,10,5729.245577277577 +MarDec,2400,10,5729.245577277577 +FedAvg,2400,10,12151.280376243312 +(MC)2MKP,2500,10,5960.870338759319 +MarIn,2500,10,5960.870338759319 +MarCo,2500,10,5960.870338759319 +MarDec,2500,10,5960.870338759319 +FedAvg,2500,10,12655.915459219372 +(MC)2MKP,2600,10,6192.495100241061 +MarIn,2600,10,6192.495100241061 +MarCo,2600,10,6192.495100241061 +MarDec,2600,10,6192.495100241061 +FedAvg,2600,10,13160.55054219543 +(MC)2MKP,2700,10,6424.119861722802 +MarIn,2700,10,6424.119861722802 +MarCo,2700,10,6424.119861722802 +MarDec,2700,10,6424.119861722802 +FedAvg,2700,10,13665.185625171489 +(MC)2MKP,2800,10,6655.744623204544 +MarIn,2800,10,6655.744623204544 +MarCo,2800,10,6655.744623204544 +MarDec,2800,10,6655.744623204544 +FedAvg,2800,10,14169.820708147547 +(MC)2MKP,2900,10,6887.369384686286 +MarIn,2900,10,6887.369384686286 +MarCo,2900,10,6887.369384686286 +MarDec,2900,10,6887.369384686286 +FedAvg,2900,10,14674.455791123604 +(MC)2MKP,3000,10,7118.994146168028 +MarIn,3000,10,7118.994146168028 +MarCo,3000,10,7118.994146168028 +MarDec,3000,10,7118.994146168028 +FedAvg,3000,10,15179.090874099662 +(MC)2MKP,3100,10,7350.61890764977 +MarIn,3100,10,7350.61890764977 +MarCo,3100,10,7350.61890764977 +MarDec,3100,10,7350.61890764977 +FedAvg,3100,10,15683.72595707572 +(MC)2MKP,3200,10,7582.2436691315115 +MarIn,3200,10,7582.2436691315115 +MarCo,3200,10,7582.2436691315115 +MarDec,3200,10,7582.2436691315115 +FedAvg,3200,10,16188.361040051777 +(MC)2MKP,3300,10,7813.868430613254 +MarIn,3300,10,7813.868430613254 +MarCo,3300,10,7813.868430613254 +MarDec,3300,10,7813.868430613254 +FedAvg,3300,10,16692.996123027835 +(MC)2MKP,3400,10,8045.493192094996 +MarIn,3400,10,8045.493192094996 +MarCo,3400,10,8045.493192094996 +MarDec,3400,10,8045.493192094996 +FedAvg,3400,10,17197.631206003894 +(MC)2MKP,3500,10,8277.117953576737 +MarIn,3500,10,8277.117953576737 +MarCo,3500,10,8277.117953576737 +MarDec,3500,10,8277.117953576737 +FedAvg,3500,10,17702.26628897995 +(MC)2MKP,3600,10,8508.742715058477 +MarIn,3600,10,8508.742715058477 +MarCo,3600,10,8508.742715058477 +MarDec,3600,10,8508.742715058477 +FedAvg,3600,10,18206.90137195601 +(MC)2MKP,3700,10,8740.36747654022 +MarIn,3700,10,8740.36747654022 +MarCo,3700,10,8740.36747654022 +MarDec,3700,10,8740.36747654022 +FedAvg,3700,10,18711.53645493207 +(MC)2MKP,3800,10,8971.992238021961 +MarIn,3800,10,8971.992238021961 +MarCo,3800,10,8971.992238021961 +MarDec,3800,10,8971.992238021961 +FedAvg,3800,10,19216.17153790813 +(MC)2MKP,3900,10,9203.616999503703 +MarIn,3900,10,9203.616999503703 +MarCo,3900,10,9203.616999503703 +MarDec,3900,10,9203.616999503703 +FedAvg,3900,10,19720.806620884185 +(MC)2MKP,4000,10,9435.241760985446 +MarIn,4000,10,9435.241760985446 +MarCo,4000,10,9435.241760985446 +MarDec,4000,10,9435.241760985446 +FedAvg,4000,10,20225.44170386024 +(MC)2MKP,4100,10,9666.866522467188 +MarIn,4100,10,9666.866522467188 +MarCo,4100,10,9666.866522467188 +MarDec,4100,10,9666.866522467188 +FedAvg,4100,10,20730.076786836304 +(MC)2MKP,4200,10,9898.49128394893 +MarIn,4200,10,9898.49128394893 +MarCo,4200,10,9898.49128394893 +MarDec,4200,10,9898.49128394893 +FedAvg,4200,10,21234.71186981236 +(MC)2MKP,4300,10,10130.116045430672 +MarIn,4300,10,10130.116045430672 +MarCo,4300,10,10130.116045430672 +MarDec,4300,10,10130.116045430672 +FedAvg,4300,10,21739.34695278842 +(MC)2MKP,4400,10,10361.740806912412 +MarIn,4400,10,10361.740806912412 +MarCo,4400,10,10361.740806912412 +MarDec,4400,10,10361.740806912412 +FedAvg,4400,10,22243.98203576448 +(MC)2MKP,4500,10,10593.365568394154 +MarIn,4500,10,10593.365568394154 +MarCo,4500,10,10593.365568394154 +MarDec,4500,10,10593.365568394154 +FedAvg,4500,10,22748.617118740534 +(MC)2MKP,4600,10,10824.990329875896 +MarIn,4600,10,10824.990329875896 +MarCo,4600,10,10824.990329875896 +MarDec,4600,10,10824.990329875896 +FedAvg,4600,10,23253.252201716594 +(MC)2MKP,4700,10,11056.615091357637 +MarIn,4700,10,11056.615091357637 +MarCo,4700,10,11056.615091357637 +MarDec,4700,10,11056.615091357637 +FedAvg,4700,10,23757.88728469265 +(MC)2MKP,4800,10,11288.23985283938 +MarIn,4800,10,11288.23985283938 +MarCo,4800,10,11288.23985283938 +MarDec,4800,10,11288.23985283938 +FedAvg,4800,10,24262.522367668706 +(MC)2MKP,4900,10,11519.86461432112 +MarIn,4900,10,11519.86461432112 +MarCo,4900,10,11519.86461432112 +MarDec,4900,10,11519.86461432112 +FedAvg,4900,10,24767.157450644765 +(MC)2MKP,5000,10,11751.489375802865 +MarIn,5000,10,11751.489375802865 +MarCo,5000,10,11751.489375802865 +MarDec,5000,10,11751.489375802865 +FedAvg,5000,10,25271.792533620825 +(MC)2MKP,1000,100,3687.280507967055 +MarIn,1000,100,3687.280507967055 +MarCo,1000,100,3687.280507967055 +MarDec,1000,100,3687.280507967055 +FedAvg,1000,100,5776.56498918897 +(MC)2MKP,1100,100,3789.107327594464 +MarIn,1100,100,3789.107327594464 +MarCo,1100,100,3789.107327594464 +MarDec,1100,100,3789.107327594464 +FedAvg,1100,100,6296.2487050607615 +(MC)2MKP,1200,100,3890.934147221873 +MarIn,1200,100,3890.934147221873 +MarCo,1200,100,3890.934147221873 +MarDec,1200,100,3890.934147221873 +FedAvg,1200,100,6815.932420932553 +(MC)2MKP,1300,100,3992.760966849282 +MarIn,1300,100,3992.760966849282 +MarCo,1300,100,3992.760966849282 +MarDec,1300,100,3992.760966849282 +FedAvg,1300,100,7335.616136804344 +(MC)2MKP,1400,100,4094.587786476691 +MarIn,1400,100,4094.587786476691 +MarCo,1400,100,4094.587786476691 +MarDec,1400,100,4094.587786476691 +FedAvg,1400,100,7855.299852676136 +(MC)2MKP,1500,100,4196.4146061041 +MarIn,1500,100,4196.4146061041 +MarCo,1500,100,4196.4146061041 +MarDec,1500,100,4196.4146061041 +FedAvg,1500,100,8374.983568547927 +(MC)2MKP,1600,100,4298.241425731509 +MarIn,1600,100,4298.241425731509 +MarCo,1600,100,4298.241425731509 +MarDec,1600,100,4298.241425731509 +FedAvg,1600,100,8894.667284419718 +(MC)2MKP,1700,100,4400.068245358917 +MarIn,1700,100,4400.068245358917 +MarCo,1700,100,4400.068245358917 +MarDec,1700,100,4400.068245358917 +FedAvg,1700,100,9414.35100029151 +(MC)2MKP,1800,100,4501.895064986326 +MarIn,1800,100,4501.895064986326 +MarCo,1800,100,4501.895064986326 +MarDec,1800,100,4501.895064986326 +FedAvg,1800,100,9934.034716163302 +(MC)2MKP,1900,100,4603.721884613735 +MarIn,1900,100,4603.721884613735 +MarCo,1900,100,4603.721884613735 +MarDec,1900,100,4603.721884613735 +FedAvg,1900,100,10453.718432035093 +(MC)2MKP,2000,100,4705.548704241144 +MarIn,2000,100,4705.548704241144 +MarCo,2000,100,4705.548704241144 +MarDec,2000,100,4705.548704241144 +FedAvg,2000,100,10973.402147906889 +(MC)2MKP,2100,100,4807.375523868553 +MarIn,2100,100,4807.375523868553 +MarCo,2100,100,4807.375523868553 +MarDec,2100,100,4807.375523868553 +FedAvg,2100,100,11493.085863778677 +(MC)2MKP,2200,100,4909.202343495961 +MarIn,2200,100,4909.202343495961 +MarCo,2200,100,4909.202343495961 +MarDec,2200,100,4909.202343495961 +FedAvg,2200,100,12012.769579650469 +(MC)2MKP,2300,100,5011.02916312337 +MarIn,2300,100,5011.02916312337 +MarCo,2300,100,5011.02916312337 +MarDec,2300,100,5011.02916312337 +FedAvg,2300,100,12532.453295522262 +(MC)2MKP,2400,100,5112.855982750778 +MarIn,2400,100,5112.855982750778 +MarCo,2400,100,5112.855982750778 +MarDec,2400,100,5112.855982750778 +FedAvg,2400,100,13052.137011394054 +(MC)2MKP,2500,100,5214.682802378187 +MarIn,2500,100,5214.682802378187 +MarCo,2500,100,5214.682802378187 +MarDec,2500,100,5214.682802378187 +FedAvg,2500,100,13571.820727265846 +(MC)2MKP,2600,100,5316.509622005596 +MarIn,2600,100,5316.509622005596 +MarCo,2600,100,5316.509622005596 +MarDec,2600,100,5316.509622005596 +FedAvg,2600,100,14091.504443137635 +(MC)2MKP,2700,100,5418.336441633004 +MarIn,2700,100,5418.336441633004 +MarCo,2700,100,5418.336441633004 +MarDec,2700,100,5418.336441633004 +FedAvg,2700,100,14611.188159009425 +(MC)2MKP,2800,100,5520.1632612604135 +MarIn,2800,100,5520.1632612604135 +MarCo,2800,100,5520.1632612604135 +MarDec,2800,100,5520.1632612604135 +FedAvg,2800,100,15130.871874881219 +(MC)2MKP,2900,100,5621.990080887823 +MarIn,2900,100,5621.990080887823 +MarCo,2900,100,5621.990080887823 +MarDec,2900,100,5621.990080887823 +FedAvg,2900,100,15650.555590753007 +(MC)2MKP,3000,100,5723.816900515231 +MarIn,3000,100,5723.816900515231 +MarCo,3000,100,5723.816900515231 +MarDec,3000,100,5723.816900515231 +FedAvg,3000,100,16170.239306624804 +(MC)2MKP,3100,100,5825.64372014264 +MarIn,3100,100,5825.64372014264 +MarCo,3100,100,5825.64372014264 +MarDec,3100,100,5825.64372014264 +FedAvg,3100,100,16689.923022496594 +(MC)2MKP,3200,100,5927.470539770049 +MarIn,3200,100,5927.470539770049 +MarCo,3200,100,5927.470539770049 +MarDec,3200,100,5927.470539770049 +FedAvg,3200,100,17209.606738368384 +(MC)2MKP,3300,100,6029.2973593974575 +MarIn,3300,100,6029.2973593974575 +MarCo,3300,100,6029.2973593974575 +MarDec,3300,100,6029.2973593974575 +FedAvg,3300,100,17729.290454240177 +(MC)2MKP,3400,100,6131.124179024866 +MarIn,3400,100,6131.124179024866 +MarCo,3400,100,6131.124179024866 +MarDec,3400,100,6131.124179024866 +FedAvg,3400,100,18248.974170111967 +(MC)2MKP,3500,100,6232.950998652275 +MarIn,3500,100,6232.950998652275 +MarCo,3500,100,6232.950998652275 +MarDec,3500,100,6232.950998652275 +FedAvg,3500,100,18768.657885983765 +(MC)2MKP,3600,100,6334.777818279684 +MarIn,3600,100,6334.777818279684 +MarCo,3600,100,6334.777818279684 +MarDec,3600,100,6334.777818279684 +FedAvg,3600,100,19288.341601855547 +(MC)2MKP,3700,100,6436.604637907092 +MarIn,3700,100,6436.604637907092 +MarCo,3700,100,6436.604637907092 +MarDec,3700,100,6436.604637907092 +FedAvg,3700,100,19808.025317727344 +(MC)2MKP,3800,100,6538.4314575345015 +MarIn,3800,100,6538.4314575345015 +MarCo,3800,100,6538.4314575345015 +MarDec,3800,100,6538.4314575345015 +FedAvg,3800,100,20327.709033599134 +(MC)2MKP,3900,100,6640.258277161911 +MarIn,3900,100,6640.258277161911 +MarCo,3900,100,6640.258277161911 +MarDec,3900,100,6640.258277161911 +FedAvg,3900,100,20847.392749470924 +(MC)2MKP,4000,100,6742.085096789319 +MarIn,4000,100,6742.085096789319 +MarCo,4000,100,6742.085096789319 +MarDec,4000,100,6742.085096789319 +FedAvg,4000,100,21367.076465342714 +(MC)2MKP,4100,100,6843.911916416728 +MarIn,4100,100,6843.911916416728 +MarCo,4100,100,6843.911916416728 +MarDec,4100,100,6843.911916416728 +FedAvg,4100,100,21886.760181214508 +(MC)2MKP,4200,100,6945.738736044137 +MarIn,4200,100,6945.738736044137 +MarCo,4200,100,6945.738736044137 +MarDec,4200,100,6945.738736044137 +FedAvg,4200,100,22406.443897086294 +(MC)2MKP,4300,100,7047.5655556715465 +MarIn,4300,100,7047.5655556715465 +MarCo,4300,100,7047.5655556715465 +MarDec,4300,100,7047.5655556715465 +FedAvg,4300,100,22926.127612958087 +(MC)2MKP,4400,100,7149.392375298956 +MarIn,4400,100,7149.392375298956 +MarCo,4400,100,7149.392375298956 +MarDec,4400,100,7149.392375298956 +FedAvg,4400,100,23445.811328829877 +(MC)2MKP,4500,100,7251.219194926364 +MarIn,4500,100,7251.219194926364 +MarCo,4500,100,7251.219194926364 +MarDec,4500,100,7251.219194926364 +FedAvg,4500,100,23965.49504470168 +(MC)2MKP,4600,100,7353.046014553774 +MarIn,4600,100,7353.046014553774 +MarCo,4600,100,7353.046014553774 +MarDec,4600,100,7353.046014553774 +FedAvg,4600,100,24485.17876057347 +(MC)2MKP,4700,100,7454.872834181182 +MarIn,4700,100,7454.872834181182 +MarCo,4700,100,7454.872834181182 +MarDec,4700,100,7454.872834181182 +FedAvg,4700,100,25004.86247644526 +(MC)2MKP,4800,100,7556.6996538085905 +MarIn,4800,100,7556.6996538085905 +MarCo,4800,100,7556.6996538085905 +MarDec,4800,100,7556.6996538085905 +FedAvg,4800,100,25524.54619231705 +(MC)2MKP,4900,100,7658.526473436 +MarIn,4900,100,7658.526473436 +MarCo,4900,100,7658.526473436 +MarDec,4900,100,7658.526473436 +FedAvg,4900,100,26044.22990818884 +(MC)2MKP,5000,100,7760.353293063408 +MarIn,5000,100,7760.353293063408 +MarCo,5000,100,7760.353293063408 +MarDec,5000,100,7760.353293063408 +FedAvg,5000,100,26563.91362406064 diff --git a/original_results/results_with_constant_marginal_costs_no_upper_limit.csv b/original_results/results_with_constant_marginal_costs_no_upper_limit.csv new file mode 100644 index 0000000..218a937 --- /dev/null +++ b/original_results/results_with_constant_marginal_costs_no_upper_limit.csv @@ -0,0 +1,420 @@ +# Description of the experiment: +# +# - We generate the costs to up to 5.000 tasks for 10 and 100 resources. +# - All costs follow linear functions (i.e., constant marginal costs) +# with RNG seeds [500..599]. +# - We schedule from 1.000 to 5.000 tasks in increments of 100. +# - We run (MC)^2MKP, MarIn, MarCo, MarDecUn, and FedAvg. +# - All resources have a lower limit of 5 and no upper limit. +# - Every result is verified and logged to a CSV file. +Scheduler,Tasks,Resources,Total Cost +(MC)2MKP,1000,10,1299.2439819770243 +MarIn,1000,10,1299.2439819770243 +MarCo,1000,10,1299.2439819770243 +MarDecUn,1000,10,1299.2439819770243 +FedAvg,1000,10,4928.083676777056 +(MC)2MKP,1100,10,1404.9136237594096 +MarIn,1100,10,1404.9136237594096 +MarCo,1100,10,1404.9136237594096 +MarDecUn,1100,10,1404.9136237594096 +FedAvg,1100,10,5415.736444327867 +(MC)2MKP,1200,10,1510.583265541795 +MarIn,1200,10,1510.583265541795 +MarCo,1200,10,1510.583265541795 +MarDecUn,1200,10,1510.583265541795 +FedAvg,1200,10,5903.389211878677 +(MC)2MKP,1300,10,1616.2529073241803 +MarIn,1300,10,1616.2529073241803 +MarCo,1300,10,1616.2529073241803 +MarDecUn,1300,10,1616.2529073241803 +FedAvg,1300,10,6391.041979429487 +(MC)2MKP,1400,10,1721.9225491065658 +MarIn,1400,10,1721.9225491065658 +MarCo,1400,10,1721.9225491065658 +MarDecUn,1400,10,1721.9225491065658 +FedAvg,1400,10,6878.694746980296 +(MC)2MKP,1500,10,1827.5921908889513 +MarIn,1500,10,1827.5921908889513 +MarCo,1500,10,1827.5921908889513 +MarDecUn,1500,10,1827.5921908889513 +FedAvg,1500,10,7366.347514531106 +(MC)2MKP,1600,10,1933.2618326713366 +MarIn,1600,10,1933.2618326713366 +MarCo,1600,10,1933.2618326713366 +MarDecUn,1600,10,1933.2618326713366 +FedAvg,1600,10,7854.000282081915 +(MC)2MKP,1700,10,2038.931474453722 +MarIn,1700,10,2038.931474453722 +MarCo,1700,10,2038.931474453722 +MarDecUn,1700,10,2038.931474453722 +FedAvg,1700,10,8341.653049632727 +(MC)2MKP,1800,10,2144.601116236107 +MarIn,1800,10,2144.601116236107 +MarCo,1800,10,2144.601116236107 +MarDecUn,1800,10,2144.601116236107 +FedAvg,1800,10,8829.305817183536 +(MC)2MKP,1900,10,2250.2707580184924 +MarIn,1900,10,2250.2707580184924 +MarCo,1900,10,2250.2707580184924 +MarDecUn,1900,10,2250.2707580184924 +FedAvg,1900,10,9316.958584734346 +(MC)2MKP,2000,10,2355.940399800878 +MarIn,2000,10,2355.940399800878 +MarCo,2000,10,2355.940399800878 +MarDecUn,2000,10,2355.940399800878 +FedAvg,2000,10,9804.611352285156 +(MC)2MKP,2100,10,2461.6100415832634 +MarIn,2100,10,2461.6100415832634 +MarCo,2100,10,2461.6100415832634 +MarDecUn,2100,10,2461.6100415832634 +FedAvg,2100,10,10292.264119835965 +(MC)2MKP,2200,10,2567.279683365649 +MarIn,2200,10,2567.279683365649 +MarCo,2200,10,2567.279683365649 +MarDecUn,2200,10,2567.279683365649 +FedAvg,2200,10,10779.916887386775 +(MC)2MKP,2300,10,2672.9493251480344 +MarIn,2300,10,2672.9493251480344 +MarCo,2300,10,2672.9493251480344 +MarDecUn,2300,10,2672.9493251480344 +FedAvg,2300,10,11267.569654937584 +(MC)2MKP,2400,10,2778.6189669304194 +MarIn,2400,10,2778.6189669304194 +MarCo,2400,10,2778.6189669304194 +MarDecUn,2400,10,2778.6189669304194 +FedAvg,2400,10,11755.222422488394 +(MC)2MKP,2500,10,2884.288608712805 +MarIn,2500,10,2884.288608712805 +MarCo,2500,10,2884.288608712805 +MarDecUn,2500,10,2884.288608712805 +FedAvg,2500,10,12242.875190039204 +(MC)2MKP,2600,10,2989.9582504951904 +MarIn,2600,10,2989.9582504951904 +MarCo,2600,10,2989.9582504951904 +MarDecUn,2600,10,2989.9582504951904 +FedAvg,2600,10,12730.527957590015 +(MC)2MKP,2700,10,3095.627892277576 +MarIn,2700,10,3095.627892277576 +MarCo,2700,10,3095.627892277576 +MarDecUn,2700,10,3095.627892277576 +FedAvg,2700,10,13218.180725140823 +(MC)2MKP,2800,10,3201.2975340599614 +MarIn,2800,10,3201.2975340599614 +MarCo,2800,10,3201.2975340599614 +MarDecUn,2800,10,3201.2975340599614 +FedAvg,2800,10,13705.833492691636 +(MC)2MKP,2900,10,3306.9671758423465 +MarIn,2900,10,3306.9671758423465 +MarCo,2900,10,3306.9671758423465 +MarDecUn,2900,10,3306.9671758423465 +FedAvg,2900,10,14193.486260242444 +(MC)2MKP,3000,10,3412.636817624732 +MarIn,3000,10,3412.636817624732 +MarCo,3000,10,3412.636817624732 +MarDecUn,3000,10,3412.636817624732 +FedAvg,3000,10,14681.139027793253 +(MC)2MKP,3100,10,3518.3064594071175 +MarIn,3100,10,3518.3064594071175 +MarCo,3100,10,3518.3064594071175 +MarDecUn,3100,10,3518.3064594071175 +FedAvg,3100,10,15168.791795344063 +(MC)2MKP,3200,10,3623.976101189503 +MarIn,3200,10,3623.976101189503 +MarCo,3200,10,3623.976101189503 +MarDecUn,3200,10,3623.976101189503 +FedAvg,3200,10,15656.444562894874 +(MC)2MKP,3300,10,3729.6457429718885 +MarIn,3300,10,3729.6457429718885 +MarCo,3300,10,3729.6457429718885 +MarDecUn,3300,10,3729.6457429718885 +FedAvg,3300,10,16144.097330445684 +(MC)2MKP,3400,10,3835.3153847542735 +MarIn,3400,10,3835.3153847542735 +MarCo,3400,10,3835.3153847542735 +MarDecUn,3400,10,3835.3153847542735 +FedAvg,3400,10,16631.750097996493 +(MC)2MKP,3500,10,3940.985026536659 +MarIn,3500,10,3940.985026536659 +MarCo,3500,10,3940.985026536659 +MarDecUn,3500,10,3940.985026536659 +FedAvg,3500,10,17119.402865547305 +(MC)2MKP,3600,10,4046.6546683190445 +MarIn,3600,10,4046.6546683190445 +MarCo,3600,10,4046.6546683190445 +MarDecUn,3600,10,4046.6546683190445 +FedAvg,3600,10,17607.055633098113 +(MC)2MKP,3700,10,4152.32431010143 +MarIn,3700,10,4152.32431010143 +MarCo,3700,10,4152.32431010143 +MarDecUn,3700,10,4152.32431010143 +FedAvg,3700,10,18094.708400648924 +(MC)2MKP,3800,10,4257.9939518838155 +MarIn,3800,10,4257.9939518838155 +MarCo,3800,10,4257.9939518838155 +MarDecUn,3800,10,4257.9939518838155 +FedAvg,3800,10,18582.361168199728 +(MC)2MKP,3900,10,4363.6635936662005 +MarIn,3900,10,4363.6635936662005 +MarCo,3900,10,4363.6635936662005 +MarDecUn,3900,10,4363.6635936662005 +FedAvg,3900,10,19070.013935750543 +(MC)2MKP,4000,10,4469.3332354485865 +MarIn,4000,10,4469.3332354485865 +MarCo,4000,10,4469.3332354485865 +MarDecUn,4000,10,4469.3332354485865 +FedAvg,4000,10,19557.666703301355 +(MC)2MKP,4100,10,4575.0028772309715 +MarIn,4100,10,4575.0028772309715 +MarCo,4100,10,4575.0028772309715 +MarDecUn,4100,10,4575.0028772309715 +FedAvg,4100,10,20045.319470852162 +(MC)2MKP,4200,10,4680.6725190133575 +MarIn,4200,10,4680.6725190133575 +MarCo,4200,10,4680.6725190133575 +MarDecUn,4200,10,4680.6725190133575 +FedAvg,4200,10,20532.972238402974 +(MC)2MKP,4300,10,4786.3421607957425 +MarIn,4300,10,4786.3421607957425 +MarCo,4300,10,4786.3421607957425 +MarDecUn,4300,10,4786.3421607957425 +FedAvg,4300,10,21020.625005953778 +(MC)2MKP,4400,10,4892.0118025781285 +MarIn,4400,10,4892.0118025781285 +MarCo,4400,10,4892.0118025781285 +MarDecUn,4400,10,4892.0118025781285 +FedAvg,4400,10,21508.277773504593 +(MC)2MKP,4500,10,4997.6814443605135 +MarIn,4500,10,4997.6814443605135 +MarCo,4500,10,4997.6814443605135 +MarDecUn,4500,10,4997.6814443605135 +FedAvg,4500,10,21995.930541055404 +(MC)2MKP,4600,10,5103.351086142899 +MarIn,4600,10,5103.351086142899 +MarCo,4600,10,5103.351086142899 +MarDecUn,4600,10,5103.351086142899 +FedAvg,4600,10,22483.583308606212 +(MC)2MKP,4700,10,5209.0207279252845 +MarIn,4700,10,5209.0207279252845 +MarCo,4700,10,5209.0207279252845 +MarDecUn,4700,10,5209.0207279252845 +FedAvg,4700,10,22971.23607615702 +(MC)2MKP,4800,10,5314.69036970767 +MarIn,4800,10,5314.69036970767 +MarCo,4800,10,5314.69036970767 +MarDecUn,4800,10,5314.69036970767 +FedAvg,4800,10,23458.888843707835 +(MC)2MKP,4900,10,5420.3600114900555 +MarIn,4900,10,5420.3600114900555 +MarCo,4900,10,5420.3600114900555 +MarDecUn,4900,10,5420.3600114900555 +FedAvg,4900,10,23946.541611258643 +(MC)2MKP,5000,10,5526.029653272441 +MarIn,5000,10,5526.029653272441 +MarCo,5000,10,5526.029653272441 +MarDecUn,5000,10,5526.029653272441 +FedAvg,5000,10,24434.194378809454 +(MC)2MKP,1000,100,3720.1557862428017 +MarIn,1000,100,3720.1557862428017 +MarCo,1000,100,3720.1557862428017 +MarDecUn,1000,100,3720.1557862428017 +FedAvg,1000,100,5863.401801610061 +(MC)2MKP,1100,100,3825.825428025187 +MarIn,1100,100,3825.825428025187 +MarCo,1100,100,3825.825428025187 +MarDecUn,1100,100,3825.825428025187 +FedAvg,1100,100,6397.7206464658975 +(MC)2MKP,1200,100,3931.4950698075727 +MarIn,1200,100,3931.4950698075727 +MarCo,1200,100,3931.4950698075727 +MarDecUn,1200,100,3931.4950698075727 +FedAvg,1200,100,6932.039491321735 +(MC)2MKP,1300,100,4037.164711589958 +MarIn,1300,100,4037.164711589958 +MarCo,1300,100,4037.164711589958 +MarDecUn,1300,100,4037.164711589958 +FedAvg,1300,100,7466.358336177572 +(MC)2MKP,1400,100,4142.834353372343 +MarIn,1400,100,4142.834353372343 +MarCo,1400,100,4142.834353372343 +MarDecUn,1400,100,4142.834353372343 +FedAvg,1400,100,8000.67718103341 +(MC)2MKP,1500,100,4248.503995154729 +MarIn,1500,100,4248.503995154729 +MarCo,1500,100,4248.503995154729 +MarDecUn,1500,100,4248.503995154729 +FedAvg,1500,100,8534.996025889246 +(MC)2MKP,1600,100,4354.173636937114 +MarIn,1600,100,4354.173636937114 +MarCo,1600,100,4354.173636937114 +MarDecUn,1600,100,4354.173636937114 +FedAvg,1600,100,9069.314870745085 +(MC)2MKP,1700,100,4459.8432787195 +MarIn,1700,100,4459.8432787195 +MarCo,1700,100,4459.8432787195 +MarDecUn,1700,100,4459.8432787195 +FedAvg,1700,100,9603.633715600921 +(MC)2MKP,1800,100,4565.512920501885 +MarIn,1800,100,4565.512920501885 +MarCo,1800,100,4565.512920501885 +MarDecUn,1800,100,4565.512920501885 +FedAvg,1800,100,10137.952560456755 +(MC)2MKP,1900,100,4671.18256228427 +MarIn,1900,100,4671.18256228427 +MarCo,1900,100,4671.18256228427 +MarDecUn,1900,100,4671.18256228427 +FedAvg,1900,100,10672.271405312595 +(MC)2MKP,2000,100,4776.852204066656 +MarIn,2000,100,4776.852204066656 +MarCo,2000,100,4776.852204066656 +MarDecUn,2000,100,4776.852204066656 +FedAvg,2000,100,11206.59025016843 +(MC)2MKP,2100,100,4882.521845849041 +MarIn,2100,100,4882.521845849041 +MarCo,2100,100,4882.521845849041 +MarDecUn,2100,100,4882.521845849041 +FedAvg,2100,100,11740.909095024266 +(MC)2MKP,2200,100,4988.191487631426 +MarIn,2200,100,4988.191487631426 +MarCo,2200,100,4988.191487631426 +MarDecUn,2200,100,4988.191487631426 +FedAvg,2200,100,12275.227939880106 +(MC)2MKP,2300,100,5093.861129413812 +MarIn,2300,100,5093.861129413812 +MarCo,2300,100,5093.861129413812 +MarDecUn,2300,100,5093.861129413812 +FedAvg,2300,100,12809.546784735941 +(MC)2MKP,2400,100,5199.530771196198 +MarIn,2400,100,5199.530771196198 +MarCo,2400,100,5199.530771196198 +MarDecUn,2400,100,5199.530771196198 +FedAvg,2400,100,13343.865629591779 +(MC)2MKP,2500,100,5305.200412978584 +MarIn,2500,100,5305.200412978584 +MarCo,2500,100,5305.200412978584 +MarDecUn,2500,100,5305.200412978584 +FedAvg,2500,100,13878.184474447618 +(MC)2MKP,2600,100,5410.870054760968 +MarIn,2600,100,5410.870054760968 +MarCo,2600,100,5410.870054760968 +MarDecUn,2600,100,5410.870054760968 +FedAvg,2600,100,14412.503319303454 +(MC)2MKP,2700,100,5516.539696543354 +MarIn,2700,100,5516.539696543354 +MarCo,2700,100,5516.539696543354 +MarDecUn,2700,100,5516.539696543354 +FedAvg,2700,100,14946.822164159292 +(MC)2MKP,2800,100,5622.20933832574 +MarIn,2800,100,5622.20933832574 +MarCo,2800,100,5622.20933832574 +MarDecUn,2800,100,5622.20933832574 +FedAvg,2800,100,15481.141009015126 +(MC)2MKP,2900,100,5727.878980108126 +MarIn,2900,100,5727.878980108126 +MarCo,2900,100,5727.878980108126 +MarDecUn,2900,100,5727.878980108126 +FedAvg,2900,100,16015.459853870963 +(MC)2MKP,3000,100,5833.54862189051 +MarIn,3000,100,5833.54862189051 +MarCo,3000,100,5833.54862189051 +MarDecUn,3000,100,5833.54862189051 +FedAvg,3000,100,16549.7786987268 +(MC)2MKP,3100,100,5939.218263672896 +MarIn,3100,100,5939.218263672896 +MarCo,3100,100,5939.218263672896 +MarDecUn,3100,100,5939.218263672896 +FedAvg,3100,100,17084.09754358264 +(MC)2MKP,3200,100,6044.887905455282 +MarIn,3200,100,6044.887905455282 +MarCo,3200,100,6044.887905455282 +MarDecUn,3200,100,6044.887905455282 +FedAvg,3200,100,17618.416388438476 +(MC)2MKP,3300,100,6150.557547237666 +MarIn,3300,100,6150.557547237666 +MarCo,3300,100,6150.557547237666 +MarDecUn,3300,100,6150.557547237666 +FedAvg,3300,100,18152.735233294312 +(MC)2MKP,3400,100,6256.227189020052 +MarIn,3400,100,6256.227189020052 +MarCo,3400,100,6256.227189020052 +MarDecUn,3400,100,6256.227189020052 +FedAvg,3400,100,18687.054078150148 +(MC)2MKP,3500,100,6361.896830802438 +MarIn,3500,100,6361.896830802438 +MarCo,3500,100,6361.896830802438 +MarDecUn,3500,100,6361.896830802438 +FedAvg,3500,100,19221.372923005987 +(MC)2MKP,3600,100,6467.566472584822 +MarIn,3600,100,6467.566472584822 +MarCo,3600,100,6467.566472584822 +MarDecUn,3600,100,6467.566472584822 +FedAvg,3600,100,19755.691767861823 +(MC)2MKP,3700,100,6573.236114367208 +MarIn,3700,100,6573.236114367208 +MarCo,3700,100,6573.236114367208 +MarDecUn,3700,100,6573.236114367208 +FedAvg,3700,100,20290.010612717662 +(MC)2MKP,3800,100,6678.905756149594 +MarIn,3800,100,6678.905756149594 +MarCo,3800,100,6678.905756149594 +MarDecUn,3800,100,6678.905756149594 +FedAvg,3800,100,20824.329457573498 +(MC)2MKP,3900,100,6784.575397931978 +MarIn,3900,100,6784.575397931978 +MarCo,3900,100,6784.575397931978 +MarDecUn,3900,100,6784.575397931978 +FedAvg,3900,100,21358.648302429334 +(MC)2MKP,4000,100,6890.245039714366 +MarIn,4000,100,6890.245039714366 +MarCo,4000,100,6890.245039714366 +MarDecUn,4000,100,6890.245039714366 +FedAvg,4000,100,21892.967147285173 +(MC)2MKP,4100,100,6995.91468149675 +MarIn,4100,100,6995.91468149675 +MarCo,4100,100,6995.91468149675 +MarDecUn,4100,100,6995.91468149675 +FedAvg,4100,100,22427.285992141005 +(MC)2MKP,4200,100,7101.584323279136 +MarIn,4200,100,7101.584323279136 +MarCo,4200,100,7101.584323279136 +MarDecUn,4200,100,7101.584323279136 +FedAvg,4200,100,22961.60483699685 +(MC)2MKP,4300,100,7207.253965061522 +MarIn,4300,100,7207.253965061522 +MarCo,4300,100,7207.253965061522 +MarDecUn,4300,100,7207.253965061522 +FedAvg,4300,100,23495.923681852684 +(MC)2MKP,4400,100,7312.923606843908 +MarIn,4400,100,7312.923606843908 +MarCo,4400,100,7312.923606843908 +MarDecUn,4400,100,7312.923606843908 +FedAvg,4400,100,24030.242526708524 +(MC)2MKP,4500,100,7418.593248626294 +MarIn,4500,100,7418.593248626294 +MarCo,4500,100,7418.593248626294 +MarDecUn,4500,100,7418.593248626294 +FedAvg,4500,100,24564.56137156436 +(MC)2MKP,4600,100,7524.262890408678 +MarIn,4600,100,7524.262890408678 +MarCo,4600,100,7524.262890408678 +MarDecUn,4600,100,7524.262890408678 +FedAvg,4600,100,25098.880216420195 +(MC)2MKP,4700,100,7629.932532191064 +MarIn,4700,100,7629.932532191064 +MarCo,4700,100,7629.932532191064 +MarDecUn,4700,100,7629.932532191064 +FedAvg,4700,100,25633.199061276035 +(MC)2MKP,4800,100,7735.60217397345 +MarIn,4800,100,7735.60217397345 +MarCo,4800,100,7735.60217397345 +MarDecUn,4800,100,7735.60217397345 +FedAvg,4800,100,26167.51790613187 +(MC)2MKP,4900,100,7841.271815755834 +MarIn,4900,100,7841.271815755834 +MarCo,4900,100,7841.271815755834 +MarDecUn,4900,100,7841.271815755834 +FedAvg,4900,100,26701.836750987713 +(MC)2MKP,5000,100,7946.94145753822 +MarIn,5000,100,7946.94145753822 +MarCo,5000,100,7946.94145753822 +MarDecUn,5000,100,7946.94145753822 +FedAvg,5000,100,27236.155595843553 diff --git a/stored_results/results_with_decreasing_marginal_costs.csv b/original_results/results_with_decreasing_marginal_costs.csv similarity index 100% rename from stored_results/results_with_decreasing_marginal_costs.csv rename to original_results/results_with_decreasing_marginal_costs.csv diff --git a/stored_results/results_with_increasing_marginal_costs.csv b/original_results/results_with_increasing_marginal_costs.csv similarity index 100% rename from stored_results/results_with_increasing_marginal_costs.csv rename to original_results/results_with_increasing_marginal_costs.csv diff --git a/original_results/results_with_random_costs.csv b/original_results/results_with_random_costs.csv new file mode 100644 index 0000000..6b17222 --- /dev/null +++ b/original_results/results_with_random_costs.csv @@ -0,0 +1,422 @@ +# Description of the experiment: +# +# - We generate the costs to up to 5.000 tasks for 10 and 100 resources. +# - All costs follow random functions (i.e., random costs) +# with RNG seeds [400..499]. +# - We schedule from 1.000 to 5.000 tasks in increments of 100. +# - We run (MC)^2MKP, MarIn, MarCo, MarDec, and FedAvg. +# - All resources have a lower limit of 5. +# - The first half of the resources have no upper limit. +# - The second half has an upper limit of 2*(tasks/resources). +# - Every result is verified and logged to a CSV file. +Scheduler,Tasks,Resources,Total Cost +(MC)2MKP,1000,10,219.12269425142705 +MarIn,1000,10,8379.518807141503 +MarCo,1000,10,22970.68776181747 +MarDec,1000,10,10066.280852659991 +FedAvg,1000,10,27039.321917007277 +(MC)2MKP,1100,10,203.95489256954667 +MarIn,1100,10,3544.1489483244563 +MarCo,1100,10,23631.84516712 +MarDec,1100,10,12565.823387692248 +FedAvg,1100,10,28671.974614683055 +(MC)2MKP,1200,10,180.40009319957556 +MarIn,1200,10,6724.884076879166 +MarCo,1200,10,22857.475812227563 +MarDec,1200,10,14577.333319134043 +FedAvg,1200,10,24977.321978918655 +(MC)2MKP,1300,10,197.73171975354782 +MarIn,1300,10,4847.736686035344 +MarCo,1300,10,21863.79021671735 +MarDec,1300,10,15217.221706919387 +FedAvg,1300,10,22495.050691077333 +(MC)2MKP,1400,10,188.28291213395318 +MarIn,1400,10,4840.80329233878 +MarCo,1400,10,19855.87492870251 +MarDec,1400,10,14937.856032658945 +FedAvg,1400,10,19073.63884529457 +(MC)2MKP,1500,10,160.88674095350487 +MarIn,1500,10,7123.869242505591 +MarCo,1500,10,21589.406772126775 +MarDec,1500,10,11810.411381054239 +FedAvg,1500,10,26990.185144048744 +(MC)2MKP,1600,10,164.32354803295098 +MarIn,1600,10,6445.8812003983585 +MarCo,1600,10,23679.61239412959 +MarDec,1600,10,15688.558434296307 +FedAvg,1600,10,29163.27783219167 +(MC)2MKP,1700,10,160.68638725438555 +MarIn,1700,10,5973.800491945734 +MarCo,1700,10,20792.523556605607 +MarDec,1700,10,15780.288845129717 +FedAvg,1700,10,31358.304074263207 +(MC)2MKP,1800,10,156.30864677871566 +MarIn,1800,10,5977.279038298237 +MarCo,1800,10,20637.0972506331 +MarDec,1800,10,13159.015334484922 +FedAvg,1800,10,28282.407567770253 +(MC)2MKP,1900,10,163.74341380075433 +MarIn,1900,10,6540.528478869282 +MarCo,1900,10,21207.5530912915 +MarDec,1900,10,10441.201008087095 +FedAvg,1900,10,34143.89343893098 +(MC)2MKP,2000,10,147.5629899125829 +MarIn,2000,10,3785.055457618769 +MarCo,2000,10,22137.26032333254 +MarDec,2000,10,14521.139974372558 +FedAvg,2000,10,21155.66956717409 +(MC)2MKP,2100,10,149.55864948327851 +MarIn,2100,10,8420.292443851498 +MarCo,2100,10,23834.091842040056 +MarDec,2100,10,11341.320245979265 +FedAvg,2100,10,27860.284547976593 +(MC)2MKP,2200,10,155.67321181881704 +MarIn,2200,10,6256.721664974133 +MarCo,2200,10,20039.872208962555 +MarDec,2200,10,14884.393654978203 +FedAvg,2200,10,30362.27442075604 +(MC)2MKP,2300,10,150.28998864970885 +MarIn,2300,10,9275.369581475836 +MarCo,2300,10,20043.048816092185 +MarDec,2300,10,11526.491542360061 +FedAvg,2300,10,31748.06210940115 +(MC)2MKP,2400,10,141.29293503238762 +MarIn,2400,10,9797.44842129647 +MarCo,2400,10,23267.01922375819 +MarDec,2400,10,14169.97227288113 +FedAvg,2400,10,25149.15173547904 +(MC)2MKP,2500,10,131.48624711706503 +MarIn,2500,10,10157.727869855888 +MarCo,2500,10,20795.58186829436 +MarDec,2500,10,12559.457847232827 +FedAvg,2500,10,18786.211721510812 +(MC)2MKP,2600,10,87.9617404679248 +MarIn,2600,10,5796.96708426558 +MarCo,2600,10,21923.40340612268 +MarDec,2600,10,15085.414006512448 +FedAvg,2600,10,28208.04976800088 +(MC)2MKP,2700,10,93.8402453065148 +MarIn,2700,10,5701.932273680224 +MarCo,2700,10,23834.05223311074 +MarDec,2700,10,13878.036562194653 +FedAvg,2700,10,25751.73863763778 +(MC)2MKP,2800,10,92.54985153505692 +MarIn,2800,10,4269.503868402366 +MarCo,2800,10,20507.56243141796 +MarDec,2800,10,14852.705654501742 +FedAvg,2800,10,24204.988966589433 +(MC)2MKP,2900,10,79.07306009458716 +MarIn,2900,10,5105.2798419344535 +MarCo,2900,10,20897.4914058847 +MarDec,2900,10,12635.069613182417 +FedAvg,2900,10,26885.158057290246 +(MC)2MKP,3000,10,83.00923245910484 +MarIn,3000,10,4038.535421684491 +MarCo,3000,10,19563.689458697147 +MarDec,3000,10,15718.94101971974 +FedAvg,3000,10,24048.16850781605 +(MC)2MKP,3100,10,82.38036186168416 +MarIn,3100,10,2229.997691303323 +MarCo,3100,10,22747.99179835054 +MarDec,3100,10,11714.767939137715 +FedAvg,3100,10,33998.819491144546 +(MC)2MKP,3200,10,66.21067898761275 +MarIn,3200,10,2672.175317860804 +MarCo,3200,10,23589.09184235637 +MarDec,3200,10,13489.739339318274 +FedAvg,3200,10,29331.117080202705 +(MC)2MKP,3300,10,74.92368989182707 +MarIn,3300,10,6349.0605057576395 +MarCo,3300,10,20571.31780887711 +MarDec,3300,10,14966.675178605932 +FedAvg,3300,10,17853.275671799875 +(MC)2MKP,3400,10,51.35924788255045 +MarIn,3400,10,4911.511801041026 +MarCo,3400,10,20399.12488874045 +MarDec,3400,10,11027.232972366737 +FedAvg,3400,10,23392.79574994267 +(MC)2MKP,3500,10,59.034755835053794 +MarIn,3500,10,6067.8866495467355 +MarCo,3500,10,21017.702711036058 +MarDec,3500,10,10233.067904754647 +FedAvg,3500,10,32097.989442489124 +(MC)2MKP,3600,10,57.17847293563737 +MarIn,3600,10,8138.525732454854 +MarCo,3600,10,21183.48444701113 +MarDec,3600,10,11902.438609948127 +FedAvg,3600,10,23599.110686404394 +(MC)2MKP,3700,10,51.4480672072163 +MarIn,3700,10,6172.881274771715 +MarCo,3700,10,19463.40029095402 +MarDec,3700,10,11625.55337257755 +FedAvg,3700,10,21992.392347944333 +(MC)2MKP,3800,10,45.44575699390261 +MarIn,3800,10,2053.927377201016 +MarCo,3800,10,23379.307300821692 +MarDec,3800,10,12625.89182257018 +FedAvg,3800,10,15952.961973364096 +(MC)2MKP,3900,10,54.308341226381685 +MarIn,3900,10,8383.614922123827 +MarCo,3900,10,19848.034246261563 +MarDec,3900,10,12280.205358848842 +FedAvg,3900,10,17175.23031322619 +(MC)2MKP,4000,10,39.59649814479124 +MarIn,4000,10,4864.4916676460225 +MarCo,4000,10,19922.798642789778 +MarDec,4000,10,11645.88161450321 +FedAvg,4000,10,20478.229225870222 +(MC)2MKP,4100,10,35.72394787918351 +MarIn,4100,10,5193.725992193529 +MarCo,4100,10,22109.159848605075 +MarDec,4100,10,13145.066746248598 +FedAvg,4100,10,25536.339137813127 +(MC)2MKP,4200,10,46.73295757611214 +MarIn,4200,10,8579.231346503848 +MarCo,4200,10,21616.219045907488 +MarDec,4200,10,13397.199254771 +FedAvg,4200,10,17536.466660728875 +(MC)2MKP,4300,10,45.458892361979615 +MarIn,4300,10,8913.577262369308 +MarCo,4300,10,21185.335514944036 +MarDec,4300,10,12496.58317185652 +FedAvg,4300,10,29421.551026003162 +(MC)2MKP,4400,10,42.67237234636136 +MarIn,4400,10,6414.456603807648 +MarCo,4400,10,23812.695561139917 +MarDec,4400,10,15217.633920122353 +FedAvg,4400,10,27647.31139290197 +(MC)2MKP,4500,10,40.87106717633658 +MarIn,4500,10,5876.790422102937 +MarCo,4500,10,18977.864598929267 +MarDec,4500,10,14858.742660436186 +FedAvg,4500,10,22862.633236929352 +(MC)2MKP,4600,10,39.097513681111806 +MarIn,4600,10,6670.927454725905 +MarCo,4600,10,19419.922715175762 +MarDec,4600,10,15348.967374993603 +FedAvg,4600,10,26057.857187196947 +(MC)2MKP,4700,10,32.18542987337639 +MarIn,4700,10,6029.189797298737 +MarCo,4700,10,21037.765433930515 +MarDec,4700,10,14207.837701311426 +FedAvg,4700,10,20720.99051082601 +(MC)2MKP,4800,10,36.37145579158144 +MarIn,4800,10,8615.43737181094 +MarCo,4800,10,19876.737395775646 +MarDec,4800,10,14515.954606486128 +FedAvg,4800,10,32774.52904325699 +(MC)2MKP,4900,10,38.208330679480035 +MarIn,4900,10,6284.9216593318815 +MarCo,4900,10,19446.655831756085 +MarDec,4900,10,15436.474037279908 +FedAvg,4900,10,23395.784435164765 +(MC)2MKP,5000,10,37.117510018015565 +MarIn,5000,10,3505.421684999826 +MarCo,5000,10,21186.83285277323 +MarDec,5000,10,16394.078217732396 +FedAvg,5000,10,27894.690087303454 +(MC)2MKP,1000,100,33668.1040478722 +MarIn,1000,100,89639.06230975057 +MarCo,1000,100,240048.22044852804 +MarDec,1000,100,201337.8697333555 +FedAvg,1000,100,266215.58832265367 +(MC)2MKP,1100,100,28598.464829721775 +MarIn,1100,100,81462.79035141623 +MarCo,1100,100,240632.86141895497 +MarDec,1100,100,205238.6783308295 +FedAvg,1100,100,232018.80986879225 +(MC)2MKP,1200,100,25082.284395840314 +MarIn,1200,100,83349.74658874974 +MarCo,1200,100,235571.52237256608 +MarDec,1200,100,195940.25897045585 +FedAvg,1200,100,266027.6299958374 +(MC)2MKP,1300,100,21771.332710282717 +MarIn,1300,100,85819.82123647218 +MarCo,1300,100,230692.62485962518 +MarDec,1300,100,194015.32324560103 +FedAvg,1300,100,245237.20766251103 +(MC)2MKP,1400,100,19105.234364568572 +MarIn,1400,100,84966.5411187908 +MarCo,1400,100,233793.37090201463 +MarDec,1400,100,193004.04090420267 +FedAvg,1400,100,273354.56110076763 +(MC)2MKP,1500,100,17032.374256598767 +MarIn,1500,100,74344.43514904425 +MarCo,1500,100,236340.96960836538 +MarDec,1500,100,191332.70524825458 +FedAvg,1500,100,238523.3792223594 +(MC)2MKP,1600,100,15446.017090738214 +MarIn,1600,100,75221.94789849865 +MarCo,1600,100,233781.3544336346 +MarDec,1600,100,201429.9544546671 +FedAvg,1600,100,247626.65651080292 +(MC)2MKP,1700,100,13994.451841865413 +MarIn,1700,100,80943.12085842855 +MarCo,1700,100,239069.9224807321 +MarDec,1700,100,199251.10761091448 +FedAvg,1700,100,238550.16407961742 +(MC)2MKP,1800,100,13039.954281760709 +MarIn,1800,100,74231.08386588831 +MarCo,1800,100,235498.16619271142 +MarDec,1800,100,199406.4740593542 +FedAvg,1800,100,252134.48114302199 +(MC)2MKP,1900,100,12300.960942647647 +MarIn,1900,100,79564.98050413186 +MarCo,1900,100,237586.19791063466 +MarDec,1900,100,198099.3383687058 +FedAvg,1900,100,230963.73545479166 +(MC)2MKP,2000,100,11542.15248959942 +MarIn,2000,100,74574.35860338793 +MarCo,2000,100,238208.27420982323 +MarDec,2000,100,201889.8865641855 +FedAvg,2000,100,230445.26363230747 +(MC)2MKP,2100,100,10889.67298873199 +MarIn,2100,100,68188.9615309935 +MarCo,2100,100,232651.50773929027 +MarDec,2100,100,196203.84314274698 +FedAvg,2100,100,255174.63308032145 +(MC)2MKP,2200,100,10222.608277184196 +MarIn,2200,100,68390.6646677803 +MarCo,2200,100,234215.68423747737 +MarDec,2200,100,190921.6704675615 +FedAvg,2200,100,270515.16775793134 +(MC)2MKP,2300,100,9621.673905843338 +MarIn,2300,100,65156.11532046926 +MarCo,2300,100,237327.32716578542 +MarDec,2300,100,197706.93748745963 +FedAvg,2300,100,257372.03404892696 +(MC)2MKP,2400,100,9189.194918798306 +MarIn,2400,100,61690.81629427531 +MarCo,2400,100,239644.01273269244 +MarDec,2400,100,206161.69042007835 +FedAvg,2400,100,222123.80324120948 +(MC)2MKP,2500,100,8833.89145896983 +MarIn,2500,100,67022.01526817838 +MarCo,2500,100,236554.29777192668 +MarDec,2500,100,190655.72399440123 +FedAvg,2500,100,258291.04656182864 +(MC)2MKP,2600,100,8471.076581534247 +MarIn,2600,100,69327.49861770544 +MarCo,2600,100,238914.90404144605 +MarDec,2600,100,200550.26609088562 +FedAvg,2600,100,220977.63559998566 +(MC)2MKP,2700,100,8228.66685326153 +MarIn,2700,100,68433.68445146945 +MarCo,2700,100,238726.0068502049 +MarDec,2700,100,205888.79230450594 +FedAvg,2700,100,267481.01827342 +(MC)2MKP,2800,100,7833.589282032641 +MarIn,2800,100,68182.35702651386 +MarCo,2800,100,234051.36072121595 +MarDec,2800,100,197685.40522523434 +FedAvg,2800,100,235670.14690452936 +(MC)2MKP,2900,100,7577.602083634291 +MarIn,2900,100,64711.15375248953 +MarCo,2900,100,230826.23148784658 +MarDec,2900,100,205058.76430112278 +FedAvg,2900,100,236992.22064225614 +(MC)2MKP,3000,100,7022.37040825501 +MarIn,3000,100,58990.85560249255 +MarCo,3000,100,234579.95148445148 +MarDec,3000,100,198549.4696166181 +FedAvg,3000,100,231364.4316044102 +(MC)2MKP,3100,100,6844.942258746 +MarIn,3100,100,57630.32382908908 +MarCo,3100,100,234721.16924116533 +MarDec,3100,100,200507.06511896435 +FedAvg,3100,100,267252.8770095608 +(MC)2MKP,3200,100,6623.207776722141 +MarIn,3200,100,67225.38182040774 +MarCo,3200,100,237147.89918222724 +MarDec,3200,100,201467.54250613396 +FedAvg,3200,100,266774.7991856303 +(MC)2MKP,3300,100,6424.38547225099 +MarIn,3300,100,64822.02717487238 +MarCo,3300,100,235500.57260159779 +MarDec,3300,100,181125.09462396227 +FedAvg,3300,100,240176.4906331365 +(MC)2MKP,3400,100,6295.937573757008 +MarIn,3400,100,68223.2649618099 +MarCo,3400,100,240208.15246301203 +MarDec,3400,100,204804.23444595444 +FedAvg,3400,100,254391.13978131875 +(MC)2MKP,3500,100,6160.845656138052 +MarIn,3500,100,65254.6965834956 +MarCo,3500,100,232386.2905764187 +MarDec,3500,100,197328.87474049616 +FedAvg,3500,100,259212.89695553778 +(MC)2MKP,3600,100,6052.30618325987 +MarIn,3600,100,65278.06431081815 +MarCo,3600,100,236193.52331739315 +MarDec,3600,100,204313.60180754217 +FedAvg,3600,100,258067.9524885461 +(MC)2MKP,3700,100,5665.148632268197 +MarIn,3700,100,59256.7950660442 +MarCo,3700,100,229848.23004138056 +MarDec,3700,100,195250.09860475268 +FedAvg,3700,100,251301.471996461 +(MC)2MKP,3800,100,5562.11849601308 +MarIn,3800,100,67713.62112483213 +MarCo,3800,100,233041.9928655029 +MarDec,3800,100,188274.06326107396 +FedAvg,3800,100,256521.4331282808 +(MC)2MKP,3900,100,5137.920182635332 +MarIn,3900,100,59322.27054424385 +MarCo,3900,100,233072.45610198594 +MarDec,3900,100,187531.1049868093 +FedAvg,3900,100,223133.12646703533 +(MC)2MKP,4000,100,5048.224665435078 +MarIn,4000,100,66940.55120016212 +MarCo,4000,100,234748.76489854962 +MarDec,4000,100,198732.95370761288 +FedAvg,4000,100,259539.16638115296 +(MC)2MKP,4100,100,4949.028095247071 +MarIn,4100,100,67520.41781840114 +MarCo,4100,100,231412.48213798768 +MarDec,4100,100,201160.65062665037 +FedAvg,4100,100,253333.8129496079 +(MC)2MKP,4200,100,4868.598784250025 +MarIn,4200,100,51929.35920641546 +MarCo,4200,100,234803.18112543458 +MarDec,4200,100,181114.7271269292 +FedAvg,4200,100,235185.5991408447 +(MC)2MKP,4300,100,4706.114225107791 +MarIn,4300,100,66577.13325233843 +MarCo,4300,100,233563.9429097916 +MarDec,4300,100,187909.65457920238 +FedAvg,4300,100,244427.3450101869 +(MC)2MKP,4400,100,4576.735786518813 +MarIn,4400,100,51308.5268470589 +MarCo,4400,100,233556.1984592324 +MarDec,4400,100,186130.4636787907 +FedAvg,4400,100,239234.012117882 +(MC)2MKP,4500,100,4504.846679011583 +MarIn,4500,100,55382.20717020267 +MarCo,4500,100,235306.80126567205 +MarDec,4500,100,198066.27372315334 +FedAvg,4500,100,231269.58266116673 +(MC)2MKP,4600,100,4423.379046821711 +MarIn,4600,100,48726.300506703476 +MarCo,4600,100,235715.06940334686 +MarDec,4600,100,196729.07572294254 +FedAvg,4600,100,235803.54088393366 +(MC)2MKP,4700,100,4356.789702302539 +MarIn,4700,100,51588.1472556215 +MarCo,4700,100,232725.7611985694 +MarDec,4700,100,186356.16100069493 +FedAvg,4700,100,246865.08496020996 +(MC)2MKP,4800,100,4290.80165046691 +MarIn,4800,100,62009.14418313244 +MarCo,4800,100,239293.25010910296 +MarDec,4800,100,206739.6616868976 +FedAvg,4800,100,245727.75986580615 +(MC)2MKP,4900,100,4201.761135345987 +MarIn,4900,100,65186.19229905608 +MarCo,4900,100,234430.7244673436 +MarDec,4900,100,191789.09107967047 +FedAvg,4900,100,263315.6923855967 +(MC)2MKP,5000,100,4052.209214948088 +MarIn,5000,100,49128.51057885034 +MarCo,5000,100,235447.49047857343 +MarDec,5000,100,187923.01751195162 +FedAvg,5000,100,246513.99191707902 diff --git a/original_results/run_all_analysis.sh b/original_results/run_all_analysis.sh new file mode 100755 index 0000000..1e0f2ce --- /dev/null +++ b/original_results/run_all_analysis.sh @@ -0,0 +1,10 @@ +#!/usr/bin/env bash + +set -x + +find . -name "Analysis*.py" | while read line; do + echo "$line" >> result_analysis.txt + python3 "$line" >> result_analysis.txt +done + +echo "Please check file result_analysis.txt for the textual results and the pdf files for the figures." diff --git a/run_all_timing_experiments.sh b/run_all_timing_experiments.sh index 8c821d8..944a7c3 100755 --- a/run_all_timing_experiments.sh +++ b/run_all_timing_experiments.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash set -x diff --git a/run_all_total_cost_experiments.sh b/run_all_total_cost_experiments.sh index e977698..4264ca7 100755 --- a/run_all_total_cost_experiments.sh +++ b/run_all_total_cost_experiments.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash set -x diff --git a/run_analysis_on_new_results.sh b/run_analysis_on_new_results.sh index 22c8eb9..d0c09aa 100755 --- a/run_analysis_on_new_results.sh +++ b/run_analysis_on_new_results.sh @@ -1,9 +1,10 @@ -#!/bin/bash +#!/usr/bin/env bash -set -x - -cp stored_results/Analysis* . +cp original_results/Analysis* . find . -maxdepth 1 -name "Analysis*.py" | while read line; do - python3 "$line" + echo "$line" >> result_analysis.txt + python3 "$line" >> result_analysis.txt done + +echo "Please check file result_analysis.txt for the textual results and the pdf files for the figures." diff --git a/stored_results/run_all_analysis.sh b/stored_results/run_all_analysis.sh deleted file mode 100755 index 8fb6886..0000000 --- a/stored_results/run_all_analysis.sh +++ /dev/null @@ -1,7 +0,0 @@ -#!/bin/bash - -set -x - -find . -name "Analysis*.py" | while read line; do - python3 "$line" -done diff --git a/unitary_tests.py b/unitary_tests.py new file mode 100644 index 0000000..62e684a --- /dev/null +++ b/unitary_tests.py @@ -0,0 +1,272 @@ +#!/usr/bin/env python3 +"""Unitary tests for the different modules.""" + +import unittest +import numpy as np +import os + +import code.devices as devices +import code.schedulers as schedulers +import code.support as support + + +class TestDevice(unittest.TestCase): + def setUp(self): + self.tests = 2 + self.size = 3 + self.matrix = np.zeros(shape=(self.tests, self.size+1)) + + def test_linear(self): + devices.create_linear_costs(0, self.matrix, 0, self.size) + self.assertEqual(self.matrix[0][0], 5.939321535345923) + self.assertEqual(self.matrix[0][1], 13.376025832697698) + self.assertEqual(self.matrix[0][2], 20.812730130049474) + self.assertEqual(self.matrix[0][3], 28.249434427401248) + + devices.create_linear_costs(10, self.matrix, 1, self.size) + self.assertEqual(self.matrix[1][0], 7.941885789400714) + self.assertEqual(self.matrix[1][1], 9.128653333635327) + self.assertEqual(self.matrix[1][2], 10.31542087786994) + self.assertEqual(self.matrix[1][3], 11.502188422104554) + + def test_quadratic(self): + devices.create_quadratic_costs(20, self.matrix, 0, self.size) + self.assertEqual(self.matrix[0][0], 6.293177209695468) + self.assertEqual(self.matrix[0][1], 24.396377326152603) + self.assertEqual(self.matrix[0][2], 60.54713057315448) + self.assertEqual(self.matrix[0][3], 114.7454369507011) + + devices.create_quadratic_costs(30, self.matrix, 1, self.size) + self.assertEqual(self.matrix[1][0], 6.797291824615015) + self.assertEqual(self.matrix[1][1], 18.191459379352082) + self.assertEqual(self.matrix[1][2], 43.52048923013118) + self.assertEqual(self.matrix[1][3], 82.78438137695233) + + def test_nlogn(self): + devices.create_nlogn_costs(40, self.matrix, 0, self.size) + self.assertEqual(self.matrix[0][0], 4.669183252722576) + self.assertEqual(self.matrix[0][1], 5.707721764703263) + self.assertEqual(self.matrix[0][2], 7.961272446810852) + self.assertEqual(self.matrix[0][3], 10.9004143246067) + + devices.create_nlogn_costs(50, self.matrix, 1, self.size) + self.assertEqual(self.matrix[1][0], 5.451414809842193) + self.assertEqual(self.matrix[1][1], 7.567418437443891) + self.assertEqual(self.matrix[1][2], 12.158987612119443) + self.assertEqual(self.matrix[1][3], 18.147436575452378) + + def test_logn(self): + devices.create_logn_costs(3, self.matrix, 0, self.size) + self.assertEqual(self.matrix[0][0], 5.957181123171179) + self.assertEqual(self.matrix[0][1], 11.067984303737756) + self.assertEqual(self.matrix[0][2], 14.057612512935624) + self.assertEqual(self.matrix[0][3], 16.178787484304333) + + devices.create_logn_costs(4, self.matrix, 1, self.size) + self.assertEqual(self.matrix[1][0], 9.70326855112309) + self.assertEqual(self.matrix[1][1], 13.8102281473317) + self.assertEqual(self.matrix[1][2], 16.212645503090638) + self.assertEqual(self.matrix[1][3], 17.91718774354031) + + def test_recursive(self): + devices.create_recursive_costs(60, self.matrix, 0, self.size) + self.assertEqual(self.matrix[0][0], 3.7078599704195687) + self.assertEqual(self.matrix[0][1], 6.390372317496471) + self.assertEqual(self.matrix[0][2], 10.299016410210445) + self.assertEqual(self.matrix[0][3], 17.290762542805226) + + devices.create_recursive_costs(70, self.matrix, 1, self.size) + self.assertEqual(self.matrix[1][0], 9.34732484757063) + self.assertEqual(self.matrix[1][1], 18.19916149725357) + self.assertEqual(self.matrix[1][2], 24.460842950613547) + self.assertEqual(self.matrix[1][3], 33.60737451412772) + + + def test_random_costs(self): + devices.create_random_costs(19, self.matrix, 0, self.size) + self.assertEqual(self.matrix[0][0], 0.29260080524835475) + self.assertEqual(self.matrix[0][1], 2.283749150024569) + self.assertEqual(self.matrix[0][2], 0.7408139194989811) + self.assertEqual(self.matrix[0][3], 0.4143950624289586) + + devices.create_random_costs(18, self.matrix, 1, self.size) + self.assertEqual(self.matrix[1][0], 1.951122725218775) + self.assertEqual(self.matrix[1][1], 1.5163601212045288) + self.assertEqual(self.matrix[1][2], 2.6358044127646556) + self.assertEqual(self.matrix[1][3], 0.5455206761146033) + + +class TestSchedulers(unittest.TestCase): + def setUp(self): + self.tasks = 8 + self.resources = 3 + self.lower_limit = np.full(shape=self.resources, + fill_value=1, dtype=int) + self.upper_limit = np.full(shape=self.resources, + fill_value=4, dtype=int) + + def test_mc2mkp(self): + cost = np.array([[0.0, 3.0, 2.0, 4.0, 6.0], + [0.0, 1.0, 5.0, 2.0, 3.0], + [0.0, 8.0, 6.0, 4.0, 2.0]]) + assignment = schedulers.mc2mkp(self.tasks, + self.resources, + cost, + self.lower_limit, + self.upper_limit) + self.assertEqual(assignment[0], 3) + self.assertEqual(assignment[1], 1) + self.assertEqual(assignment[2], 4) + + def test_marin(self): + cost = np.array([[0.1, 1.1, 3.1, 7.1, 12.1], + [0.0, 1.5, 4.0, 7.5, 11.5], + [0.0, 2.0, 4.5, 7.5, 10.5]]) + assignment = schedulers.marin(self.tasks, + self.resources, + cost, + self.lower_limit, + self.upper_limit) + self.assertEqual(assignment[0], 2) + self.assertEqual(assignment[1], 2) + self.assertEqual(assignment[2], 4) + + def test_marco(self): + cost = np.array([[0.1, 1.1, 2.1, 3.1, 4.1], + [0.0, 1.5, 3.0, 4.5, 6.0], + [0.0, 2.0, 4.0, 6.0, 8.0]]) + assignment = schedulers.marco(self.tasks, + self.resources, + cost, + self.lower_limit, + self.upper_limit) + self.assertEqual(assignment[0], 4) + self.assertEqual(assignment[1], 3) + self.assertEqual(assignment[2], 1) + + def test_mardecun(self): + cost = np.array([[0.0, 4.0, 7.0, 9.0, 10.0], + [0.0, 3.0, 6.0, 9.0, 12.0], + [0.0, 3.0, 5.0, 7.0, 8.5]]) + tasks = 6 + assignment = schedulers.mardecun(tasks, + self.resources, + cost, + self.lower_limit) + self.assertEqual(assignment[0], 1) + self.assertEqual(assignment[1], 1) + self.assertEqual(assignment[2], 4) + + def test_mardec(self): + cost = np.array([[0.0, 4.0, 7.0, 9.0, 10.0], + [0.0, 3.0, 6.0, 9.0, 12.0], + [0.0, 3.0, 5.0, 7.0, 8.5], + [0.0, 4.0, 7.0, 10.0, 11.0]]) + tasks = 6 + resources = 4 + lower_limit = np.array([1, 1, 1, 0]) + upper_limit = np.array([4, 3, 4, 2]) + assignment = schedulers.mardec(tasks, + resources, + cost, + lower_limit, + upper_limit) + self.assertEqual(assignment[0], 1) + self.assertEqual(assignment[1], 1) + self.assertEqual(assignment[2], 4) + self.assertEqual(assignment[3], 0) + + def test_fedavg(self): + tasks = 10 + resources = 3 + assignment = schedulers.fedavg(tasks, resources) + self.assertEqual(assignment[0], 4) + self.assertEqual(assignment[1], 3) + self.assertEqual(assignment[2], 3) + + +class TestSupport(unittest.TestCase): + def setUp(self): + self.tasks = 4 + self.resources = 3 + self.cost = np.array([[0.5, 0.5, 1.5, 2.0, 200], + [0.0, 0.4, 2.0, 3.0, 4.0], + [0.0, 1.5, 2.5, 3.5, 4.5]]) + + def test_total_cost(self): + assignment = np.array([0, 0, 0]) + total_cost = support.get_total_cost(self.cost, assignment) + self.assertEqual(total_cost, 0.5) + + assignment = np.array([4, 4, 4]) + total_cost = support.get_total_cost(self.cost, assignment) + self.assertEqual(total_cost, 208.5) + + assignment = np.array([2, 1, 1]) + total_cost = support.get_total_cost(self.cost, assignment) + self.assertEqual(total_cost, 3.4) + + def test_check_limits(self): + assignment = np.array([4, 4, 4]) + lower_limit = np.array([1, 1, 1]) + upper_limit = np.array([5, 5, 5]) + check = support.check_limits(assignment, lower_limit, upper_limit) + self.assertTrue(check) + + check = support.check_limits(assignment, assignment, assignment) + self.assertTrue(check) + + assignment = np.array([0, 4, 4]) + check = support.check_limits(assignment, lower_limit, upper_limit) + self.assertFalse(check) + + assignment = np.array([4, 4, 6]) + check = support.check_limits(assignment, lower_limit, upper_limit) + self.assertFalse(check) + + assignment = np.array([0, 4, 6]) + check = support.check_limits(assignment, lower_limit, upper_limit) + self.assertFalse(check) + + def test_check_total_assigned(self): + assignment = np.array([4, 4, 4]) + tasks = 12 + check = support.check_total_assigned(tasks, assignment) + self.assertTrue(check) + + tasks = 13 + check = support.check_total_assigned(tasks, assignment) + self.assertFalse(check) + + tasks = 11 + check = support.check_total_assigned(tasks, assignment) + self.assertFalse(check) + + +class TestLogger(unittest.TestCase): + def setUp(self): + self.filename = 'dummy_file.txt' + self.logger = support.Logger(self.filename) + + def test_attributes(self): + self.assertFalse(self.logger.verbosity) + self.assertEqual(self.filename, self.logger.filename) + + def test_logger(self): + a = 'a' + b = 'b' + c = 'c' + expected_log = 'ca\nb\n' + self.logger.header(c) + self.logger.store(a) + self.logger.store(b) + self.logger.finish() + + with open(self.filename, 'r') as logfile: + written_log = logfile.read() + self.assertEqual(expected_log, written_log) + os.remove(self.filename) + + +if __name__ == '__main__': + unittest.main()