diff --git a/introduction_to_applying_machine_learning/visual_object_detection/visual_object_detection.ipynb b/introduction_to_applying_machine_learning/visual_object_detection/visual_object_detection.ipynb index 63da4410e3..1aa383da49 100644 --- a/introduction_to_applying_machine_learning/visual_object_detection/visual_object_detection.ipynb +++ b/introduction_to_applying_machine_learning/visual_object_detection/visual_object_detection.ipynb @@ -2,7 +2,6 @@ "cells": [ { "cell_type": "markdown", - "id": "7180d03e", "metadata": { "papermill": { "duration": 0.075676, @@ -24,14 +23,14 @@ "\n", "Training a model from scratch in general is time-consuming and requires large compute resources. When the training data is small, we cannot expect to train a very performant model. A better alternative is to finetune a pretrained model on the target dataset. AWS Sagemaker provides high-quality pretrained models that were trained on very large datasets. Finetuning these models on new dataset takes only fractional training time compard to training from scratch.\n", "\n", - "In this notebook, we will demonstrate how to use two types of Amazon Sagemaker built-in OD models to finetune on the *[Steel Surface Defect](https://github.com/siddhartamukherjee/NEU-DET-Steel-Surface-Defect-Detection)* dataset, which is used in this solution. \n", + "In this notebook, we demonstrate how to use two types of Amazon Sagemaker built-in OD models to finetune on the *[Steel Surface Defect](https://github.com/siddhartamukherjee/NEU-DET-Steel-Surface-Defect-Detection)* dataset, which is used in this solution. \n", "* Type 1 (legacy): uses a built-in legacy [Object Detection algorithm](https://docs.aws.amazon.com/sagemaker/latest/dg/object-detection.html) and uses the *Single Shot multibox Detector* (SSD) model with either VGG or ResNet backbone, and was pretrained on the ImageNet dataset. \n", "* Type 2 (latest): provides [9 pretrained OD models](https://sagemaker.readthedocs.io/en/stable/doc_utils/pretrainedmodels.html?highlight=jumpstart#built-in-algorithms-with-pre-trained-model-table), including 8 SSD models and 1 FasterRCNN model. These models use VGG, ResNet, or MobileNet as backbone, and were pretrained on COCO, VOC, or FPN datasets. \n", "\n", "\n", - "For each type of model, besides training with default hyperparameters, we will also perform hyperparameter tuning (i.e., HPO) using [Sagemaker Automatic Model Tuning (AMT)](https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning.html) to train even better model. \n", + "For each type of model, besides training with default hyperparameters, we also perform hyperparameter tuning (i.e., HPO) using [Sagemaker Automatic Model Tuning (AMT)](https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning.html) to train even better model. \n", "\n", - "Running the whole notebook will take about 8 hours. The most time-consuming part is running HPO jobs for both types of models. You could choose to run more HPO jobs in parallel in order to reduce running time if there are more EC2 instances available.\n", + "Running the whole notebook takes about 8 hours. The most time-consuming part is running HPO jobs for both types of models. You could choose to run more HPO jobs in parallel in order to reduce running time if there are more EC2 instances available.\n", "\n", "---\n", "\n", @@ -49,7 +48,7 @@ "\n", "** **ATTENTION** ** \n", "\n", - "* Running the notebook end-to-end will take 8~9 hours. We changed some parameter values so that the notebook took much shorter time to finish, at the cost of model trainig non-convergence.\n", + "* Running the notebook end-to-end takes 8~9 hours. We changed some parameter values so that the notebook took much shorter time to finish, at the cost of model trainig non-convergence.\n", "* Please change them back when you want to train till convergence. These parameters include `num_epochs=100` for training all models, and `max_jobs=20`, `max_parallel_jobs=10` for hyperparameter tuning.\n", "* The shown results in this notebook is for fully-convergent models.\n", "\n", @@ -59,7 +58,6 @@ { "cell_type": "code", "execution_count": null, - "id": "ccd623bb", "metadata": {}, "outputs": [], "source": [ @@ -71,7 +69,6 @@ { "cell_type": "code", "execution_count": 14, - "id": "02e197d6", "metadata": { "papermill": { "duration": 6.156173, @@ -97,7 +94,6 @@ { "cell_type": "code", "execution_count": 4, - "id": "7d19e33a", "metadata": { "papermill": { "duration": 4.916572, @@ -150,7 +146,6 @@ { "cell_type": "code", "execution_count": 15, - "id": "8ca002c0", "metadata": { "pycharm": { "name": "#%%\n" @@ -172,7 +167,6 @@ { "cell_type": "code", "execution_count": 16, - "id": "ed62388a", "metadata": { "pycharm": { "name": "#%%\n" @@ -202,7 +196,6 @@ }, { "cell_type": "markdown", - "id": "7eff1dc5", "metadata": { "papermill": { "duration": 0.087127, @@ -223,17 +216,16 @@ "The *steel surface dataset* used in this solution contains one xml file for each image as annotation. However, \n", "neither model uses xml annotations. The Type 1 (legacy) OD model requires either RecordIO or image format in either [*file mode* or *pipe mode*](https://docs.aws.amazon.com/sagemaker/latest/dg/model-access-training-data.html). The Type 2 (latest) OD model requires the input must be a directory with a sub-directory of images and a `annotations.json` file. Please check Section 3 of this [notebook](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/jumpstart_object_detection/Amazon_JumpStart_Object_Detection.ipynb) for more explanation.\n", "\n", - "In this notebook, we will split the data to be train:val:test = 64:16:20. We allocate 20% data as test data to numerically compare all trained models in the end of the notebook. The steel surface dataset has 1800 images in 6 categories, we randomly allocate 20% images from each category to the test data.\n", + "In this notebook, we split the data to be train:val:test = 64:16:20. We allocate 20% data as test data to numerically compare all trained models in the end of the notebook. The steel surface dataset has 1800 images in 6 categories, we randomly allocate 20% images from each category to the test data.\n", "\n", - "We provide a script to convert the remaining 80% xmls to a single `annotations.json` for training the Type 2 (latest) OD model (under the hood, the source code will automatically split the data to be train:val=80:20, equivalent to 64% of all data as train and 16% as val). We provide another script to convert the `annotations.json` and corresponding images to RecordIO data for the Type 1 (legacy) OD model. \n", + "We provide a script to convert the remaining 80% xmls to a single `annotations.json` for training the Type 2 (latest) OD model (under the hood, the source code automatically splits the data to be train:val=80:20, equivalent to 64% of all data as train and 16% as val). We provide another script to convert the `annotations.json` and corresponding images to RecordIO data for the Type 1 (legacy) OD model. \n", "\n", "If your dataset follows the required input format for Type 1 (legacy) or Type 2 (latest) OD model, you do *not* need these conversions. " ] }, { "cell_type": "code", - "execution_count": 7, - "id": "35e28b79", + "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2022-08-05T08:10:55.622902Z", @@ -254,27 +246,7 @@ "scrolled": true, "tags": [] }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\n", - "category\tnum of images\n", - "--------------------\n", - "crazing:\t300\n", - "inclusion:\t300\n", - "patches:\t300\n", - "pitted:\t300\n", - "rolled-in:\t300\n", - "scratches:\t300\n", - "--------------------\n", - "\n", - "train_val:\t1440\n", - "test:\t360\n" - ] - } - ], + "outputs": [], "source": [ "# Allocate 20% data for testing different models in the end, convert\n", "# their xmls to test_annotations.json and the remaining 80% to annotations.json\n", @@ -288,7 +260,6 @@ { "cell_type": "code", "execution_count": 8, - "id": "cd678395", "metadata": { "execution": { "iopub.execute_input": "2022-08-05T08:10:56.347290Z", @@ -340,7 +311,6 @@ }, { "cell_type": "markdown", - "id": "c7dd8c4f", "metadata": { "papermill": { "duration": 0.088721, @@ -363,13 +333,12 @@ "**Input data**: follow the [instruction](https://docs.aws.amazon.com/sagemaker/latest/dg/object-detection.html), the legacy OD model supports both RecordIO and image types for training in `file` mode, or RecordIO in `pipe` mode. In this notebook, we use RecordIO in file mode.\n", "We provide a script for converting the `annotations.json` to RecordIO format. The [document](https://cv.gluon.ai/build/examples_datasets/detection_custom.html#lst-label-for-gluoncv-and-mxnet) and [example](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/object_detection_birds/object_detection_birds.html#Generate-RecordIO-files) provide some context for understanding the script.\n", "\n", - "This script first splits the data to train:val = 80:20 according to the `train-ratio`. This is equivalent to use 64% of all data for training and 16% for validation. Then converts each partition, including images and annotations, to a .rec file. We will use the validation data for selecting the best job in HPO training in the next section, and will use the test data for numerically comparing all finetuned models. " + "This script first splits the data to train:val = 80:20 according to the `train-ratio`. This is equivalent to use 64% of all data for training and 16% for validation. Then converts each partition, including images and annotations, to a .rec file. We use the validation data for selecting the best job in HPO training in the next section, and use the test data for numerically comparing all finetuned models. " ] }, { "cell_type": "code", "execution_count": 13, - "id": "90c94ae2", "metadata": { "execution": { "iopub.execute_input": "2022-08-05T08:11:13.050208Z", @@ -397,7 +366,6 @@ { "cell_type": "code", "execution_count": 2, - "id": "a3b00e8e", "metadata": {}, "outputs": [], "source": [ @@ -433,7 +401,6 @@ { "cell_type": "code", "execution_count": 12, - "id": "76de65c5", "metadata": { "papermill": { "duration": 1773.804904, @@ -516,7 +483,6 @@ }, { "cell_type": "markdown", - "id": "50836e20", "metadata": { "papermill": { "duration": 0.213955, @@ -533,13 +499,12 @@ "source": [ "### Visualize Training Progress\n", "\n", - "During training, the loss function is the sum of CrossEntropy loss and SmoothL1 loss. We will visualize the two losses on the training data as well as the mean Average Precision (mAP) on the validation data." + "During training, the loss function is the sum of CrossEntropy loss and SmoothL1 loss. We visualize the two losses on the training data as well as the mean Average Precision (mAP) on the validation data." ] }, { "cell_type": "code", - "execution_count": 41, - "id": "451353cf", + "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2022-08-05T08:40:58.683010Z", @@ -559,35 +524,7 @@ }, "tags": [] }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "Warning: No metrics called train:throughput found\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "All metrics: ['validation:mAP', 'train:progress', 'train:cross_entropy', 'train:smooth_l1']\n", - "Final validation:mAP = 0.0532\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAABCUAAAFNCAYAAAA3l9UCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAB+AElEQVR4nOzdd3hUZfrG8e+ThN5L6F16L6EIqGDv2KWpKIoo2HVXt6iru/pzXcUCFhBEpYnYULGLoChIQu8gvfcuJeT5/TEHNxspAZKcSXJ/rutcyZw5M3NPCG9mnnnO+5q7IyIiIiIiIiKS1WLCDiAiIiIiIiIiuZOKEiIiIiIiIiISChUlRERERERERCQUKkqIiIiIiIiISChUlBARERERERGRUKgoISIiIiIiIiKhUFFCopaZdTCz1akuzzWzDuk59iQe6zUz+/vJ3l5EJLvRuCciIiLRQEUJyTbcvYG7f3+q92NmPczsxzT33dvdnzzV+84oZva4mbmZtU6zv4eZHTKz3Wa208xmmNmlYeUUkfCY2XIzO/dkbx9t416YjvR3QUQkmuS0QvLJfqBoZkPN7J+ZkUnCo6KESJQxMwNuBLYGX9P62d0LA8WBwcBoMyuRdQlFJNqZWVzYGSB6cmQEM4sNO4OIZF+5vZAcfNhWM4PvM6+ZjQl+tn60jmqJfipKSKYzsz+b2Zg0+140s5fM7GYzm29mu8xsqZndfoz7+X0wN7MCQaV0m5nNA1qmOfZhM/s1uN95ZnZlsL8e8BpwetBtsD3Y/z9VVzO7zcyWmNlWMxtrZhVSXedm1tvMFpvZdjMbEBQSjpS5h5lNMrN+wbFLzaxtsH+VmW00s5vS3OwMoDxwN9DZzPIe6b7dPQUYAhQATjvaz01Ech4zeweoAnwSjGV/Csamnma2EvguOO49M1tvZjvMbKKZNUh1H7+Pe4c/sTKzB4JxaZ2Z3XyMxy9gZs+Z2Yrgvn8M9lVLm8PMYszsb8GxG83sbTMrFtxPfjMbZmZbgjFyqpmVDa7rEYyZu8xsmZl1S8fP5Zbgb8o2M/vSzKqmuu6IY/dx/i68ambjzGwP0NHM6pnZ98Ht55rZ5Wl+nq+Z2ddB5gmHHz94rOfSZB1rZvcd7zmJSM5nOaiAG4Ifge7A+rCDyMlTUUKywijgYjMrAr9/2nQdMALYCFwKFAVuBvqZWfN03OdjRN6InwZcAKR9Y/8rkTf3xYB/AMPMrLy7zwd6E3QbuHvxtHdsZmcDTwcZywMrgueQ2qVECiGNg+MuCG5bJXixWiXVsa2BWUCp4DmPCm5bk8gg2t/MCqc6/ibgE2B0cPmyI/0Agj9gtwK7gcVHOkZEciZ3vwFYCVwWdE4dHi/OAuoRjEnA50AtoAwwDRh+jLstR2TMrAj0BAZY0IVlZl3NbFaqY/8DtADaAiWBPwEpqa5PnaNHsHUEagCFgf7BcTcFj1mZyBjZG/jNzAoBLwEXuXuR4HFmHOtnYmadgL8AVwHxwA/AyDSH/WHsPs7fha7Av4AiwBQiY/NXRH6edwHDzaxOquO7AU8CpYO8h3/ebwFdzCwmyFoaOJfI3wQRyeGioJB8sUU+pNtlZmvM7ME09/OnVPdzRXD8Iot8OPeXVPeTz8xeMLO1wfaCmeVLdf0RP9Qzs4nBITOD5399qtuk6zkcibsfcPcX3P1H4NCJ3Faii4oSkuncfQWRF8NXBrvOBva6+2R3/8zdf/WICURe7J2Rjru9DviXu29191VEXrymfsz33H2tu6e4+7tE3rS3SmfkbsAQd5/m7vuBR4h8glYt1TH/5+7b3X0lMB5oGjzuSncvHuw/bJm7v+nuh4B3ibz4fsLd97v7V8ABIgUKzKwgcC0wwt0PAmP44ykcbYJP8tYDXYAr3X1HOp+biORsj7v7Hnf/DcDdh7j7rmAsexxocrhL4QgOEhmbDrr7OCIFzzrB/Yxw98YAwRvrW4B73H2Nux9y95+CxzhSjm7A8+6+1N13ExlTOweF1YNEihE1g/tJcvedwX2kAA3NrIC7r3P3ucd57r2Bp919vrsnA08BTVN3S3CUsfsYPnb3SUFnWlMiBZX/C14Ifwd8SmQcPuwzd58Y/Cz+SuRvR2V3/wXYAZwTHNcZ+N7dNxzn8UUkB4iCQvJg4PagyNuQoAiS6n7yB/fzKDCIyIdmLYi8Jv+7mVUPjv0r0IbIeNiEyGvrvwWPedQP9dz9zOD2TYLi77vHew6Su6goIVllBP994dY1uIyZXWRmk4OK6nbgYiKfMB1PBWBVqssrUl9pZjdaZBLI7cH9Nkzn/R6+79/vL3gRvYXIgHlY6haxvUReqB5N6hedh98opN13+PZXAsnAuODycOAiM4tPdfzkoPBR2t3buPs3x39KIpJL/D4umlmsmf2fRU5l2wksD6462li4JXgzf9jRxrbSRF7A/pqeHKQZU4Pv44CywDvAl8Co4FO3f5tZHnffA1xPpNCwzsw+M7O6x3g8gKrAi6nG/a2AcfJj95Gex6qgQJH6uVQ80vHB346twe0g0i3RPfi+O5HnLiK5W6YXklMdW9/Mirr7Nneflua6fwUfho0iMsa/GOSYC8wjUoCASJH5CXff6O6biHQj35DquuN9qJfu5yC5i4oSklXeAzqYWSUib7xHBO1e7xNpAy4btMyOI/Ii8njWEek4OOz30yWCT8UGAX2BUsH9zkl1v36c+15L5MXt4fsrROSTvDXpyHWqbiLyInmlma0n8nPLQ6SQIyKS2pHGstT7ugKdiJwmUAyoFuxPzxh7LJuBfRx7LpvUOf5nTCUyXicDG4IXov9w9/pETtG4lKA7zN2/dPfziHzitoDIuH4sq4h8Elg81VbA3X9Kx3M62t+FtM+j8uFTMFI9l9R/G37/uxScllcyuB3AMKCTmTUh8snoR+nIJSI5W1YUkgGuJvLB3wqLzHdzepr7OXzqw2/B16N9eHakInOFI113lA/1TuU5SA6mooRkiaCa+j3wJpHTGeYDeYF8wCYg2cwuAs5P512OBh4xsxJBoeOuVNcVIvJCchNAcH5aw1TXbwAq2VEmkCRyDvLNZtY0KJw8BUxx9+XpzHZSzKwikdbeS4m0xTUlUpl+hiOvwiEiudsGInM0HE0RYD+RF4UFiYxlp8z/O8nu82ZWIXghfXrq84rTGAncZ2bVgzfqTwHvunuymXU0s0YWmWtoJ5FPzVLMrKyZdQqKwvuJfHqWcpT7P+w1In8XGgCYWTEzuzadT+t4fxcgMqfEXuBPZpbHIrO8X8b/zjl0sZm1D+7nSSKdbasA3H01MJVIh8T7hz8ZFZFcI6xCMu4+1d07ETkt5CP+e/rIiTpSkXntka7L4g/1JJtTUUKy0ghSTezl7ruIrDAxGthGZDAem877+geRauwyIvNQ/N4G6+7zgOeAn4m80GwETEp12++AucB6M9uc9o6D0yH+TqSLYx2RTwM7pyeURSa63G3/O9Flet0AzHD3r9x9/eGNyHwZjc2s4XFuLyK5y9PA34JTFa45wvVvExkn1xBpv518sg9kZt3MLPWcDg8Cs4m8yd5KpHh6tNcUQ4iM0ROJjNn7+G8huRyRuXN2AvOBCcGxMcD9RF7kbiVy3vUdx8ro7h8GOUYFnzLOAS5K51M85t+F4P4PEClCXESkW+QV4EZ3X5DqsBFEJmLeSuR87O5p7uYtIn+TdOqGSO4TSiHZIstmdjOzYsEpGjs5fpH3aEYS+bsTb5EJex8l0gV2+Lpjfah3vOd/UoLJN/MHF/NaZFWnUy7kSNYy9+N1souIiIjIsZjZUGC1u//tGMecSeQFfFXXCzCRXMUiKwS9TGTFuX8CzwJ5Dp++EHSSDScyIfxWIh+QvQXUcvclqceYoFNrmLtXSnX/y4Fb3f0biyyh/Bd3bxB0bo0lshpcLLAQuM/df0x7P6kmIK5+uJhgZj8Cr7n7sODN/7+JTMoOkdOM/+Tu+4JjewMPASWAn4DeQZfY4eseI7KUfS8iK/Ad9Tkc4+f4+88h1W2qpjmsumdyh7NkLBUlRERERE7R8YoSZpaHyKkeM939iazMJiIiEs10+oaIiIhkC2b2WnCKXNrttbCzHYuZ1QO2E5m084VQw4iIiEQZdUqIiIiIiIhIVAjmMEp7SgZEVlgantV5JPOpKCEiIiIiIiIiodDpGyIiIiIiIiISiriwA2SU0qVLe7Vq1cKOISLyB0lJSZvdPT7sHFlBY7GIRCONwyIi4TvaWJxjihLVqlUjMTEx7BgiIn9gZivCzpBVNBaLSDTSOCwiEr6jjcU6fUNEREREREREQqGihIiIiIiIiIiEQkUJEREREREREQmFihIiIiIiIiIiEgoVJUREREREREQkFCpKiIiIiIiIiEgoVJQQEREREclEZnahmS00syVm9vARru9nZjOCbZGZbU91XRUz+8rM5pvZPDOrlpXZRUQyW1zYAUREREREcioziwUGAOcBq4GpZjbW3ecdPsbd70t1/F1As1R38TbwL3f/2swKAylZk1xEJGuoU0JEREREJPO0Apa4+1J3PwCMAjod4/guwEgAM6sPxLn71wDuvtvd92Z2YBGRrKSihIhIOmzZvZ+3flqOu4cdJUdZt+M3RkxZGXYMEZHMVBFYlery6mDfH5hZVaA68F2wqzaw3cw+MLPpZvZs0HmR9na9zCzRzBI3bdp0wgHfmbyCdTt+O+HbiYhkBBUlRESOY+ueA3R7YwpPjZvPii36gCojvTlpOX/5cDZfzl0fdhQRkWjQGRjj7oeCy3HAGcCDQEugBtAj7Y3cfaC7J7h7Qnx8/Ak94Iad+3jqs/mc328ioxNXqfguIllORQkRkWPYvvcA3d+YwtLNe3jjpgSqlS4UdqQc5YHza9O4UjEeHD2TZZv3hB1HRCQzrAEqp7pcKdh3JJ0JTt0IrAZmBKd+JAMfAc0zMlzZovn5/J4zqFeuKH8aM4tbhk5l/Y59GfkQIiLHpKKEiMhR7Nh7kO6Dp7Bk424G3tCCM2qd2KdPcnz54mJ5pVtzYmONO4YlsfdActiRREQy2lSglplVN7O8RAoPY9MeZGZ1gRLAz2luW9zMDv8BOhuYl/a2p6pa6UKM6tWGRy+tz89Lt3Bevwm8p64JEckiKkqIiBzBzn0HuXHIFBau38VrNzSnQ50yYUfKsSqVKMiLnZuxcMMu/vrhHL0IFpEcJehw6At8CcwHRrv7XDN7wswuT3VoZ2CUpxoEg9M4HgS+NbPZgAGDMiNnTIxxS/vqfHHPmdQtV4SH1DUhIllES4KKiKSxa99BbhryC3PX7uTV7i04u27ZsCPleGfVjufec2rT75tFNK9aghvaVA07kohIhnH3ccC4NPseTXP58aPc9mugcaaFS6Na6UK82+t0hv60nH9/uYDz+k3g0Uvrc02LSphZVsUQkVxEnRIiIqns3p9MjzenMnv1Dvp3bc559VWQyCp3nV2TDnXieeKTucxYtT3sOCIiudbhronPU3VN9HwrUV0TIpIpVJQQEQns2Z/MLW9OZcaq7bzUpRkXNiwXdqRcJSbGeOH6ppQtmp87hyWxdc+BsCOJiORq1YOuib9fWp+fft3Mef0mMCZptU6zE5EMpaKEiAiw90AytwydSuKKrbxwfVMublQ+7Ei5UvGCeXm1Wws27znAPaOmcyhFL3xFRMIUE2P0DLom6pQtwoPvzVTXhIhkKBUlRCTX++3AIW59K5Gpy7fS7/qmXNakQtiRcrVGlYrxxOUN+GHxZl74ZlHYcUREhKBr4vb/dk2cr64JEckgKkqISK627+Aher2TyM9Lt/DsNU3o1LRi2JEE6NyqCtclVOLl75bw3YINYccREREgNlXXRO1UXRMbdqprQkROnooSIpJr7U8+xO3vJPHD4s08c3Vjrm5RKexIksoTnRrSoEJR7h01g5Vb9oYdR0REAoe7Jv52ST0mLdnMec9P4H11TYjISVJRQkRypf3Jh7hj2DQmLNrE01c14rqEymFHkjTy54nl1W4tALhjeBL7Dh4KOZGIiBwWG2PcekYNvrg30jXxwHszuVVdEyJyEjK1KGFmF5rZQjNbYmYPH+H6fGb2bnD9FDOrFuyvZma/mdmMYHstM3OKSO5yIDmFviOm892CjfzzioZ0aVUl7EhyFFVKFaTf9U2Zu3Ynj348J+w4IiKSRuquiR/VNSEiJyHTihJmFgsMAC4C6gNdzKx+msN6AtvcvSbQD3gm1XW/unvTYOudWTlFJHc5eCiFu0dO5+t5G3iiUwO6t6kadiQ5jnPqleWus2syOnE1705dGXYcERFJ43DXxOf3nEEtdU2IyAnKzE6JVsASd1/q7geAUUCnNMd0At4Kvh8DnGNmlomZRCQXSz6Uwr2jZvDF3PU8eml9bjy9WtiRJJ3uPbc2Z9Qqzd8/nsvs1TvCjiMiIkdQI74wo9N0TXwwTV0TInJsmVmUqAisSnV5dbDviMe4ezKwAygVXFfdzKab2QQzOyMTc4pILpB8KIX7Rs/ks9nr+Nsl9bilffWwI8kJiI0xXuzcjNKF8nLH8CS27z0QdiQRETmCw10T44KuiftHz+S2txPZqK4JETmKaJ3och1Qxd2bAfcDI8ysaNqDzKyXmSWaWeKmTZuyPKSIZA+HUpwH35vJJzPX8vBFdbn1jBphR5KTULJQXgZ0a86Gnfu4790ZpKTokzcRkWh1WqquiR8Wb+ZcdU2IyFFkZlFiDZB6OvtKwb4jHmNmcUAxYIu773f3LQDungT8CtRO+wDuPtDdE9w9IT4+PhOegohkd4dSnIfGzOSjGWt56II69D7rtLAjySloVqUEj17WgPELN9F//JKw44iIyDGk7pqoWaawuiZE5IgysygxFahlZtXNLC/QGRib5pixwE3B99cA37m7m1l8MFEmZlYDqAUszcSsIpIDpaQ4j3wwiw+mreG+c2vTp2PNsCNJBujeugpXNqtIv28WMWGRuuRERKLdafGFea93W/56caRr4rx+E/lwuromRCQi04oSwRwRfYEvgfnAaHefa2ZPmNnlwWGDgVJmtoTIaRqHlw09E5hlZjOITIDZ2923ZlZWEcl5UlKcv340m9GJq7n77Jrcc26tsCNJBjEznrqyEXXKFuGeUdNZvW1v2JFEROQ4YmOM286MdE2cFl+I+96dyW1vJ6lrQkQyd04Jdx/n7rXd/TR3/1ew71F3Hxt8v8/dr3X3mu7eyt2XBvvfd/cGwXKgzd39k8zMKSI5i7vz6Ng5jPxlFXd2OI37zvvD2V+SzRXIG8ur3Vtw6JDTZ/g09icfCjuSiIikw/92TWzivH4T+Wj6GnVNiORi0TrRpYjISXF3/vHJPIZNXsntZ9bgoQvqoJWG/8jMhpjZRjObc5Tri5nZJ2Y208zmmtnNWZ3xeKqXLsR/rmvCzNU7eOKTeWHHERGRdErbNXHvuzPUNSGSi6koISI5hrvz5KfzGfrTcm5tX52HL6qrgsTRDQUuPMb1fYB57t4E6AA8F8wPFFUuaFCO28+qwfApK3k/aXXYcURE5ASoa0JEQEUJEckh3J2nxs1nyKRl9Ghbjb9eUk8FiWNw94nAsebqcaCIRX6IhYNjk7Mi24l66Pw6tKlRkr9+NJv563aGHUdERE5A6q6JGkHXRK93kti4S10TIrmFihIiku25O898sZBBPyzjhjZVeeyy+ipInLr+QD1gLTAbuMfdU450oJn1MrNEM0vctCnrV8OIi43h5S7NKVYgD72HJbHjt4NZnkFERE7NafGFGdO7LX+5uC4TFm3ivOfVNSGSW6goISLZmrvz3FeLeG3Cr3RtXYV/XN5ABYmMcQEwA6gANAX6m1nRIx3o7gPdPcHdE+Lj47MuYSrxRfIxoGtz1mz7jQffm0lKil7EiohkN7ExRq8zT2Pc3eqaEMlNVJQQkWztxW8X03/8Eq5PqMw/OzUkJkYFiQxyM/CBRywBlgF1Q850TAnVSvKXi+vx9bwNvD5xadhxRETkJNUsE+maeOSiSNfE+f0m8vEMdU2I5FQqSohItvXyt4t54ZvFXNOiEk9f1UgFiYy1EjgHwMzKAnWAqH+nf3O7alzauDzPfrmAn5ZsDjuOiIicpNgY4/azIl0T1UsX4p5RM7hdXRMiOZKKEiKSLb3y/RKe+3oRVzaryDNXN1ZB4gSZ2UjgZ6COma02s55m1tvMegeHPAm0NbPZwLfAn9096t/lmxnPXN2YGvGFuWvkdNbv0ItXEZHsLHXXxPfqmhDJkVSUEJFsZ+DEX/n3Fwu5vEkF/nNtE2JVkDhh7t7F3cu7ex53r+Tug939NXd/Lbh+rbuf7+6N3L2huw8LO3N6FcoXx2vdW7Dv4CHuHJ7EgeQjzs8pIiLZxH+7JtpTrZS6JkRyGhUlRCRbGfzjMp4at4BLGpfn+etUkJAjq1mmMM9c05hpK7fz1Lj5YccREZEMULNMEd6/Q10TIjmNihIikm0MnbSMJz+dx0UNy/HC9U2Ji9UQJkd3aeMK3NKuOkN/Ws7HM9aEHUdERDLAkbomeg9LYtOu/WFHE5GTpFf0IpItvPPzch7/ZB7n1y/LS12akUcFCUmHRy6uS0LVEjz8/mwWbdgVdhwREckgNcsUYUzv03n4orqMX7iJ8/pNUNeESDalV/UiEvVGTFnJ3z+ey7n1ytC/a3MVJCTd8sTGMKBbcwrli6P3sCR27TsYdiQREckgcbEx9A66JqoGXRN3DJumrgmRbEav7EUkqo2euoq/fDibjnXiGdCtOXnjNGzJiSlbND/9uzZjxZa9/Pn9WfoUTUQkh6lZpgjvB10T3y3cyPn9JjB25lqN9yLZhF7di0jUGpO0mj9/MIszapXm1e4tyBcXG3Ykyaba1CjFny6ow7jZ6xn847Kw44iISAY73DXx2V3tqVKqEHePnK6uCZFsQkUJEYlKH05fzUNjZtL2tFIMujGB/HlUkJBT0+vMGlzYoBxPf76AX5ZtDTuOiIhkglplI10Tf76wLt8tiHRNfKKuCZGopqKEiESdsTPX8sDombSuXpI3bmypgoRkCDPj39c2pkrJgvQZMY2NO7W+vYhIThQXG8MdHU7js7sjXRN3BV0Tm3era0IkGqkoISJR5bNZ67jv3RkkVCvJkB4tKZBXBQnJOEXz5+G17i3YvS+ZviOmc/BQStiRREQkk6TtmjjveXVNiEQjFSVEJGp8MWcdd4+aTrPKxXmzR0sK5o0LO5LkQHXKFeHpqxrxy/Kt/PuLBWHHERGRTPQ/XRMlC3LXyOncOVxdEyLRREUJEYkKX81dT98R02lSqRhDb2lFoXwqSEjmuaJZRW48vSqDfljGuNnrwo4jIiKZrFbZIrx/R1v+dGEdvp0f6Zr4dNbasGOJCCpKiEgU+Hb+BvqMmEaDipGCRGEVJCQL/PWSejStXJw/jZnFr5t2hx1HREQyWVxsDHd2qMmnQddE3xHTuXN4kromREKmooSIhGr8wo3cMWwadcsV5e1bWlE0f56wI0kukS8ulle6NSdvXAy930liz/7ksCOJiEgWqJ2qa+KbeRs5v99EdU2IhEhFCREJzcRFm7j9nSRqlS3MOz1bUayAChKStSoUL8BLnZvx66bdPPLBbE1+JiKSS6TumqhUooC6JkRCpKKEiIRi0pLN3PZ2IjVKF2JYz9YUL5g37EiSS7WvVZoHzq/D2JlrefvnFWHHERGRLFS7bBE+uKMtD13w366Jz2ZpriGRrKSihIhkuZ9/3ULPt6ZSrVQhht/amhKFVJCQcN1x1mmcW68M//xsHkkrtoUdR0REslBcbAx9Ov63a6LPiGncOTyJLeqaEMkSKkqISJb6ZdlWbhk6lcolCjL8ttaUKpwv7EgixMQYz13blPLFCtBHS8WJSAYzswvNbKGZLTGzh49wfT8zmxFsi8xse5rri5rZajPrn2Whc6G0XRPnqWtCJEuoKCEiWSZx+VZ6vPkLFYrnZ8RtbSitgoREkWIF8/Bq9+Zs23uAu0dOJ/lQStiRRCQHMLNYYABwEVAf6GJm9VMf4+73uXtTd28KvAx8kOZungQmZkHcXO9w18Qnd/23a6LP8GnqmhDJRCpKiEiWSFqxjZuG/EK5ovkZeVsb4ouoICHRp0GFYvzziob89OsWnv96UdhxRCRnaAUscfel7n4AGAV0OsbxXYCRhy+YWQugLPBVpqaU/1Gn3H+7Jr6at15dEyKZSEUJEcl0M1Ztp8eQXyhdJB8jbmtDmaL5w44kclTXJlSmS6vKvPL9r3w1d33YcUQk+6sIrEp1eXWw7w/MrCpQHfguuBwDPAc8mMkZ5Qh+n2virjOoWDzomhihrgmRjKaihIhkqtmrd3DD4CkUL5SHkbe1oVwxFSQk+j12WQMaVSzGA+/NZPnmPWHHEZHcozMwxt0PBZfvBMa5++pj3cjMeplZopklbtq0KdND5jZ1yhXhwzuDrom56zm/30TGzVbXhEhGUVFCRDLNnDU76D54CkXzRwoSFYoXCDuSSLrkzxPLK92aExtj9B6WxG8HDh3/RiIiR7YGqJzqcqVg35F0JtWpG8DpQF8zWw78B7jRzP4v7Y3cfaC7J7h7Qnx8fMaklv+RumuiQvEC3DlcXRMiGUVFCRHJFPPW7qT74CkUyhvLqF5tqFSiYNiRRE5I5ZIFeeH6pizcsIu/fjQbdw87kohkT1OBWmZW3czyEik8jE17kJnVBUoAPx/e5+7d3L2Ku1cjcgrH2+7+h9U7JOvUKVeED+5sy4Pn1/69a+JzdU2InBIVJUQkwy1cv4vug6eQPy6Wkb3aULmkChKSPXWoU4a7z67FB9PWMPKXVce/gYhIGu6eDPQFvgTmA6Pdfa6ZPWFml6c6tDMwylUBjXp5YmPoe3at37sm7gi6JrbuORB2NJFsKS7sACKSsyzesIuugyYTF2OM7NWGqqUKhR1J5JTcc04tZqzazuNj59KgQlGaVC4ediQRyWbcfRwwLs2+R9Ncfvw49zEUGJrB0eQUHO6aeH3Cr7z47WIm/7qFf17RkIsalQ87mki2ok4JEckwSzbupsugKcQEBYnqpVWQkOwvJsZ44fqmxBfJx53Dp7FNn4SJiEjgcNfEJ3e1p3zx/NwxfBp91TUhckJUlBCRDLF00266DpoMOCNva81p8YXDjiSSYUoUysur3Zuzadd+7nl3BodS1F0tIiL/VbdcUT68sx0PnFebL+eu5/x+E/hijuaaEEkPFSVE5JQt37yHLoMmk5zijLitDTXLFAk7kkiGa1ypOI9f3oCJizbx0reLw44jIiJRJk9sDHedU4uxfdtTrlh+eg+bxl0jp6trQuQ4VJQQkVOycsteugyazIHkFIbf2praZVWQkJyrS6vKXNOiEi99t5jxCzeGHUdERKJQvfL/7Zr4Ys46dU2IHIeKEiJy0lZtjRQk9h44xLBbW1OvfNGwI4lkKjPjyU4NqVuuKPeOmsGqrXvDjiQiIlEodddE2aLqmhA5lkwtSpjZhWa20MyWmNkf1lQ2s3xm9m5w/RQzq5bm+ipmttvMHszMnCJy4tZs/42ub0xm176DDL+1NQ0qFAs7kkiWKJA3lte6NyfFnTuHT2PfwUNhRxIRkShVr3xRPurTjvv/p2tifdixRKJKphUlzCwWGABcBNQHuphZ/TSH9QS2uXtNoB/wTJrrnwc+z6yMInJy1u34ja6DJrN9z0He6dmahhVVkJDcpWqpQvS7rimz1+zgH5/MDTuOiIhEsTyxMdz9P10TSdw1crpWcxIJZGanRCtgibsvdfcDwCigU5pjOgFvBd+PAc4xMwMwsyuAZYBe7YlEkQ0799F10BS27D7AWz1b0aRy8bAjiYTi3Ppl6dPxNEb+sorRiavCjiMiIlEubdfEeeqaEAEytyhREUj9Km11sO+Ix7h7MrADKGVmhYE/A//IxHwicoI27tpHl0GT2bhzH2/d0pLmVUqEHUkkVPefV4d2NUvx94/mMGfNjrDjiIhIlDtS18Td6pqQXC5aJ7p8HOjn7ruPdZCZ9TKzRDNL3LRpU9YkE8mlNu3aT9dBU1i/Yx9Db2lFi6olw44kErrYGOOlzs0oWSgvdwxPYsfeg2FHEhGRbCB118Tnc9ZxXr+JfDlXXROSO2VmUWINUDnV5UrBviMeY2ZxQDFgC9Aa+LeZLQfuBf5iZn3TPoC7D3T3BHdPiI+Pz/AnICIRW3bvp9sbk1m9bS9DerSkZTUVJEQOK1U4HwO6NWf9jn3cP3oGKSkediQREckGUndNlCmSj9vfSeKeUeqakNwnM4sSU4FaZlbdzPICnYGxaY4ZC9wUfH8N8J1HnOHu1dy9GvAC8JS798/ErCJyFFv3HKDbG1NYsWUvQ25qSZsapcKOJBJ1mlcpwd8vrc+3CzbyyvdLwo4jIiLZSL3yRfm4bzvuO7c2n81S14TkPplWlAjmiOgLfAnMB0a7+1wze8LMLg8OG0xkDoklwP3AH5YNFZHwbN97gO5vTGHp5j28cVMCbWuWDjuSSNS6oU1VOjWtwHNfL+KHxTqlUERE0i9PbAz3nKuuCcmd4jLzzt19HDAuzb5HU32/D7j2OPfxeKaEE5Fj2rH3IN0HT2HJxt0MvLEFZ9TSKVIix2JmPH1VI+av28k9o2bw6V3tqVC8QNixREQkG6lfIdI18cr4X3n5u8VMWrKFp65syPkNyoUdTSTTROtElyISop37DnLjkCksXL+L125oToc6ZcKOJJItFMwbx2vdW3AgOYU7hk9jf/KhsCOJiEg2k7Zrotc7Sdw7ajrb96prQnImFSVE5H/s2neQm4b8wty1O3mlWwvOrls27Egi2UqN+ML859rGzFy1nX9+Oj/sOCIikk0d7pq499xafDprHec+P5GvNNeE5EAqSojI73bvT6bHm1OZvXoH/bs257z6KkiInIwLG5an15k1eGfyCj6cvjrsOCIikk3liY3h3nNr83HfdsSra0JyKBUlRASAPfuTueXNqcxYtZ2XujTjwoY6d1HkVPzpgjq0ql6SRz6YzYL1O8OOIyIi2ViDCsX4uM9/uybO6zeRr+dtCDuWSIZQUUJE2HsgmVuGTiVxxVZeuL4pFzcqH3YkkWwvLjaG/l2bUTR/Hu4YNo2d+w6GHUlERLKxvHH/7ZooXTgft72dyH3vzlDXhGR7KkqI5HK/HTjErW8lMnX5Vvpd35TLmlQIO5JIjlGmSH4GdGvOyq17eei9mbh72JFERCSbO9w1cc85tfhk5lrO6zeRb9Q1IdmYihIiudi+g4fo9U4iPy/dwrPXNKFT04phR5IsYmZDzGyjmc05xjEdzGyGmc01swlZmS8naVmtJI9cVJcv525g4MSlYccREZEcIG9cDPed99+uiVvVNSHZmIoSIrnU/uRD3P5OEj8s3swzVzfm6haVwo4kWWsocOHRrjSz4sArwOXu3gC4Nmti5Uw921fnkkbleeaLBfz865aw44iISA6hrgnJCVSUEMmF9icf4o5h05iwaBNPX9WI6xIqhx1Jspi7TwS2HuOQrsAH7r4yOH5jlgTLocyMZ65pTLXShbhr5DQ27NwXdiQREckhDndNfNSnHaUK5eXWtxO5/90Z7NiruYwke1BRQiSXOZCcQt8R0/luwUb+eUVDurSqEnYkOQVmdpeZlciEu64NlDCz780sycxuzITHyFUK54vj9e4t2HvgEH2GT+PgoZSwI4mISA7SsGIxxvZtz93n1GLszLWc12+CuiYkW1BRQiQXOXgohbtHTufreRt4olMDurepGnYkOXVlgalmNtrMLjQzy6D7jQNaAJcAFwB/N7PaRzrQzHqZWaKZJW7atCmDHj5nqlW2CP93dWMSV2zj6XELwo4jIiI5TN64GO4PuiZKqmtCsgkVJURyieRDKdw7agZfzF3Po5fW58bTq4UdSTKAu/8NqAUMBnoAi83sKTM77RTvejXwpbvvcffNwESgyVEyDHT3BHdPiI+PP8WHzfkub1KBHm2rMWTSMj6dtTbsOCIikgMdqWvi2/nqmpDopKKESC6QfCiF+0bP5LPZ6/jbJfW4pX31sCNJBvLIOpPrgy0ZKAGMMbN/n8Ldfgy0N7M4MysItAbmn3JYAeAvF9ejRdUS/GnMLJZs3BV2HBERyYHSdk30fCuR+0era0Kij4oSIjncoRTnwfdm8snMtTx8UV1uPaNG2JEkA5nZPWaWBPwbmAQ0cvc7iJx6cfUxbjcS+BmoY2arzaynmfU2s94A7j4f+AKYBfwCvOHuR10+VE5M3rgYBnRtTsG8sdz+ThK79yeHHUlERHKo37smzq7JxzPWcv4LE/hugbomJHqoKCGSgx1KcR4aM5OPZqzloQvq0PusU+3olyhUErjK3S9w9/fc/SCAu6cAlx7tRu7exd3Lu3sed6/k7oPd/TV3fy3VMc+6e313b+juL2T6M8llyhXLz0tdmrFs8x7+/P4sIg0vIiIiGS9vXAz3n1+Hj/u0o0TBvNwyVF0TEj1UlBDJoVJSnEc+mMUH09Zw37m16dOxZtiRJBO4+2NAKTO7O1iJo3mq63S6RZRre1ppHrqgLp/NWseQScvDjiMiIjmcuiYkGqkoIZIDpaQ4f/1oNqMTV3P32TW559xaYUeSTGJmfwfeAkoBpYE3zexv4aaSE9H7rBqcX78sT4+bz9TlW8OOIyIiOdyRuiYeGD1TXRMSGhUlRHIYd+fRsXMY+csq7uxwGvedd8RVHCXn6A60dPfHgq6JNsANIWeSE2Bm/Oe6JlQqUYA+w6excde+sCOJiEgucLhr4q6za/LRjDXqmpDQqCghkoO4O//4ZB7DJq/k9jNr8NAFdTCzsGNJ5loL5E91OR+wJqQscpKK5s/Dq91bsHPfQe4aMZ3kQylhRxKRdDCzwmFnEDkVeeNieOD8Onx0ZzuKF4h0TTz43kx2/KauCck6KkqI5BDuzpOfzmfoT8vp2b46D19UVwWJ3GEHMNfMhprZm8AcYLuZvWRmL4WcTU5AvfJFeerKRkxZtpVnv1oYdhwRSZ95YQcQyQiNKhVj7F3tuOvsmnw4fQ3n95vA+AUbw44luUTc0a4wszLAX4CawGzgaXffmVXBRCT93J2nP1/AkEnL6NG2Gn+7pJ4KErnHh8F22Pch5ZAMcFXzSkxbuY3XJyylWeUSXNiwXNiRRHI9M7v/aFcB6pSQHCNfXCwPnF+H8+uX48H3ZnLz0Knc3K4aj15aX68rJVMdtSgBvA0kAS8TWVbuJaBHFmQSkRPg7jzzxUIGTlzKDW2q8thl+sORm7j7W2aWFzg8ecjCw8uCSvb090vrM3vNTh58bya1yxamRrze84iE7CngWSD5CNep61hynMNdE0+PW8Cbk5ZTpkh+7uigZeUl8xyrKFHe3f8afP+lmU3LikAikn7uznNfLeK1Cb/StXUV/nF5AxUkchkz60Bk9Y3lRD61q2xmN7n7xBBjySnIFxfLK92ac+lLP3DHsGl82KctBfMe68+1iGSyacBH7p6U9gozuzWEPCKZLl9cLI9dVp+tew7wzBcLqFqqIBc3Kh92LMmhjlndNbMSZlbSzEoCsWkui0jIXvx2Mf3HL+H6hMr8s1NDYmJUkMiFngPOd/ez3P1M4AKgX8iZ5BRVLF6AFzs3Y9HGXfz1wzm4e9iRRHKzm4GVR7kuISuDiGQlM+Pf1zSmeZXi3PfuDGas2h52JMmhjlWUKEbk9I3DW1EileIkIDHzo4nIsbz87WJe+GYx17SoxNNXNVJBIvfK4+6/z4ro7ouAPCHmkQxyZu147j+3Nh9OX8OwySvCjiOSa7n7QnffdJTrtH6i5Gj588Qy6MYEyhTNx61vJbJ6296wI0kOdNR+UHevloU5ROQEvPL9Ep77ehFXNqvIM1c3VkEid0syszeAYcHlbqhwnGP06ViT6au288Sn82hYsRjNqpQIO5JIrmNmnwBHbVdy98uzMI5IlitVOB9v9mjJla/8RM+hiYy543SK5NfnH5JxTmhyHjM7zcz+bmZzMyuQiBzbwIm/8u8vFnJ5kwr859omxKogkdv1JrIk3d3BNg+4I9REkmFiYox+1zWlbNH83Dl8Glt27w87kkhu9B8ip8odbRPJ8WqWKcJr3Vvw66bd9B0xneRDKWFHkhzkuEUJM6tgZveZ2VRgbnCbzpmeTET+YPCPy3hq3AIuaVye569TQSK3M7NYYKa7P+/uVwVbP3fXO9ccpFjBPLzWvQVb9hzgnlEzOJSi+SVEspK7TzjWdvg4M3v/aPdhZhea2UIzW2JmDx/h+n5mNiPYFpnZ9mB/UzP72czmmtksM7s+U56kSDq0q1maJ69oyIRFm/jHJ/M035FkmKMWJcysl5mNJ7LmfSmgJ7DO3f/h7rOzKJ+IBIZOWsaTn87joobleOH6psTFahWy3M7dDwELzaxK2FkkczWsWIx/dmrIj0s20+/rRWHHEZEjq3GknUEBeQBwEVAf6GJm9VMf4+73uXtTd28KvAx8EFy1F7jR3RsAFwIvmFnxzIkvcnxdWlXh9jNr8M7kFbw5aXnYcSSHONYaY/2Bn4Gu7p4IYGYqh4mE4J2fl/P4J/M4v35ZXurSjDwqSMh/lQDmmtkvwJ7DO3WOc85zXcvKJK3YRv/xS2hWpTjn1CsbdiQR+V9He53cClji7ksBzGwU0InI6XZH0gV4DH6fvJjg+7VmthGIB7ZnUGaRE/bnC+uyfMsenvxsHlVKFuTc+vp7JKfmWO9sygMjgeeCdrMn0YzuIlluxJSV/P3juZxbrwz9uzZXQULS+jtwKfAEOsc5x/tHpwY0rFiU+96dwcotmgFdJJuoCKxKdXl1sO8PzKwqUB347gjXtQLyAr9mQkaRdIuJMV64vhmNKhbj7lHTmbt2R9iRJJs76rsbd9/i7q+5+1nAOUQqshvMbL6ZPZVVAUVys9FTV/GXD2fTsU48A7o1J2+cChLyBxcf4fzmi8MOJZkjf55YXu3WAjOj97Ak9h08FHYkEfmvjJjoqTMwJjg97793bFYeeAe42d3/MMNgcNp1opklbtp0xNVLRTJUgbyxvHFjAsUK5KHn0ETW79gXdiTJxtL1DsfdV7v7c+6eAFwO6LdOJJONSVrNnz+YxRm1SvNq9xbki4sNO5JEp/OOsO+iLE8hWaZyyYK8cH1T5q3byd8/mqOJxkSix5+Psn8NUDnV5UrBviPpTKRT+XdmVhT4DPiru08+0o3cfaC7J7h7Qnx8/ImlFjlJZYrmZ0iPluzad5Ceb01lz/7ksCNJNpWe1TdizexyM7vbzO4HLgN2Z340kdzrw+mreWjMTNqeVopBNyaQP48KEvK/zOwOM5sN1AlmZD+8LQM0GXEO17FuGe4+uybvJa3m3amrjn8DETllZtbOzL4OVsdYambLzGzp4evd/auj3HQqUMvMqptZXiKFh7FHuP+6ROYJ+jnVvrzAh8Db7j4mI5+PSEaoV74o/bs2Z/66nVohSk5aejolPgF6EFmBo0iwFc7ETCK52tiZa3lg9ExaVy/JGze2VEFCjmYEkSLx2ODr4a2Fu3cLM5hkjXvOrc0ZtUrz6Ni5zF6t83lFssBg4HmgPdASSAi+HpO7JwN9gS+B+cBod59rZk+YWepJiTsDo/x/25+uA84EeqRaMrRphjwbkQzSsW4ZHrusAd/M38D/fT4/7DiSDR1r9Y3DKrl740xPIiJ8Nmsd9707g4RqJRnSoyUF8qogIUfm7juAHUSWlosFyhIZ0wubWWF3XxlqQMl0sTHGi52bcdnLP9J7WBKf3d2e4gXzhh1LJCfb4e6fn8wN3X0cMC7NvkfTXH78CLcbBgw7mccUyUo3ta3Gss17GPTDMqqVLkS31lXDjiTZSHo6JT43s/MzPYlILvfFnHXcPWo6zSoX580eLSmYNz01Q8ntzKwvsAH4msg5x58Bn4YaSrJMyUJ5eaVbczbt2s+9784gRW2zIhnOzJqbWXNgvJk9a2anH94X7BcR4G+X1KNjnXge/XguExdpwlVJv/S865kMfGhmMcBBIjMLu7sXzdRkIrnIV3PX03fEdJpUKsbQW1pRKJ8KEpJu9wJ13H1L2EEkHE0qF+fRy+rzt4/m8PJ3S7jn3FphRxLJadIus5yQ6nsHzs7CLCJRKy42hpe7NueaV3+iz/BpvH9nW2qXLRJ2LMkG0tMp8TxwOlDQ3Yu6exEVJEQyzrfzN9BnxDQaVIwUJAqrICEnZhWR0zgkF+vWugpXNa/IC98u4vuFG8OOI5KjuHtHd+8I9Dz8fap9t4adTySaFM4Xx5AeLcmfN5ab35zKpl37w44k2UB6ihKrgDlpJt0RkQwwfuFG7hg2jbrlivL2La0omj9P2JEk+1kKfG9mj5jZ/Ye3sENJ1jIz/nVFI+qULcK9785g9ba9YUcSyYmOtPrFe1meQiTKVShegME3JbBlz35uezuRfQcPhR1Jolx6ihIn/YLXzC40s4VmtsTMHj7C9fnM7N3g+ilmVi3Y3yrVDMMzzezKE3pWItnAxEWbuP2dJGqVLcw7PVtRrIAKEnJSVhKZTyIv/10hSb2SuVCBvLG81r0Fhw45dw6fpheBIhnEzOqa2dVAMTO7KtXWA8gfcjyRqNS4UnFeuL4ZM1dv54HRMzXnkRxTevrElwVb3mBLl2A2+AHAecBqYKqZjXX3eakO6wlsc/eaZtYZeAa4HpgDJLh7spmVB2aa2SfBkkoi2d6kJZu57e1EapQuxLCerTVjvpw0d/8HgJkVdHd9PJ7LVStdiOeua0Kvd5J44tN5PHVlo7AjieQEdYBLgeJEll4+bBdwWxiBRLKDCxuW45GL6vLUuAVUL12IBy+oE3YkiVLHLUocfsF7EloBS9x9KYCZjQI6AamLEp2Ax4PvxwD9zczSvLDOT2QSIZEc4edft9DzralUK1WI4be2pkQhFSTk5JnZ6cBgoDBQxcyaALe7+53hJpOwnN+gHHd0OI1Xv/+V5lVKcE2LSmFHEsnW3P1j4GMzO93dfw47j0h2ctsZNVi2eQ/9xy+haqmCXJtQOexIEoXSc/oGZtbrWJePoiKR+SgOWx3sO+IxQRfEDqBU8BitzWwuMBvorS4JyQl+WbaVW4ZOpXKJggy/rTWlCucLO5Jkfy8AFwBbANx9JnBmmIEkfA+cV5vTa5Tirx/OZt7anWHHEckpVpnZh2a2MdjeNzNV/USOwcx4olND2tcszV8+nM3Pv2qxMPmjdBUliCwDeqzLGc7dp7h7A6Al8IiZ/eGcPTPrZWaJZpa4aZPWwpXolrh8Kz3e/IUKxfMz/LbWlFZBQjKIu69Ks0uTCeRykWXZmlG8YB7uGJ7Ejt8Ohh1JJCd4ExgLVAi2T4J9InIMeWJjGNCtOVVLFaL3sCSWbtoddiSJMukqSrj768e6fBRrgNT9OZWCfUc8xszigGIEn/aleqz5wG6g4RFyDXT3BHdPiI+PT0ckkXBMW7mNHm9OpVzR/Iy8rQ1limheLMkwq8ysLeBmlsfMHgTmhx1Kwle6cD5e6dacNdt+44HRMzTJmMipK+Pub7p7crANBfQCVCQdihXIw5s9WhIXY9wydCrb9hwIO5JEkeMWJYIVMrqa2V/M7NHDWzrueypQy8yqm1leoDOR6nJqY4Gbgu+vAb5zdw9uExc8flWgLrA8nc9JJKrMWLWdmwb/QqnCeRlxWxvKFFVBQjJUb6APkdPh1gBNg8sitKhakr9dUo9v5m/k1Qm/hh1HJLvbbGbdzSw22LqT5sM0ETm6yiULMvDGFqzdsY/b30lif7IaOyUiPZ0SHxOZkDIZ2JNqO6ZgDoi+wJdEPrUb7e5zzewJM7s8OGwwUMrMlgD3A4eXDW1PZMWNGcCHwJ3uvjndz0okSsxevYMbBk+heKE8jLytDeWKqSAhGcvdN7t7N3cv6+5l3L27u//+ItnMHgkzn4TvprbVuLxJBZ77aiGTluhPqcgpuAW4DlgfbNcAN4eaSCSbaVG1JP+5tgm/LN/KI+/Pxl1dfJK+JUErufuFJ3Pn7j4OGJdm36Opvt8HXHuE270DvHMyjykSLeas2UH3wVMomj9SkKhQvEDYkSR3uhZ4OuwQEh4z4+mrGjF/3U7uHjmdT+9uT/liGo9ETpS7rwAuP+6BInJMlzepwIrNe3ju60VUL12Iu86pFXYkCVl6OiV+MjMtdC5yAuat3Un3wVMolDeWUb3aUKlEwbAjSe6V6RMTS/QrlC+OV7u3YN/BQ9w5fBoHklPCjiSS7ZhZJa2+IZIx+p5dk6uaVeS5rxfx8Yy00w5KbpOeokR7IMnMFprZLDObbWazMjuYSHa1cP0uug+eQv64WEb2akPlkipISKjUFykA1CxTmGevbcL0ldv512fzwo4jkh1p9Q2RDGJmPH11I1pVK8lDY2aRtGJr2JEkROkpSlwE1ALOBy4DLg2+ikgaizfsouugycTFGCN7taFqqUJhRxJRp4T87uJG5bm1fXXe+nmFPpkSOXHxWn1DJOPki4vl9RtaUKFYfnq9ncTKLXvDjiQhOW5RIjh/rjiRQsRlQPFgn4iksmTjbroMmkJMUJCoXloFCYkK74UdQKLLny+qS6tqJXn4/dks2rAr7Dgi2ckWrb4hkrFKFMrLkB4tSU5xbnlrKjt+Oxh2JAlBepYEvQcYDpQJtmFmdldmBxPJTpZu2k3XQZMBZ+RtrTktvnDYkSSXMLN/m1lRM8tjZt+a2abghTIA7v5UmPkk+uSJjaF/12YUzh9H73eS2LVPLwBF0in16hvr0OobIhmiRnxhXr+hBSu27OHO4UkcPKR5j3Kb9Jy+0RNo7e6PBitntAFuy9xYItnH8s176DJoMskpzojb2lCzTJGwI0nucr677yRyat1yoCbwUKiJJOqVKZqf/l2asWLrXv40ZpaWZBNJB3df4e6Xu3t8sATzFe6+MuxcIjlBmxqlePqqxkxasoW/fzRHf5dymfQUJQw4lOryIXSOsggAK7fspcugyRxITmH4ra2pXVYFCclyh5d2vgR4z913hBlGso/WNUrx8IV1+XzOet74YVnYcUSinplVN7PnzewDMxt7eAs7l0hOcU2LSvTtWJNRU1cx6IelYceRLBR3/EN4E5hiZh8Gl68ABmdaIpFsYtXWSEFi74FDjLitNfXKFw07kuROn5rZAuA34A4ziwf2hZxJsolbz6jOtJXb+L8vFtC4UjFa1ygVdiSRaPYRkdfAnwDqLxfJBPefV5tlW/bw9OcLqFKyEBc2LBd2JMkC6Zno8nki58ttDbab3f2FTM4lEtXWbP+Nrm9MZte+gwy/tTUNKhQLO5LkUu7+MNAWSHD3g8AeoFO4qSS7MDP+fU1jqpYsSN+R09m4U/UskWPY5+4vuft4d59weAs7lEhOEhNjPHdtE5pUKs69705n1urtYUeSLHDUooSZFQ2+liRynvKwYFsR7BPJldbt+I2ugyazfc9B3unZmoYVVZCQ8JjZtcBBdz9kZn8jMk5XCDmWZCNF8ufhtRtasHtfMn1GTNMEYyJH96KZPWZmp5tZ88Nb2KFEcpr8eWIZdGMCpQvno+dbiazd/lvYkSSTHatTYkTwNQlITLUdviyS62zYuY+ug6awZfcB3urZiiaVi4cdSeTv7r7LzNoD5xJpLX415EySzdQuW4T/u7oRU5dv45nPF4QdRyRaNSIy2fv/Ac8F239CTSSSQ8UXyceQHi3Zd+AQtwydyu79yWFHkkx01KKEu18afK3u7jVSbdXdvUbWRRSJDht37aPLoMls3LmPt25pSfMqJcKOJAL/nYj4EmCgu38G5D3ejcxsiJltNLM5xzmupZklm9k1GZBVolinphW56fSqvPHjMsbNXhd2HJFodC1Qw93PcveOwXZ22KFEcqraZYswoFtzFm/czV0jppGsTr4c67hzSpjZt+nZJ5KTbd69n66DprB+xz6G3tKKFlV1BpNEjTVm9jpwPTDOzPKRvpWVhgIXHusAM4sFngG+OtWQkj389ZL6NKtSnIfem8mSjbvDjiMSbeYAxcMOIZKbnFk7nic6NWD8wk3887P5YceRTHKsOSXyB3NHlDazEmZWMtiqARWzLKFIyLbs3k/XQZNZvW0vQ3q0pGU1FSQkqlwHfAlc4O7bgZLAQ8e7kbtPJDJ58bHcBbwPbDzFjJJN5I2L4ZVuzcmfJ5Y7hiWxR+2yIqkVBxaY2ZdaElQk63RrXZVb21dn6E/LGTpJS1jnRMdaEvR24F4iE6YlARbs3wn0z9xYItFh654DdHtjCiu27OXNHi1po+XyJMq4+14z+xW4wMwuAH5w91PubDCzisCVQEeg5anen2Qf5YsV4KUuzbhh8BQe/mA2L3Vuipkd/4YiOd9jYQcQya0eubgeK7bu5YlP51G1VCE61i0TdiTJQMeaU+JFd68OPJhqLonq7t7E3VWUkBxv+94DdH9jCks37+GNmxJoW7N02JFE/sDM7gGGA2WCbZiZ3ZUBd/0C8Gd3P+4JnGbWy8wSzSxx06ZNGfDQErZ2NUvzwPl1+GTmWt76aXnYcUSiRSKRwu8EYB1QDPgp3EgiuUNsjPFi56bUK1+UviOmMW/tzrAjSQY67nnH7v6ymTU0s+vM7MbDW1aEEwnLjr0H6T54Cks27mbgDS04o1Z82JFEjqYn0NrdH3X3R4E2RGaHP1UJwCgzWw5cA7xiZlcc6UB3H+juCe6eEB+v/ys5xR1nnca59cryz8/mk7TieGf6iOQKE4H8QSfZV8ANRObnEZEsUDBvHINvakmR/Hno+dZUNu7cF3YkySDpmejyMeDlYOsI/Bu4PJNziYRm576D3DhkCgvX7+K1G5rToY7awySqGf9dgYPg+1PutQ8646q5ezVgDHCnu390qvcr2UdMjPHcdU2oWKIAdw6fxubd+8OOJBI2c/e9wFXAK+5+LdAw5EwiuUq5YvkZ3COBHb8dpOdbiew9oLmPcoL0zNB+DXAOsN7dbwaaEGlXE8lx1m7/jS4DJzN37U5e6daCs+uWDTuSyPG8CUwxs8fN7HFgMjD4eDcys5HAz0AdM1ttZj3NrLeZ9c7cuJKdFCuQh1e7tWD73oPcNWK6lmOT3M7M7HSgG/BZsC89r6VFJAM1qFCMl7s0Y+7aHdz37gxSUjzsSHKK0jOQ/hacU5xsZkWJzMJeOXNjiWS9aSu3cXn/SazYspdBNyZwXn0VJCS6mVkMkSLEzURW0tgK3OzuLxzvtu7exd3Lu3sed6/k7oPd/TV3f+0Ix/Zw9zEZnV+yh/oVivKvKxvx89ItPPf1orDjiITpHuAR4EN3n2tmNYDxIWcSyZXOqVeWv11Sny/nbuCZLxaEHUdO0bFW3zgs0cyKA4OIrMKxm8inayI5xvtJq3nkg9mUL56fkbe1plbZImFHEjkud08xswHu3gyYFnYeybmuaVGJaSu38er3v9KscnHOb1Au7EgiWS5YSnliqstLgbvDSySSu93crhrLNu/h9YlLqVa6EF1aVQk7kpyk9Ex0eae7bw8+PTsPuCk4jUMk2zuU4jw1bj4PvDeTFlVL8NGd7VSQkOzmWzO72rRmo2SyRy+tT+NKxXhg9EyWb94TdhyRUJhZr2NdFpGsY2Y8dll9zqodz98/msOPizeHHUlO0lGLEmbWPO0GlATigu9FsrWd+w5y61tTGThxKTeeXpW3e7aiRKG8YccSOVG3A+8B+81sp5ntMjOtkyUZLn+eWF7p1pzYWKP3sCR+O3Do+DcSyXnSFoBVEBYJUVxsDP27NuO0+MLcMTyJxRt2hR1JTsKxOiWeC7YBwBRgIJFTOKYE+0SyreWb93DVKz/xw+LN/POKhjzRqSF5YjVXlWQ/7l7E3WPcPa+7Fw0uFw07l+RMlUoU5MXOzVi4YRd//XA27ppcTHIXd3/9WJdFJOsVyZ+HwT0SyBcXyy1vTdVqUdnQUd+FuXtHd+8IrAOaB2vQtwCaAWuyKqBIRvtpyWY6DZjE5t37ebtnK7q3qRp2JJGTZmZXmlmxVJeLm9kVIUaSHO6s2vHce05tPpi+huFTVoYdRyTLmFk+M+tqZn8xs0cPb2HnEpFI0fyNmxLYtGs/vd5OZN9BdfNlJ+n5aLiOu88+fMHd5wD1Mi+SSOZwd97+eTk3DPmFskXzMbZPe9qeVjrsWCKn6jF333H4grtvBx4LL47kBnedXZMOdeJ54pN5zFi1Pew4IlnlY6ATkAzsSbWJSBRoWrk4/a5ryrSV23lozCx182Uj6Vl9Y5aZvQEMCy53A2ZlXiSRjHfwUAqPjZ3LiCkrObdeGfpd35Qi+fOEHUskIxypuJyesV3kpMXEGC9c35RLXvqRO4cl8endZ1BSc/JIzlfJ3S8MO4SIHN1Fjcrz5wvr8swXC6heqiD3n18n7EiSDunplLgZmEtkbeZ7gHnBPpFsYeueA3R/Ywojpqzkjg6n8foNCSpISE6SaGbPm9lpwfY8keWbRTJV8YJ5ea17CzbvOcA9o6ZzKEWfSEmO95OZNQo7hIgcW++zanB9QmVe+m4J7yetDjuOpEN6lgTd5+793P3KYOvn7vuyIpzIqVq4fhedBvzI9FXb6Xd9E/58YV1iYzRRtuQodwEHgHeBUcA+oE+oiSTXaFSpGE9c3oAfFm/mxW8WhR1HJLO1B5LMbKGZzTKz2Wam7mGRKGNmPHlFQ9qeVoqHP5jFlKVbwo4kx3HUFl8zG+3u15nZbOAPH3+4e+NMTSZyir6et4F7R02nUL443u3VhmZVSoQdSSTDufse4OGwc0judX3LyiSt2MZL3y2haZXinF23bNiRRDLLRSd7QzO7EHgRiAXecPf/S3N9P6BjcLEgUMbdiwfX3QT8Lbjun+7+1snmEMkt8sbF8Gq3Flz56iRuH5bER3e2o1rpQmHHkqM4VqfEPcHXS4HLjrCJRCV355Xvl9DrnUROK1OYsX3bqyAhOZqZ9TrWZZHMdPgTqfrli3LfuzNZtXVv2JFEMpSZHV5meddRtuPdPhYYQKSoUR/oYmb1Ux/j7ve5e1N3bwq8DHwQ3LYkkcmLWwOtgMfMTC9qRNKhWME8vNmjJQbcMnQq2/ceCDuSHMWxlgRdF3xdcaQt6yKKpN++g4e4990Z/PuLhVzauAKjbz+dcsXyhx1LJLOlPSdJ5yhJlsqfJ5bXurfA3ek9LElLsUlOMyL4mgQkBl+TUl0+nlbAEndf6u4HiJxq1+kYx3cBRgbfXwB87e5b3X0b8DWgyTZF0qlqqUIMvDGB1dt+4/Z3kjiQnBJ2JDmCoxYlzGyXme08wrbLzHZmZUiR9Niwcx/Xv/4zH89Yy0MX1OGlzk3Jnyc27Fgimc7dXz/WZZGsUKVUQfpd35S5a3fy2Mdzw44jkmHc/dLga3V3rxF8PbzVSMddVARWpbq8Otj3B2ZWFagOfHcitzWzXmaWaGaJmzZtSkckkdyjZbWSPHttY6Ys28ojH8zWUqFR6KhzSrh7kawMInIqZq7aTq93Etm1L5nXb2jBBQ3KhR1JJEuYWT7gaqAaqcZ0d38irEySe51Tryx9O9ak//glNK9anOtbVgk7kkiGCk6dqAX83obp7hMz8CE6A2Pc/YTajdx9IDAQICEhQe+4RNLo1LQiyzbv4YVvFlMjvhB9OtYMO5Kkku617M2sDP87AK/MlEQiJ+jjGWv405hZlC6cj/fvaEu98kWPfyORnONjYAeRNuL9IWcR4b7zajNj1Xb+/vFcGlQoRsOKxcKOJJIhzOxWInOuVQJmAG2An4Gzj3PTNUDlVJcrBfuOpDP/u4LSGqBDmtt+n87IIpLKPefUYtnmPTz75UKqlirIpY0rhB1JAsddEtTMLjezxcAyYAKwHPg8k3OJHFdKivPslwu4Z9QMmlQqzti+7VSQkNyokrtf7+7/dvfnDm9hh5LcKzbGeLFzU0oXykvvYUmaWExyknuAlsAKd+8INAO2p+N2U4FaZlbdzPISKTyMTXuQmdUFShApdBz2JXC+mZUIujTOD/aJyAkyM565ujEJVUtw/+iZTFu5LexIEjhuUQJ4kkgleJG7VwfOASZnaiqR49i9P5nbhyUxYPyvdGlVmWG3tqZU4XxhxxIJw09m1ijsECKplSqcjwHdmrNh5z7ue3cGKSnqJpccYZ+774PIqXPuvgCoc7wbuXsy0JdIMWE+MNrd55rZE2Z2eapDOwOjPNUJ7+6+lchr8anB9kSwT0ROQv48sQy8MYFyRfPT6+1ErRgVJdJTlDjo7luAGDOLcffxQEIm5xI5qlVb93L1Kz/x3YKNPH5ZfZ66shF549LzqyySI7UHksxsoZnNMrPZZjYr7FAizaqU4NFL6zN+4SYGjF8SdhyRjLDazIoDHwFfm9nHQLpWpHP3ce5e291Pc/d/BfsedfexqY553N0fPsJth7h7zWB7M0OeiUguVrJQXob0aMmB5BRuGTqVnfsOhh0p10vPnBLbzaww8AMw3Mw2AnsyN5bIkU1euoU7hiVxKMUZenNLzqgVH3YkkbBdFHYAkaPp3qYq01Zu5/lvFtG0SnGN2ZKtufuVwbePm9l4oBjwRYiRROQk1SxTmNe6t+DGIb/QZ/g0hvRoSZ5YfcgZlvT85A8PuvcQGXh/BS7LzFAiRzJiykq6vzGFEoXy8lGfdnpxK7mamR2eQGXXUTaR0JkZ/7qyIbXLFOHukdNZs/23sCOJnBQzizWzBYcvu/sEdx/r7po0RSSbaluzNE9d1YgfFm/msbFztVRoiNJTlIgDviIy028R4N3gdI7jMrMLg5biJWb2h3Y0M8tnZu8G108xs2rB/vPMLCloQ04ys+PNaiw5WPKhFB77eA5/+XA27WqW5sM721EjvnDYsUTCNiL4mgQkBl+TUl0WiQoF88bx2g0tSD7k3Dl8GvuTT2ilQ5GoECzRudDMtM6tSA5yXUJl7uhwGiOmrGTwj8vCjpNrHbco4e7/cPcGRJYnKg9MMLNvjnc7M4sFBhBpLa4PdDGz+mkO6wlsc/eaQD/gmWD/ZuAyd28E3AS8k87nIznM9r0HuOnNX3jr5xXc2r46Q3q0pFiBPGHHEgmdu18afK3u7jWCr4e3GmHnE0mteulCPHttE2au2s6Tn84LO47IySoBzDWzb81s7OEt7FAicmoeOr8OFzUsx7/GzeeruevDjpMrnciJMxuB9cAWoEw6jm8FLHH3pUFr2yigU5pjOgFvBd+PAc4xM3P36e6+Ntg/FyhgZlpaIZdZsnEXVwyYxNRl23j2msb87dL6xMZY2LFEok6wVFwrMzvz8BZ2JpG0LmxYjtvPqsGwySv5YNrqsOOInIz8wKXAE8BzwPNA2VATicgpi4kxnr+uKY0rFeeeUTOYs2ZH2JFyneMWJczsTjP7HvgWKAXc5u6N03HfFYFVqS6vDvYd8ZhguaQdwWOkdjUwzd33HyFbLzNLNLPETZs2pSOSZBfjF27kygE/sXt/MiN7tebahMphRxKJSmZ2KzCRyFJz/wi+Ph5mJpGjeej8OrSpUZK/fDib+et2hh1H5ETFBXNJHN6+BwqEHUpETl2BvLEMurEFJQvlpedbU1m3Q3MgZaX0dEpUBu519wbBUkVZ1ndpZg2InNJx+5Gud/eB7p7g7gnx8Zr0MCdwdwZNXErPoVOpXLIgH/dtT4uqJcOOJRLN7gFaAivcvSPQDNgeaiKRo4iLjeHlLs0pmj8PdwxL0jJski2Y2R1mNhuoEyy9fHhbBmgJZpEcokyR/AzukcCe/YfoOTSRPfuTw46Ua6RnTolH3H3GSdz3GiIFjcMqBfuOeIyZxRFZ5WNLcLkS8CFwo7v/ehKPL9nM/uRDPDRmFv8aN58LGpRjzB2nU7G4PoAQOY597r4PIpMHu/sCoE7ImUSOKr5IPl7p1pzV237jwdEzNdu5ZAcjiKw8Nzb4enhr4e7dwwwmIhmrbrmi9O/ajIUbdnH3yOkcStHfqKyQmYuxTgVqmVl1M8sLdCYymKc2lshElgDXAN+5u5tZceAz4GF3n5SJGSVKbNy1jy4DJzMmaTX3nFOLAV2bUzBvXNixRLKD1cGY+RHwtZl9DKwINZHIcSRUK8lfLq7HV/M28PrEpWHHETkmd9/h7svdvYu7r0i1bQ07m4hkvA51yvD45Q34dsFG/vXZ/LDj5AqZ9q7P3ZPNrC+R85tjgSHuPtfMngAS3X0sMBh4x8yWAFuJFC4A+gI1gUfN7NFg3/nuvjGz8kp45qzZwW1vJ7Jt7wEGdG3OJY3Lhx1JJNtw9yuDbx83s/FEOs6+CDGSSLrc3K4a01Zu499fLKBxpWK0Pa102JFEREQAuKFNVZZt2sOQScuoXrogN5xeLexIOVqmfhTt7uOAcWn2PZrq+33AtUe43T+Bf2ZmNokOn81axwPvzaBkwbyM6d2WhhWLhR1JJNsIll6e6+51Adx9QsiRRNLNzHjm6sYsWB9pkf30rjMoVyx/2LFEREQA+Osl9VixZQ+PfzKPyiUL0qFOehaglJORmadviBxVSorT7+tF9Bkxjfrli/Jx3/YqSIicIHc/BCw0syphZxE5GYXyxfFa9+b8duAQfUZM40ByStiRREREAIiNMV7q0ow6ZYvQd8R0FqzXqlGZRUUJyXJ7DyTTZ8Q0Xvx2Mde0qMTIXm2IL5Iv7Fgi2VUJYK6ZfWtmYw9vYYcSSa+aZYrwzDWNSVqxjac/17m7IiISPQrli2NwjwQK5Yul59BENu7aF3akHElFCclSa7b/xjWv/syXc9fz14vr8ew1jckXFxt2LJHsLD9wKfAE8BzwPFA21EQiJ+jSxhW4pV113py0nLEz14YdR0RE5HflixVg8E0t2brnALe9ncRvBw6FHSnHUVFCskzi8q106v8jq7buZXCPltx2Zg3MLOxYItldnLtPSLV9D2gtXcl2Hrm4LglVS/Dw+7NYvGFX2HFERER+17BiMV7s3JRZq7dz/+gZpGip0AylooRkidGJq+gyaDKF88XxYZ+2dNREMSKnxMzuMLPZQB0zm5VqWwbMCjufyInKExvDgG6R5aB7D0ti9/7ksCOJiIj87vwG5fjrxfX4fM56nv1qYdhxchQVJSRTHUpx/vnpPP40Zhatq5fioz7tqFmmSNixRHKCEcBlwNjg6+Gthbt3DzOYyMkqWzQ//bs2Y/mWvfxpzEzc9UmUiIhEj57tq9OtdRVe/f5XRk9dFXacHENFCck0O347yC1Dp/LGj8vo0bYaQ29uSfGCecOOJZIjuPsOd1/u7l3cfUWqbWvY2URORZsapfjTBXUYN3s9V736E4nL9SstIiLRwcx4/PIGnFGrNH/5cDY/LdkcdqQcQUUJyRTLNu/hylcmMWnJZp66shGPX96AuFj9uomIyPH1OrMG/76mMWu3/8Y1r/3M7e8ksnTT7rBjiYiI/H66YY34QvQelsSSjfr7dKr0LlEy3A+LN9Gp/49s23OAYbe2pmvrKmFHEhGRbMTMuC6hMuMf7MAD59Xmx8WbOb/fRB79eA5bdu8PO56IiORyRfPnYfBNLckbF8MtQ6eydc+BsCNlaypKSIZxd96ctIweb06lfLECjO3bnjY1SoUdS0REsqmCeeO465xafP9QRzq3qszwKSs569nvGTB+iZZkExGRUFUuWZCBNyawYec+er2dyP5k/V06WSpKSIY4kJzCXz6czT8+mcfZdcvw/p1tqVyyYNixREQkB4gvko9/XtGIL+89k9NPK8WzXy6k43++573EVRzSsmwiIhKS5lVK8Nx1TUhcsY0/jZmlCZpPkooScsq27N5P9zemMPKXVfTpeBqvd29B4XxxYccSEZEcpmaZwgy6MYF3e7WhbLH8PDRmFpe89AMTFm0KO5qIiORSlzauwEMX1OHjGWt58dvFYcfJllSUkFMyf91OLu8/iZmrt/Ni56Y8dEFdYmIs7FgiIpKDta5Rio/ubMvLXZqx50AyNw35hRsGT2He2p1hRxMRkVzozg6ncU2LSrzwzWI+mr4m7DjZjooSctK+nLueq1/9ieSUFEbffjqdmlYMO5KIiOQSZsZlTSrwzf1n8fdL6zN7zQ4uefkHHhg9k3U7fgs7noiI5CJmxlNXNqJ19ZL8acwsLWd9glSUkBPm7vT/bjG3v5NErTKFGdu3PU0qFw87loiI5EL54mLp2b46Ex7sSK8zavDJrLV0ePZ7/v3FAnbuOxh2PBERySXyxsXw+g0tqFiiAL3eSWLFlj1hR8o2VJSQE7Lv4CHuHjWD/3y1iE5NK/Du7adTtmj+sGOJiEguV6xgHh65uB7fPXAWFzUsxyvf/0qHZ7/nrZ+Wc/BQStjxREQkFyheMC9DerQkxZ2bh05lx14Vx9NDRQlJt/U79nHtaz/z6ay1/OnCOrxwfVPy54kNO5aInAQzG2JmG81szlGu72Zms8xstpn9ZGZNsjqjyMmoVKIgL3Ruxid921OnbBEeGzuX8/tN5Is56zQruoiIZLrqpQsx8IYEVm3dS+9hSRxIVmH8eFSUkHSZvnIbl/X/kaWbdjPohgTu7FATM01oKZKNDQUuPMb1y4Cz3L0R8CQwMCtCiWSURpWKMeK21gzpkUBcjNF72DSuee1nklZsCzuaiIjkcK2ql+SZqxvz89It/O2j2SqKH4eKEnJcH05fzfUDJ5M/Twwf3NmOc+uXDTuSiJwid58IHHUWJnf/yd0Pv3ubDFTKkmAiGcjMOLtuWT6/5wyevqoRK7fu5epXf+KOYUks26xzfUVEJPNc1bwSd59dk9GJq3ltwtKw40S1uLADSPQ6lOI8++VCXpvwK62rl+TV7i0oWShv2LFEJOv1BD4PO4TIyYqLjaFLqypc3qQCg35YysCJS/l63ga6t6nK3efU0t82ERHJFPedV5vlW/byzBcLqFqqIBc3Kh92pKikooQc0a59B7l31Ay+XbCRrq2r8PhlDcgbp8YakdzGzDoSKUq0P8YxvYBeAFWqVMmiZCInrlC+OO49tzZdW1fhhW8W887kFbyftJo7Op7GLe2qa54kERHJUGbGv69pzOpte7nv3RlUKF6Aplq18A/0LlP+YOWWSHvr94s28WSnBjx1ZSMVJERyITNrDLwBdHL3LUc7zt0HunuCuyfEx8dnXUCRk1SmSH6eurIRX957Bq1rlOTfXyyk43++Z0zSalJSdN6viIhknPx5Yhl0YwJliubj1rcSWb1tb9iRoo7eacr/+OnXzVw+4Ec27NzP27e04obTq4UdSURCYGZVgA+AG9x9Udh5RDJDzTJFeOOmlozq1Yb4Ivl48L2ZXPLyj/yweFPY0UREJAcpVTgfb/Zoyf7kQ/QcmsiufVoqNDUVJeR370xewY2Df6F04Xx83Kcd7WqWDjuSiGQSMxsJ/AzUMbPVZtbTzHqbWe/gkEeBUsArZjbDzBJDCyuSydrUKMVHd7bjxc5N2bXvIDcM/oUbh/zC/HU7w44mIiI5RM0yRXitewt+3bSbviOmk3xIS4UepjklhIOHUnjik3m8M3kFHevE82KXZhTNnyfsWCKSidy9y3GuvxW4NYviiIQuJsbo1LQiFzYsxzs/r+Dl75Zw8Us/cE3zSjxwfh3KFcsfdkQREcnm2tUszZNXNOSRD2bzj0/m8USnBphZ2LFCp6JELrdtzwHuHD6Nn5du4fYza/CnC+sSG6P/GCIikjvli4vl1jNqcE2LSgwYv4S3flrBJ7PWcmv7Gtx+Vg2KqGgvIiKnoEurKizfvIfXJy6leulC3NK+etiRQqfTN3KxxRt2ccUrk0hasY3nrm3CIxfXU0FCREQEKF4wL3+9pD7fPnAW59cvR//xS+jw7Pe8/fNyDqrlVkRETsGfL6zLBQ3K8uRn8/hm3oaw44RORYlc6rsFG7jylZ/Ys/8QI3u14eoWlcKOJCIiEnUqlyzIS12a8XGfdtQsU5hHP57LBf0m8sWc9bhrpQ4RETlxMTHGC9c3o1HFYtw9ajpz1+4IO1KoVJTIZdyd1yb8Ss+3EqlWuiBj+7ajRdUSYccSERGJak0qF2dUrzYMvimBmBij97Akrnv9Z6at3BZ2NBERyYYK5I3ljRsTKFYgDz2HJrJ+x76wI4VGRYlcZN/BQzwweib/9/kCLm5Unvdub0uF4gXCjiUiIpItmBnn1CvLF/ecwVNXNmLZ5r1c9cpP9Bk+jRVb9oQdT0REspkyRfMzpEdLdu07SM+3prJnf3LYkUKhokQusXHnPjoPnMwH09dw/3m16d+lGQXyxoYdS0REJNuJi42ha+sqTHioA/ecU4vvFmzk3Ocn8I9P5rJtz4Gw40kUMrMLzWyhmS0xs4ePcsx1ZjbPzOaa2YhU+/8d7JtvZi+ZpuoXyVHqlS9K/67Nmb9uJ/eMmsGhlNx3aqCKErnA7NU7uLz/JBau38Vr3Ztz9zm1tPSMiIjIKSqUL477zqvNhIc6cE2LSrz103LOfHY8r034lX0HD4UdT6KEmcUCA4CLgPpAFzOrn+aYWsAjQDt3bwDcG+xvC7QDGgMNgZbAWVkWXkSyRMe6ZXjssgZ8M38D//f5/LDjZDkVJXK4T2au5drXfyI2xhhzx+lc2LB82JFERERylDJF8/P0VY354t4zaVmtJP/3+QLOeW4CH0xbTUou/MRL/qAVsMTdl7r7AWAU0CnNMbcBA9x9G4C7bwz2O5AfyAvkA/IAmqpfJAe6qW01erStxqAfljF8yoqw42QpFSVyqJQU5/mvFnLXyOk0rFCMj/u2o0GFYmHHEhERybFqly3CkB4tGXFba0oUysP9o2dyWf8fmbRkc9jRJFwVgVWpLq8O9qVWG6htZpPMbLKZXQjg7j8D44F1wfalu+e+j1FFcom/XVKPjnXiefTjuUxctCnsOFlGRYkcaM/+ZO4YnsRL3y3huoRKDL+tNaUL5ws7loiISK7Q9rTSjO3Tnheub8r2vQfp9sYUerz5CwvX7wo7mkSvOKAW0AHoAgwys+JmVhOoB1QiUsg428zOSHtjM+tlZolmlrhpU+55IyOS08TFxvBy1+bUKlOYPsOnsWhD7vi7oaJEDrN6216ufvUnvp63gb9fWp9nrm5MvjhNaCkiIpKVYmKMK5pV5NsHzuIvF9dl2optXPTiRP48ZhYbdubeZd9yqTVA5VSXKwX7UlsNjHX3g+6+DFhEpEhxJTDZ3Xe7+27gc+D0tA/g7gPdPcHdE+Lj4zPlSYhI1iicL44hPVqSP28sN785lU279ocdKdOpKJGDTF2+lU79J7Fm+2+8eXMreravrgktRUREQpQ/Tyy9zjyNCQ915OZ21flg+mo6PPs9z3+1kN25dOm3XGgqUMvMqptZXqAzMDbNMR8R6ZLAzEoTOZ1jKbASOMvM4swsD5FJLnX6hkgOV6F4AQbflMCWPfu57e3EHD95sooSOcS7U1fSddBkihbIw0d92nFWbVXJRUREokWJQnn5+6X1+fb+DpxTrwwvfbeEDs+OZ9jkFSQfSgk7nmQid08G+gJfEikojHb3uWb2hJldHhz2JbDFzOYRmUPiIXffAowBfgVmAzOBme7+SZY/CRHJco0rFeeF65sxc/V2Hhg9M0dPnGzuOePJJSQkeGJiYtgxslzyoRT+NW4+b05azhm1StO/S3OKFcwTdiwRScXMktw9IewcWSG3jsUiJ2rGqu089dl8flm+lRrxhXj4wrqcV7+sOhwzicZhEcmuBk78lafGLaBvx5o8eEGdsOOckqONxZnaKWFmF5rZQjNbYmYPH+H6fGb2bnD9FDOrFuwvZWbjzWy3mfXPzIzZ2Y69B7l56FTenLScW9pV580eLVWQEBERyQaaVi7Ou7e3YeANLQDo9U4S1w+czIxV28MNJiIiUeW2M2rQpVVl+o9fwnuJq45/g2wo04oSZhYLDAAuAuoDXcysfprDegLb3L0m0A94Jti/D/g78GBm5cvuft20mytfmcTkpVt45upGPHpZfeJidTaOiIhIdmFmnN+gHF/eeyZPXtGQpZt2c8WASfQdMY2VW/aGHU9ERKKAmfFEp4a0r1mav3w4m59/3RJ2pAyXme9iWwFL3H2pux8ARgGd0hzTCXgr+H4McI6ZmbvvcfcfiRQnJI0JizZxxYBJ7PjtIMNvbcP1LauEHUlEREROUp7YGG5oU5XvH+rI3WfX5Jv5Gzjn+e958tN5bNtzIOx4IiISsjyxMQzo1pyqpQrRe1gSSzftDjtShsrMokRFIHV/yepg3xGPCSYB2gGUysRM2Zq7M/jHZdz85i9ULF6Aj/u2o1X1kmHHEhERkQxQOF8c959fhwkPdeSqZpV4c9Iyznp2PK9P+DXHz7wuIiLHVqxAHt7s0ZK4GOOWoVNzVNE6W/f7m1kvM0s0s8RNmzaFHSdT7U8+xJ/fn8WTn87jvPplef+OtlQqUTDsWCIiIpLByhbNzzPXNGbcPWfQvGoJnv58Aec8N4GPpq/J0bOvi4jIsVUuWZCBN7Zg7Y593P5OEvuTc0bBOjOLEmuAyqkuVwr2HfEYM4sDigHpPknG3Qe6e4K7J8TH59wlMDfv3k+3QVMYnbiau8+uyavdWlAoX1zYsURERCQT1S1XlKE3t2L4ra0pXjAP9747g04DJvHTr5vDjiYiIiFpUbUk/7m2Cb8s38oj788mJ6ymmZlFialALTOrbmZ5gc7A2DTHjAVuCr6/BvjOc8JPNQPNW7uTTv0nMXvNDl7u0oz7z69DTIyWCxMREckt2tUszSd929Pv+iZs3XOAroOmcMvQqSzesCvsaCIiEoLLm1TggfNq88H0NfT/bknYcU5Zpn3c7u7JZtYX+BKIBYa4+1wzewJIdPexwGDgHTNbAmwlUrgAwMyWA0WBvGZ2BXC+u8/LrLzR6Is567jv3ZkUK5CHMb3b0qhSsbAjiYiISAhiYowrm1XiooblGfrTcgaMX8IFL0zk+paVue/c2pQpmj/siCIikoX6nl2TZZv38NzXi6hSqiCdmqadvjH7yNRzANx9HDAuzb5HU32/D7j2KLetlpnZopm78/J3S3j+60U0rVycgTe00IsNERERIX+eWHqfdRrXJVTm5e8WM2zyCj6avpbbzqzB7WfW0OmdIiK5hJnx9NWNWL3tNx4aM4tKJQrQomr2XAQhW090mRP9duAQfUdO5/mvF3FVs4qM6tVGBQkRERH5HyUL5eWxyxrwzf1ncXbdMrz07WLOevZ7hk9ZQfKhlLDjiYhIFsgXF8vrN7SgQrH89Ho7iZVb9oYd6aSoKBFF1m7/jWtf/4lxs9fxyEV1ee66JuTPExt2LBEREYlSVUsVYkC35nxwZ1uqlSrIXz+cw4Uv/sA38zbkiMnPRETk2EoUysuQHi1JTnFueWsqO347GHakE6aiRJSYtnIbl/efxPLNe3njxgRuP+s0zDShpYiIiBxf8yoleK/36bzWvQWHUpxb306k88DJzFq9PexoIiKSyWrEF+b1G1qwYsse7hyexMFs1jGnokQUeD9pNZ1fn0zBvLF8cGdbzqlXNuxIIiIiks2YGRc2LMdX953Jk50asGTjbi7vP4m7R05n1dbs2dIrIiLp06ZGKZ6+qjGTlmzh7x/NyVbdcpoNKUSHUpxnvljAwIlLOb1GKV7p1pwShfKGHUtERESysTyxMdxwejWuaFaR1ycs5Y0fl/LFnPXc1LYqfTrWpHhBvdYQEcmJrmlRieWb99B//BJqxBei15mnhR0pXVSUCMmufQe5e+R0xi/cxA1tqvLoZfXJE6vGFREREckYRfLn4cEL6tCtTRWe/2oRb/y4jNGJq+nbsSY3tq1KvjjNWyUiktPcf15tlm3Zw9OfL6BKyUJc2LBc2JGOS++CQ7B88x6ufOUnfli8mX9e0ZAnr2iogoSIiIhkivLFCvDstU0Yd/cZNKlcnH+Nm885z03g4xlrSEnJPu29IiJyfDExxnPXNqFJpeLc++70bDG3kN4JZ7Gflmym04BJbN69n7d7tqJ7m6phRxIREZFcoF75orx9Syve6dmKIvnzcM+oGVzxyiQmL90SdjQREclA+fPEMujGBEoXzkfPtxJZu/23sCMdk4oSWcTdefvn5dww5BfKFMnHx33a0fa00mHHEhERkVzmjFrxfHpXe/5zbRM27dpP54GTufWtqSzZuCvsaCIikkHii+RjSI+W7DtwiFuGTmX3/uSwIx2VihJZ4OChFP760Rwe/XguHWrH88GdbalaqlDYsURERCSXio0xrmlRifEPduChC+oweelWLnjhB/7y4Ww27toXdjwREckAtcsWYUC35izeuJu7RkwjOUqXClVRIpNt3XOA7m9MYcSUlfQ+6zQG3phAkfx5wo4lIiIiQv48sfTpWJMJD3XghjZVGT11FR2e/Z4Xv1nM3gPR+6maiIikz5m143miUwPGL9zEPz+bH3acI1JRIhMtXL+LTgN+ZPqq7fS7vgkPX1SX2BgLO5aIiIjI/yhVOB+PX96Ar+8/i7Nqx9Pvm0Wc9ez3jPxlZdR+siYiIunTrXVVbm1fnaE/LWfopGVhx/kDFSUyydfzNnDVK5PYdzCFd3u14cpmlcKOJCIiInJM1UsX4tXuLXj/jtOpXKIAj3wwm4te/IHvFmzAXSt1iIhkV49cXI/z6pfliU/nMX7BxrDj/A8VJTKYu/PK90vo9U4iNeILM7ZvO5pVKRF2LBEREZF0a1G1JO/f0ZZXuzXn4KEUbhmaSNdBU5i9ekfY0URE5CTExhgvdm5KvfJF6TtiGvPW7gw70u9UlMhA+w4e4t53Z/DvLxZyaeMKvNf7dMoXKxB2LBEREZETZmZc1Kg8X99/Fv+4vAELN+zisv4/cu+o6azaujfseCIicoIK5o1j8E0tKZI/Dz3fmsrGndExsbGKEhlkw859XP/6z3w8Yy0Pnl+blzo3JX+e2LBjiYiIiJySPLEx3NS2Gt8/1IE7O5zG53PWc85zE3hq3Hx27D0YdjwRETkB5YrlZ3CPBHb8dpCebyVGxaTGKkpkgJmrtnN5/x9ZvHE3r9/Qgr5n18JME1qKiIhIzlE0fx7+dGFdxj/YgcuaVGDQD0s56z/jeeOHpexPPhR2PBERSacGFYrxcpdmzF27g/venUFKSrhzBqkocYo+nrGG617/mbiYGN6/oy0XNCgXdiQRERGRTFOheAGeu64Jn97VnkYVi/HPz+Zz7vMT+GTmWk2GKSKSTZxTryx/u6Q+X87dwDNfLAg1i4oSJyklxXn2ywXcM2oGTSoVZ2zfdtQrXzTsWCIiIiJZokGFYrzTszVv3dKKQnnjuGvkdK545Sd+WbY17GgiIpION7erxg1tqvL6xKWM/GVlaDniQnvkbGz3/mTue3cGX8/bQOeWlXmiU0Pyxqm+IyIiIrnPWbXjaV+zNB9MW81zXy3iutd/5tx6ZXn4orrULFM47HgiInIUZsZjl9Vn5da9/P2jOVQuUZD2tUpneQ69kz5Bq7bu5epXfuK7BRt5/LL6PH1VIxUkREREJFeLjTGuTajM+Ac78NAFdZi8dAsXvDCRv344m0279ocdT0REjiIuNob+XZtxWnxh7hiexOINu7I8g95Nn4DJS7fQacAk1u34jaE3t6RHu+qa0FJEREQkUCBvLH061uT7hzrQrXUV3p26ig7PjuelbxdHxQzvIiLyR0Xy52FwjwTyxcVyy1tT2bw7a4vJKkqk04gpK+n+xhSKF8zDR33acUat+LAjiYiIiESl0oXz8USnhnx135m0r1Wa579eRMf/fM+7U1dyKORZ3kVE5I8qlSjIGzclsGnXfnq9nci+g1m3qpKKEseRfCiFxz6ew18+nE27mqX58M521IjX+ZEikr2Z2RAz22hmc45yvZnZS2a2xMxmmVnzrM4oItlfjfjCvH5DAu/1Pp0KxQvw5/dnc/GLPzB+4Uat1CEiEmWaVi5Ov+uaMm3ldh4aMyvLxmkVJY5h+94D3PTmL7z18wpubV+dIT1aUqxAnrBjiYhkhKHAhce4/iKgVrD1Al7NgkwikkO1rFaSD+5oy4CuzdmXfIib35xK98FTmLNmR9jRREQklYsalefPF9blk5lr6ff1oix5TK2+cRRLNu7m1remsnb7Pv59TWOuS6gcdiQRkQzj7hPNrNoxDukEvO2REvlkMytuZuXdfV3WJBSRnMbMuKRxec6rX5Zhk1fw0neLuaz/j1zZtCIPXFCHisULhB1RRESA3mfVYPnmPbz03RKqlirE1S0qZerjqShxBOMXbuTuEdPJlyeGkb1a06JqybAjiYhktYrAqlSXVwf7/lCUMLNeRLopqFKlSpaEE5HsK29cDLe0r87VLSrx6ve/MmTSMj6dvY6b21Xjzg411ZUqIhIyM+PJKxqyatteHv5gFpVKFKB1jVKZ9ng6fSMVd2fQxKX0HDqVSiUL8nHf9ipIiIgch7sPdPcEd0+Ij9ckwCKSPsUK5OHhi+oy/sEOXNqoPAMnLqXDs+MZ8uMyDiSnhB1PRCRXyxsXw6vdWlC5ZEFuH5bE8s17Mu2xVJQI7E8+xENjZvGvcfO5oEE53r/jdLURikhutgZIfd5apWCfiEiGqli8AM9f35RP+ranfoWiPPHpPM7rN4HPZq3TZJgiIiEqVjAPb/ZoiQG3DJ3K9r0HMuVxVJQANu7aR5eBkxmTtJp7zqnFgK7NKZhXZ7aISK42FrgxWIWjDbBD80mISGZqWLEYw3q25s2bW5I/LpY+I6Zx1as/kbh8a9jRRERyraqlCjHwxgRWb/uN299JypROtlxflJizZgdX9J/EvHU7GdC1OfedV5uYGAs7lohIpjKzkcDPQB0zW21mPc2st5n1Dg4ZBywFlgCDgDtDiioiuYiZ0bFOGcbdcwbPXN2INdt+45rXfub2dxJZuml32PFERHKlltVK8uy1jZmybCuPfDA7w7vYcnU7wLjZ63hg9ExKFMzDmN5taVixWNiRRESyhLt3Oc71DvTJojgiIv8jNsa4vmUVLmtSgcE/LOO1Cb/yzfyJdG1VhXvOrUXpwvnCjigikqt0alqRZZv38MI3i6kRX4g+HWtm2H3n2k6JjTv3cf/oGdQrX4SP+rZTQUJEREQkyhTMG8dd59Ti+4c60qVVZUb8spLnvloYdqwTZmYXmtlCM1tiZg8f5ZjrzGyemc01sxGp9lcxs6/MbH5wfbUsCy4ikso959TiiqYV6Pf1IlZt3Zth95trOyXKFM3PsJ6taVSpGPniYsOOIyIiIiJHEV8kH/+8ohE92lancL7s9fLVzGKBAcB5RJZXnmpmY919XqpjagGPAO3cfZuZlUl1F28D/3L3r82sMKClSUQkFGbGM9c0pnubqlQuWTDD7jd7jeoZLKGalvsUERERyS5qlikcdoST0QpY4u5LAcxsFNAJmJfqmNuAAe6+DcDdNwbH1gfi3P3rYL8m1hCRUOWLi83w99G59vQNEREREZEsUBFYlery6mBfarWB2mY2ycwmm9mFqfZvN7MPzGy6mT0bdF6IiOQYKkqIiIiIiIQrDqgFdAC6AIPMrHiw/wzgQaAlUAPokfbGZtbLzBLNLHHTpk1ZFFlEJGOoKCEiIiIiknnWAJVTXa4U7EttNTDW3Q+6+zJgEZEixWpghrsvdfdk4COgedoHcPeB7p7g7gnx8fGZ8RxERDKNihIiIiIiIplnKlDLzKqbWV6gMzA2zTEfEemSwMxKEzltY2lw2+JmdrjScDb/OxeFiEi2p6KEiIiIiEgmCToc+gJfAvOB0e4+18yeMLPLg8O+BLaY2TxgPPCQu29x90NETt341sxmAwYMyvpnISKSeTJ19Y1gkp4XgVjgDXf/vzTX5yOyzFELYAtwvbsvD657BOgJHALudvcvMzOriIiIiEhmcPdxwLg0+x5N9b0D9wdb2tt+DTTO7IwiImHJtE6JVGsyXwTUB7oEyxql1hPY5u41gX7AM8Ft6xNpbWsAXAi8opmGRURERERERHKWzDx94/c1md39AHB4TebUOgFvBd+PAc4xMwv2j3L3/cFkP0uC+xMRERERERGRHCIzixLpWZP592OC8+12AKXSeVstfyQiIiIiIiKSjWXriS61/JGIiIiIiIhI9pWZE12mZ03mw8esNrM4oBiRCS/Tc9v/kZSUtNnMVpxEztLA5pO4XWZTrhMXrdmiNRdEb7aclqtqRgeJVic5FkfrvzdEbzblOnHRmk25TtzJZNM4fHzR+m+uXCcuWrNFay6I3mw5LdcRx2KLTPab8YIiwyLgHCIFhalAV3efm+qYPkAjd+9tZp2Bq9z9OjNrAIwgMo9EBeBboFawLFJG50x094SMvt9TpVwnLlqzRWsuiN5sypW7RPPPNVqzKdeJi9ZsynXiojlbdhatP1flOnHRmi1ac0H0ZsstuTKtU8Ldk83s8JrMscCQw2syA4nuPhYYDLxjZkuArURW3CA4bjQwD0gG+mRGQUJEREREREREwpOZp2+kZ03mfcC1R7ntv4B/ZWY+EREREREREQlPtp7oMoMMDDvAUSjXiYvWbNGaC6I3m3LlLtH8c43WbMp14qI1m3KduGjOlp1F689VuU5ctGaL1lwQvdlyRa5Mm1NCRERERERERORY1CkhIiIiIiIiIqHINUUJM6tsZuPNbJ6ZzTWze4L9Jc3sazNbHHwtEUK2/Gb2i5nNDLL9I9hf3cymmNkSM3vXzPJmdbYgR6yZTTezT6Mll5ktN7PZZjbDzBKDfaH/WwY5ipvZGDNbYGbzzez0sLOZWZ3gZ3V422lm94adK8h2X/B7P8fMRgb/H0L/HQuy3RPkmmtm9wb7Qv+ZZWfROhZrHD7pXFE5FkfjOBzk0lh84rk0DmewaB2Hgwwai088U1SOw0GOqBuLNQ6fdLZMHYtzTVGCyCoeD7h7faAN0MfM6gMPA9+6ey0iS48+HEK2/cDZ7t4EaApcaGZtgGeAfu5eE9gG9AwhG8A9wPxUl6MlV0d3b5pqOZpo+LcEeBH4wt3rAk2I/OxCzebuC4OfVVOgBbAX+DDsXGZWEbgbSHD3hkRW6ulMFPyOmVlD4DYiSxM3AS41s5pEz+9ZdhWtY7HG4ZMXjWNx1I3DoLH4JHJpHM4c0ToOg8bikxWN4zBE4ViscfiksmX+WOzuuXIDPgbOAxYC5YN95YGFIecqCEwDWgObgbhg/+nAlyHkqRT8kp0NfApYlORaDpROsy/0f0ugGLCMYL6WaMqWKsv5wKRoyAVUBFYBJYmsBvQpcEGU/I5dCwxOdfnvwJ/C/pnltC0ax2KNwyeULerG4uwwDgcZNBYfP5fG4az5OUfdOBxk0FicvlxRNw4Hjxv1Y7HG4XRny/SxODd1SvzOzKoBzYApQFl3XxdctR4oG1KmWDObAWwEvgZ+Bba7e3JwyGoiv6xZ7QUiv3QpweVSUZLLga/MLMnMegX7ouHfsjqwCXgzaO97w8wKRUm2wzoDI4PvQ83l7muA/wArgXXADiCJ6PgdmwOcYWalzKwgcDFQmej6t8zWom0s1jh8UqJxLM4O4zBoLE4PjcOZLNrG4SCTxuITE43jMGSPsVjjcPpk+lic64oSZlYYeB+41913pr7OI2WeUJYjcfdDHmkjqkSkNaZuGDlSM7NLgY3unhR2liNo7+7NgYuItB2emfrKEP8t44DmwKvu3gzYQ5pWpjB/z4Lz0C4H3kt7XRi5gnPPOhH5w1UBKARcmJUZjsbd5xNpmfsK+AKYARxKc0xo/5bZXTSOxRqHT0o0jsVRPQ6DxuL00jicuaJxHA4eW2PxiYnGcRiifCzWOJx+WTEW56qihJnlITL4Dnf3D4LdG8ysfHB9eSJV2dC4+3ZgPJH2nOJmFhdcVQlYk8Vx2gGXm9lyYBSRdrUXoyDX4Woi7r6RyHlgrYiOf8vVwGp3nxJcHkNkQI6GbBD5gzXN3TcEl8POdS6wzN03uftB4AMiv3eh/44BuPtgd2/h7mcSOY9vEeH/zLK9aB+LNQ6nX5SOxdE+DoPG4nTTOJw5on0cBo3F6RWl4zBE/1iscfgEZPZYnGuKEmZmwGBgvrs/n+qqscBNwfc3ETmvLquzxZtZ8eD7AkTO65tPZCC+Jqxs7v6Iu1dy92pE2pu+c/duYecys0JmVuTw90TOB5tDFPxbuvt6YJWZ1Ql2nQPMi4ZsgS78t00Nws+1EmhjZgWD/6OHf16h/o4dZmZlgq9VgKuAEYT/M8vWonUs1jh84qJ1LM4G4zBoLE43jcMZL1rH4SCbxuITEK3jMGSLsVjj8AnI9LHYQ5pYJKs3oD2RlpJZRFpOZhA5H6YUkUlrFgPfACVDyNYYmB5kmwM8GuyvAfwCLCHSWpQvxJ9fB+DTaMgVPP7MYJsL/DXYH/q/ZZCjKZAY/Ht+BJSIhmxE2sC2AMVS7YuGXP8AFgS/++8A+cL+HUuV7QcifxBmAudEy88sO2/ROhZrHD6pPFE7FkfrOBxk01h8Yrk0Dmf8zzQqx+Egm8biE8sSteNwkCMqx2KNwyeVLVPHYgvuUEREREREREQkS+Wa0zdEREREREREJLqoKCEiIiIiIiIioVBRQkRERERERERCoaKEiIiIiIiIiIRCRQkRERERERERCYWKEiInwcw6mNmnYecQEcnNNBaLiIRL47BkBBUlRERERERERCQUKkpIjmZm3c3sFzObYWavm1msme02s35mNtfMvjWz+ODYpmY22cxmmdmHZlYi2F/TzL75//bt58XmKIzj+PsjJYzIwsaCWKH8SFmYrPwDFiNFk6xt7KRI+R8Uy5FZiNgri6lZIZGSldWUmo2EIo3HYs5iyGpq5nzv9X6t7n3uuU/fU/d+Fk/nJHmT5FWS/a39RJJHSd4nmU2SbhuVpAEziyWpL3NYQ+ZQQmMryQHgHDBZVUeBJeACsBV4WVWHgDngZvvKPeBqVR0G3q6ozwK3q+oIcBL42OrHgCvAQWAfMLnGW5KkkWMWS1Jf5rCGbmPvB5DW0GngOPCiDWw3A4vAL+BBW3MfeJxkO7CjquZafQZ4mGQbsLuqngBU1XeA1u95VS2096+BvcD8mu9KkkaLWSxJfZnDGjSHEhpnAWaq6tofxeTGX+tqlf1/rHi9hP8nSfoXs1iS+jKHNWhe39A4ewZMJdkFkGRnkj0s/+6n2przwHxVfQY+JTnV6tPAXFV9ARaSnGk9NiXZsp6bkKQRZxZLUl/msAbNKZbGVlW9S3IdeJpkA/ATuAx8A060zxZZvmMHcBG40wL2A3Cp1aeBu0lutR5n13EbkjTSzGJJ6ssc1tClarWndKTRlORrVU30fg5J+p+ZxZLUlzmsofD6hiRJkiRJ6sKTEpIkSZIkqQtPSkiSJEmSpC4cSkiSJEmSpC4cSkiSJEmSpC4cSkiSJEmSpC4cSkiSJEmSpC4cSkiSJEmSpC5+AyAe9HeBs0TjAAAAAElFTkSuQmCC\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "job_name = od_model.latest_training_job.job_name\n", "df = TrainingJobAnalytics(job_name).dataframe()\n", @@ -626,7 +563,6 @@ }, { "cell_type": "markdown", - "id": "a597ec77", "metadata": { "papermill": { "duration": 0.218728, @@ -648,8 +584,7 @@ }, { "cell_type": "code", - "execution_count": 10, - "id": "acdb9837", + "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2022-08-05T08:41:00.893151Z", @@ -669,16 +604,7 @@ }, "tags": [] }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "sagemaker-soln-dfd-c17c2c11-Type1-2022-08-05-08-41-00-890\n", - "----------!" - ] - } - ], + "outputs": [], "source": [ "od_type1_endpoint_name = name_from_base(\"od-Type1\")\n", "print(od_type1_endpoint_name)\n", @@ -689,7 +615,6 @@ }, { "cell_type": "markdown", - "id": "c4195a0a", "metadata": { "papermill": { "duration": 0.22511, @@ -708,7 +633,7 @@ "\n", "Now we run HPO to find better hyperparameters which lead to better model. You could find all [finetunable hyperparameters](https://docs.aws.amazon.com/sagemaker/latest/dg/object-detection-tuning.html) for the Type 1 (legacy) OD model. In this notebook, we only finetune learning rate, momentum, and weight decay. \n", "\n", - "We will use [Sagemaker Automatic Model Tuning](https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning.html) (AMT) to run HPO. We need to provide hyperparameter ranges and objective metrics. AMT will monitor the log and parse the objective metrics. For object detection, we use mean Average Precision (mAP) on the validation dataset as our metric. mAP is the standard evaluation metric used in the [COCO Challenge](https://cocodataset.org/#detection-eval) for object detection tasks. Here is a nice [blog post](https://jonathan-hui.medium.com/map-mean-average-precision-for-object-detection-45c121a31173) explaining mAP for object detection. \n", + "We use [Sagemaker Automatic Model Tuning](https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning.html) (AMT) to run HPO. We need to provide hyperparameter ranges and objective metrics. AMT monitors the log and parses the objective metrics. For object detection, we use mean Average Precision (mAP) on the validation dataset as our metric. mAP is the standard evaluation metric used in the [COCO Challenge](https://cocodataset.org/#detection-eval) for object detection tasks. Here is a nice [blog post](https://jonathan-hui.medium.com/map-mean-average-precision-for-object-detection-45c121a31173) explaining mAP for object detection. \n", "\n", "We run `max_jobs=20` jobs in this HPO. You could run more jobs to find even better hyperparameters, at the cost of more compute resources and training time. This HPO job takes about 1 hour using p3.2xlarge EC2 instance and run `max_parallel_jobs=10` jobs in parallel." ] @@ -716,7 +641,6 @@ { "cell_type": "code", "execution_count": 4, - "id": "df2c3f9c", "metadata": { "papermill": { "duration": 3580.578995, @@ -805,8 +729,7 @@ }, { "cell_type": "code", - "execution_count": 12, - "id": "aa495c57", + "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2022-08-05T09:45:48.928117Z", @@ -826,388 +749,7 @@ }, "tags": [] }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "tuning_job_name sagemaker-soln-dfd-c-220805-0846\n", - "Number of training jobs completed and with valid objective: 20 / 20\n", - "{'lowest': 0.049028314650058746, 'highest': 0.6942322850227356}\n" - ] - }, - { - "data": { - "text/html": [ - "
\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
learning_ratemomentumweight_decayTrainingJobNameTrainingJobStatusFinalObjectiveValueTrainingStartTimeTrainingEndTimeTrainingElapsedTimeSeconds
60.0100000.9153440.000757sagemaker-soln-dfd-c-220805-0846-014-f4010610Completed0.6942322022-08-05 09:17:48+00:002022-08-05 09:44:01+00:001573.0
20.0082740.9179440.000406sagemaker-soln-dfd-c-220805-0846-018-5de8ba38Completed0.6863022022-08-05 09:18:00+00:002022-08-05 09:44:03+00:001563.0
70.0100000.8909210.000819sagemaker-soln-dfd-c-220805-0846-013-bcc2a540Completed0.6807882022-08-05 09:17:52+00:002022-08-05 09:43:56+00:001564.0
110.0076600.9107670.000881sagemaker-soln-dfd-c-220805-0846-009-2e6be776Completed0.6803422022-08-05 08:48:13+00:002022-08-05 09:14:54+00:001601.0
50.0100000.9290580.000285sagemaker-soln-dfd-c-220805-0846-015-a3f5360dCompleted0.6796282022-08-05 09:17:50+00:002022-08-05 09:43:58+00:001568.0
80.0100000.8698720.000414sagemaker-soln-dfd-c-220805-0846-012-3bd6b6ebCompleted0.6785192022-08-05 09:17:50+00:002022-08-05 09:44:49+00:001619.0
40.0100000.9463880.000255sagemaker-soln-dfd-c-220805-0846-016-49063caeCompleted0.6763662022-08-05 09:17:53+00:002022-08-05 09:43:57+00:001564.0
10.0083080.9056970.000727sagemaker-soln-dfd-c-220805-0846-019-55b2f862Completed0.6756102022-08-05 09:17:58+00:002022-08-05 09:43:59+00:001561.0
00.0094340.9006890.000128sagemaker-soln-dfd-c-220805-0846-020-b9beb058Completed0.6750382022-08-05 09:17:56+00:002022-08-05 09:43:59+00:001563.0
90.0100000.8317600.000831sagemaker-soln-dfd-c-220805-0846-011-4708d540Completed0.6722972022-08-05 09:17:42+00:002022-08-05 09:44:05+00:001583.0
30.0073360.9824960.000315sagemaker-soln-dfd-c-220805-0846-017-2270497fCompleted0.5797162022-08-05 09:17:58+00:002022-08-05 09:44:06+00:001568.0
100.0008140.9531450.000305sagemaker-soln-dfd-c-220805-0846-010-c1d32027Completed0.4330362022-08-05 08:48:14+00:002022-08-05 09:14:07+00:001553.0
160.0028960.8130390.000251sagemaker-soln-dfd-c-220805-0846-004-c68543e2Completed0.4227222022-08-05 08:48:10+00:002022-08-05 09:15:01+00:001611.0
120.0003430.9732920.000299sagemaker-soln-dfd-c-220805-0846-008-99e662feCompleted0.3695582022-08-05 08:48:10+00:002022-08-05 09:14:43+00:001593.0
150.0003000.9732000.000288sagemaker-soln-dfd-c-220805-0846-005-ed88073aCompleted0.3303962022-08-05 08:48:09+00:002022-08-05 09:14:12+00:001563.0
190.0011270.8298490.000103sagemaker-soln-dfd-c-220805-0846-001-a6aa1deaCompleted0.1874912022-08-05 08:47:56+00:002022-08-05 09:13:49+00:001553.0
130.0003350.9266170.000239sagemaker-soln-dfd-c-220805-0846-007-097b8bafCompleted0.1163812022-08-05 08:48:07+00:002022-08-05 09:14:00+00:001553.0
140.0003310.8748010.000427sagemaker-soln-dfd-c-220805-0846-006-adedfb8cCompleted0.0652662022-08-05 08:48:12+00:002022-08-05 09:15:01+00:001609.0
170.0002750.8795870.000414sagemaker-soln-dfd-c-220805-0846-003-ce6cddeaCompleted0.0595742022-08-05 08:47:57+00:002022-08-05 09:14:00+00:001563.0
180.0001810.9112740.000968sagemaker-soln-dfd-c-220805-0846-002-d9edb50aCompleted0.0490282022-08-05 08:48:03+00:002022-08-05 09:13:56+00:001553.0
\n", - "
" - ], - "text/plain": [ - " learning_rate momentum weight_decay \\\n", - "6 0.010000 0.915344 0.000757 \n", - "2 0.008274 0.917944 0.000406 \n", - "7 0.010000 0.890921 0.000819 \n", - "11 0.007660 0.910767 0.000881 \n", - "5 0.010000 0.929058 0.000285 \n", - "8 0.010000 0.869872 0.000414 \n", - "4 0.010000 0.946388 0.000255 \n", - "1 0.008308 0.905697 0.000727 \n", - "0 0.009434 0.900689 0.000128 \n", - "9 0.010000 0.831760 0.000831 \n", - "3 0.007336 0.982496 0.000315 \n", - "10 0.000814 0.953145 0.000305 \n", - "16 0.002896 0.813039 0.000251 \n", - "12 0.000343 0.973292 0.000299 \n", - "15 0.000300 0.973200 0.000288 \n", - "19 0.001127 0.829849 0.000103 \n", - "13 0.000335 0.926617 0.000239 \n", - "14 0.000331 0.874801 0.000427 \n", - "17 0.000275 0.879587 0.000414 \n", - "18 0.000181 0.911274 0.000968 \n", - "\n", - " TrainingJobName TrainingJobStatus \\\n", - "6 sagemaker-soln-dfd-c-220805-0846-014-f4010610 Completed \n", - "2 sagemaker-soln-dfd-c-220805-0846-018-5de8ba38 Completed \n", - "7 sagemaker-soln-dfd-c-220805-0846-013-bcc2a540 Completed \n", - "11 sagemaker-soln-dfd-c-220805-0846-009-2e6be776 Completed \n", - "5 sagemaker-soln-dfd-c-220805-0846-015-a3f5360d Completed \n", - "8 sagemaker-soln-dfd-c-220805-0846-012-3bd6b6eb Completed \n", - "4 sagemaker-soln-dfd-c-220805-0846-016-49063cae Completed \n", - "1 sagemaker-soln-dfd-c-220805-0846-019-55b2f862 Completed \n", - "0 sagemaker-soln-dfd-c-220805-0846-020-b9beb058 Completed \n", - "9 sagemaker-soln-dfd-c-220805-0846-011-4708d540 Completed \n", - "3 sagemaker-soln-dfd-c-220805-0846-017-2270497f Completed \n", - "10 sagemaker-soln-dfd-c-220805-0846-010-c1d32027 Completed \n", - "16 sagemaker-soln-dfd-c-220805-0846-004-c68543e2 Completed \n", - "12 sagemaker-soln-dfd-c-220805-0846-008-99e662fe Completed \n", - "15 sagemaker-soln-dfd-c-220805-0846-005-ed88073a Completed \n", - "19 sagemaker-soln-dfd-c-220805-0846-001-a6aa1dea Completed \n", - "13 sagemaker-soln-dfd-c-220805-0846-007-097b8baf Completed \n", - "14 sagemaker-soln-dfd-c-220805-0846-006-adedfb8c Completed \n", - "17 sagemaker-soln-dfd-c-220805-0846-003-ce6cddea Completed \n", - "18 sagemaker-soln-dfd-c-220805-0846-002-d9edb50a Completed \n", - "\n", - " FinalObjectiveValue TrainingStartTime TrainingEndTime \\\n", - "6 0.694232 2022-08-05 09:17:48+00:00 2022-08-05 09:44:01+00:00 \n", - "2 0.686302 2022-08-05 09:18:00+00:00 2022-08-05 09:44:03+00:00 \n", - "7 0.680788 2022-08-05 09:17:52+00:00 2022-08-05 09:43:56+00:00 \n", - "11 0.680342 2022-08-05 08:48:13+00:00 2022-08-05 09:14:54+00:00 \n", - "5 0.679628 2022-08-05 09:17:50+00:00 2022-08-05 09:43:58+00:00 \n", - "8 0.678519 2022-08-05 09:17:50+00:00 2022-08-05 09:44:49+00:00 \n", - "4 0.676366 2022-08-05 09:17:53+00:00 2022-08-05 09:43:57+00:00 \n", - "1 0.675610 2022-08-05 09:17:58+00:00 2022-08-05 09:43:59+00:00 \n", - "0 0.675038 2022-08-05 09:17:56+00:00 2022-08-05 09:43:59+00:00 \n", - "9 0.672297 2022-08-05 09:17:42+00:00 2022-08-05 09:44:05+00:00 \n", - "3 0.579716 2022-08-05 09:17:58+00:00 2022-08-05 09:44:06+00:00 \n", - "10 0.433036 2022-08-05 08:48:14+00:00 2022-08-05 09:14:07+00:00 \n", - "16 0.422722 2022-08-05 08:48:10+00:00 2022-08-05 09:15:01+00:00 \n", - "12 0.369558 2022-08-05 08:48:10+00:00 2022-08-05 09:14:43+00:00 \n", - "15 0.330396 2022-08-05 08:48:09+00:00 2022-08-05 09:14:12+00:00 \n", - "19 0.187491 2022-08-05 08:47:56+00:00 2022-08-05 09:13:49+00:00 \n", - "13 0.116381 2022-08-05 08:48:07+00:00 2022-08-05 09:14:00+00:00 \n", - "14 0.065266 2022-08-05 08:48:12+00:00 2022-08-05 09:15:01+00:00 \n", - "17 0.059574 2022-08-05 08:47:57+00:00 2022-08-05 09:14:00+00:00 \n", - "18 0.049028 2022-08-05 08:48:03+00:00 2022-08-05 09:13:56+00:00 \n", - "\n", - " TrainingElapsedTimeSeconds \n", - "6 1573.0 \n", - "2 1563.0 \n", - "7 1564.0 \n", - "11 1601.0 \n", - "5 1568.0 \n", - "8 1619.0 \n", - "4 1564.0 \n", - "1 1561.0 \n", - "0 1563.0 \n", - "9 1583.0 \n", - "3 1568.0 \n", - "10 1553.0 \n", - "16 1611.0 \n", - "12 1593.0 \n", - "15 1563.0 \n", - "19 1553.0 \n", - "13 1553.0 \n", - "14 1609.0 \n", - "17 1563.0 \n", - "18 1553.0 " - ] - }, - "execution_count": 12, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "tuning_job_name = tuner.latest_tuning_job.name\n", "print(\"tuning_job_name\", tuning_job_name)\n", @@ -1236,7 +778,6 @@ { "cell_type": "code", "execution_count": 13, - "id": "1a6c0eb7", "metadata": { "execution": { "iopub.execute_input": "2022-08-05T09:45:50.790276Z", @@ -1333,7 +874,6 @@ }, { "cell_type": "markdown", - "id": "aefe467d", "metadata": { "papermill": { "duration": 0.718616, @@ -1355,8 +895,7 @@ }, { "cell_type": "code", - "execution_count": 14, - "id": "e1a4057d", + "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2022-08-05T09:45:55.265867Z", @@ -1376,22 +915,7 @@ }, "tags": [] }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "sagemaker-soln-dfd-c17c2c11-Type1-HPO-2022-08-05-09-45-55-263\n", - "\n", - "2022-08-05 09:44:01 Starting - Preparing the instances for training\n", - "2022-08-05 09:44:01 Downloading - Downloading input data\n", - "2022-08-05 09:44:01 Training - Training image download completed. Training in progress.\n", - "2022-08-05 09:44:01 Uploading - Uploading generated training model\n", - "2022-08-05 09:44:01 Completed - Training job completed\n", - "-----------!" - ] - } - ], + "outputs": [], "source": [ "od_type1_hpo_endpoint_name = name_from_base(\"od-Type1-HPO\")\n", "print(od_type1_hpo_endpoint_name)\n", @@ -1402,7 +926,6 @@ }, { "cell_type": "markdown", - "id": "7173e802", "metadata": { "papermill": { "duration": 0.727216, @@ -1436,14 +959,13 @@ "\n", "\n", "There are two major differences between training the two types of OD models: \n", - "1. The entry point `transfer_learning.py` for finetuning a Type 2 (latest) OD model does not accept a validation data channel. Instead, it splits the input data provided through `estimator.fit({\"training\": s3_input_train})` to be train:val=80:20, corresponding to use 64% of total data for training and 16% for validation. Note, the train/val data will be different from train/val for training Type 1 (legacy) OD model. \n", + "1. The entry point `transfer_learning.py` for finetuning a Type 2 (latest) OD model does not accept a validation data channel. Instead, it splits the input data provided through `estimator.fit({\"training\": s3_input_train})` to be train:val=80:20, corresponding to use 64% of total data for training and 16% for validation. Note, the train/val data are different from train/val for training Type 1 (legacy) OD model. \n", "2. The evaluation metrics are different. While Type 1 (legacy) OD model reports mAP on the validation data, which is standard, the Type 2 (latest) OD model only reports CrossEntropy loss and SmoothL1 loss on the validation data. " ] }, { "cell_type": "code", "execution_count": null, - "id": "ccf32143", "metadata": { "execution": { "iopub.execute_input": "2022-08-05T09:51:34.423432Z", @@ -1558,7 +1080,6 @@ { "cell_type": "code", "execution_count": 16, - "id": "a084a69d", "metadata": { "execution": { "iopub.execute_input": "2022-08-05T11:20:35.631189Z", @@ -1612,7 +1133,6 @@ }, { "cell_type": "markdown", - "id": "a84ffcd3", "metadata": { "papermill": { "duration": 0.929678, @@ -1632,8 +1152,7 @@ }, { "cell_type": "code", - "execution_count": 17, - "id": "52474d43", + "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2022-08-05T11:20:40.714712Z", @@ -1653,16 +1172,7 @@ }, "tags": [] }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "sagemaker-soln-dfd-c17c2c11-Type2-2022-08-05-11-20-40-823\n", - "---------!" - ] - } - ], + "outputs": [], "source": [ "scope = \"inference\"\n", "inference_instance_type = \"ml.m4.xlarge\"\n", @@ -1695,7 +1205,6 @@ }, { "cell_type": "markdown", - "id": "5eabbc82", "metadata": { "papermill": { "duration": 0.889437, @@ -1718,7 +1227,6 @@ { "cell_type": "code", "execution_count": null, - "id": "0e1edb3a", "metadata": { "execution": { "iopub.execute_input": "2022-08-05T11:25:47.553857Z", @@ -1807,8 +1315,7 @@ }, { "cell_type": "code", - "execution_count": 19, - "id": "e233497c", + "execution_count": 8, "metadata": { "execution": { "iopub.execute_input": "2022-08-05T14:27:26.353071Z", @@ -1828,324 +1335,7 @@ }, "tags": [] }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "tuning_job_name sagemaker-soln-dfd-c-220805-1125\n", - "Number of training jobs completed and with valid objective: 20 / 20\n", - "{'lowest': 2.191999912261963, 'highest': 2.3610000610351562}\n" - ] - }, - { - "data": { - "text/html": [ - "
\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
adam-learning-rateTrainingJobNameTrainingJobStatusFinalObjectiveValueTrainingStartTimeTrainingEndTimeTrainingElapsedTimeSeconds
170.000330sagemaker-soln-dfd-c-220805-1125-003-3d4b78ccCompleted2.1922022-08-05 11:27:48+00:002022-08-05 12:54:27+00:005199.0
00.000388sagemaker-soln-dfd-c-220805-1125-020-965acb39Completed2.2202022-08-05 12:58:37+00:002022-08-05 14:22:07+00:005010.0
150.043813sagemaker-soln-dfd-c-220805-1125-005-2b911124Completed2.2282022-08-05 11:27:45+00:002022-08-05 12:53:09+00:005124.0
90.000215sagemaker-soln-dfd-c-220805-1125-011-87f69d05Completed2.2292022-08-05 12:58:19+00:002022-08-05 14:23:02+00:005083.0
110.000764sagemaker-soln-dfd-c-220805-1125-009-d05e188dCompleted2.2412022-08-05 11:27:49+00:002022-08-05 12:51:17+00:005008.0
120.000536sagemaker-soln-dfd-c-220805-1125-008-57b3cd7fCompleted2.2422022-08-05 11:27:52+00:002022-08-05 12:52:22+00:005070.0
140.051138sagemaker-soln-dfd-c-220805-1125-006-72634ffbCompleted2.2442022-08-05 11:27:50+00:002022-08-05 12:51:23+00:005013.0
40.000320sagemaker-soln-dfd-c-220805-1125-016-eaacf33cCompleted2.2512022-08-05 12:58:35+00:002022-08-05 14:24:13+00:005138.0
180.000273sagemaker-soln-dfd-c-220805-1125-002-1afaf8bcCompleted2.2542022-08-05 11:27:41+00:002022-08-05 12:52:15+00:005074.0
100.085353sagemaker-soln-dfd-c-220805-1125-010-c9d4e9d6Completed2.2562022-08-05 11:28:02+00:002022-08-05 12:51:44+00:005022.0
80.000232sagemaker-soln-dfd-c-220805-1125-012-e182e0ebCompleted2.2592022-08-05 12:58:19+00:002022-08-05 14:22:06+00:005027.0
10.000385sagemaker-soln-dfd-c-220805-1125-019-c561ffd3Completed2.2602022-08-05 12:58:49+00:002022-08-05 14:22:26+00:005017.0
70.000256sagemaker-soln-dfd-c-220805-1125-013-2eb9bba8Completed2.2672022-08-05 12:58:22+00:002022-08-05 14:22:40+00:005058.0
160.000428sagemaker-soln-dfd-c-220805-1125-004-75bf8a5eCompleted2.2702022-08-05 11:27:39+00:002022-08-05 12:52:27+00:005088.0
190.000111sagemaker-soln-dfd-c-220805-1125-001-a4003eb5Completed2.2852022-08-05 11:27:38+00:002022-08-05 12:52:04+00:005066.0
30.000325sagemaker-soln-dfd-c-220805-1125-017-930908e7Completed2.2862022-08-05 12:58:33+00:002022-08-05 14:23:23+00:005090.0
20.000338sagemaker-soln-dfd-c-220805-1125-018-d16dd68dCompleted2.2962022-08-05 12:58:33+00:002022-08-05 14:23:41+00:005108.0
60.000275sagemaker-soln-dfd-c-220805-1125-014-958bd465Completed2.3182022-08-05 12:58:23+00:002022-08-05 14:23:07+00:005084.0
50.030932sagemaker-soln-dfd-c-220805-1125-015-f43b94beCompleted2.3352022-08-05 12:58:36+00:002022-08-05 14:22:09+00:005013.0
130.005296sagemaker-soln-dfd-c-220805-1125-007-6e0ffed0Completed2.3612022-08-05 11:27:55+00:002022-08-05 12:52:14+00:005059.0
\n", - "
" - ], - "text/plain": [ - " adam-learning-rate TrainingJobName \\\n", - "17 0.000330 sagemaker-soln-dfd-c-220805-1125-003-3d4b78cc \n", - "0 0.000388 sagemaker-soln-dfd-c-220805-1125-020-965acb39 \n", - "15 0.043813 sagemaker-soln-dfd-c-220805-1125-005-2b911124 \n", - "9 0.000215 sagemaker-soln-dfd-c-220805-1125-011-87f69d05 \n", - "11 0.000764 sagemaker-soln-dfd-c-220805-1125-009-d05e188d \n", - "12 0.000536 sagemaker-soln-dfd-c-220805-1125-008-57b3cd7f \n", - "14 0.051138 sagemaker-soln-dfd-c-220805-1125-006-72634ffb \n", - "4 0.000320 sagemaker-soln-dfd-c-220805-1125-016-eaacf33c \n", - "18 0.000273 sagemaker-soln-dfd-c-220805-1125-002-1afaf8bc \n", - "10 0.085353 sagemaker-soln-dfd-c-220805-1125-010-c9d4e9d6 \n", - "8 0.000232 sagemaker-soln-dfd-c-220805-1125-012-e182e0eb \n", - "1 0.000385 sagemaker-soln-dfd-c-220805-1125-019-c561ffd3 \n", - "7 0.000256 sagemaker-soln-dfd-c-220805-1125-013-2eb9bba8 \n", - "16 0.000428 sagemaker-soln-dfd-c-220805-1125-004-75bf8a5e \n", - "19 0.000111 sagemaker-soln-dfd-c-220805-1125-001-a4003eb5 \n", - "3 0.000325 sagemaker-soln-dfd-c-220805-1125-017-930908e7 \n", - "2 0.000338 sagemaker-soln-dfd-c-220805-1125-018-d16dd68d \n", - "6 0.000275 sagemaker-soln-dfd-c-220805-1125-014-958bd465 \n", - "5 0.030932 sagemaker-soln-dfd-c-220805-1125-015-f43b94be \n", - "13 0.005296 sagemaker-soln-dfd-c-220805-1125-007-6e0ffed0 \n", - "\n", - " TrainingJobStatus FinalObjectiveValue TrainingStartTime \\\n", - "17 Completed 2.192 2022-08-05 11:27:48+00:00 \n", - "0 Completed 2.220 2022-08-05 12:58:37+00:00 \n", - "15 Completed 2.228 2022-08-05 11:27:45+00:00 \n", - "9 Completed 2.229 2022-08-05 12:58:19+00:00 \n", - "11 Completed 2.241 2022-08-05 11:27:49+00:00 \n", - "12 Completed 2.242 2022-08-05 11:27:52+00:00 \n", - "14 Completed 2.244 2022-08-05 11:27:50+00:00 \n", - "4 Completed 2.251 2022-08-05 12:58:35+00:00 \n", - "18 Completed 2.254 2022-08-05 11:27:41+00:00 \n", - "10 Completed 2.256 2022-08-05 11:28:02+00:00 \n", - "8 Completed 2.259 2022-08-05 12:58:19+00:00 \n", - "1 Completed 2.260 2022-08-05 12:58:49+00:00 \n", - "7 Completed 2.267 2022-08-05 12:58:22+00:00 \n", - "16 Completed 2.270 2022-08-05 11:27:39+00:00 \n", - "19 Completed 2.285 2022-08-05 11:27:38+00:00 \n", - "3 Completed 2.286 2022-08-05 12:58:33+00:00 \n", - "2 Completed 2.296 2022-08-05 12:58:33+00:00 \n", - "6 Completed 2.318 2022-08-05 12:58:23+00:00 \n", - "5 Completed 2.335 2022-08-05 12:58:36+00:00 \n", - "13 Completed 2.361 2022-08-05 11:27:55+00:00 \n", - "\n", - " TrainingEndTime TrainingElapsedTimeSeconds \n", - "17 2022-08-05 12:54:27+00:00 5199.0 \n", - "0 2022-08-05 14:22:07+00:00 5010.0 \n", - "15 2022-08-05 12:53:09+00:00 5124.0 \n", - "9 2022-08-05 14:23:02+00:00 5083.0 \n", - "11 2022-08-05 12:51:17+00:00 5008.0 \n", - "12 2022-08-05 12:52:22+00:00 5070.0 \n", - "14 2022-08-05 12:51:23+00:00 5013.0 \n", - "4 2022-08-05 14:24:13+00:00 5138.0 \n", - "18 2022-08-05 12:52:15+00:00 5074.0 \n", - "10 2022-08-05 12:51:44+00:00 5022.0 \n", - "8 2022-08-05 14:22:06+00:00 5027.0 \n", - "1 2022-08-05 14:22:26+00:00 5017.0 \n", - "7 2022-08-05 14:22:40+00:00 5058.0 \n", - "16 2022-08-05 12:52:27+00:00 5088.0 \n", - "19 2022-08-05 12:52:04+00:00 5066.0 \n", - "3 2022-08-05 14:23:23+00:00 5090.0 \n", - "2 2022-08-05 14:23:41+00:00 5108.0 \n", - "6 2022-08-05 14:23:07+00:00 5084.0 \n", - "5 2022-08-05 14:22:09+00:00 5013.0 \n", - "13 2022-08-05 12:52:14+00:00 5059.0 " - ] - }, - "execution_count": 19, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "tuning_job_name = tuner.latest_tuning_job.name\n", "print(\"tuning_job_name\", tuning_job_name)\n", @@ -2174,7 +1364,6 @@ { "cell_type": "code", "execution_count": 20, - "id": "56fb51c4", "metadata": { "execution": { "iopub.execute_input": "2022-08-05T14:27:31.544907Z", @@ -2250,8 +1439,7 @@ }, { "cell_type": "code", - "execution_count": 21, - "id": "22f6f2c4", + "execution_count": 9, "metadata": { "execution": { "iopub.execute_input": "2022-08-05T14:27:38.045462Z", @@ -2271,22 +1459,7 @@ }, "tags": [] }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "sagemaker-soln-dfd-c17c2c11-Type2-HPO-m-2022-08-05-14-27-38-042\n", - "\n", - "2022-08-05 12:54:27 Starting - Preparing the instances for training\n", - "2022-08-05 12:54:27 Downloading - Downloading input data\n", - "2022-08-05 12:54:27 Training - Training image download completed. Training in progress.\n", - "2022-08-05 12:54:27 Uploading - Uploading generated training model\n", - "2022-08-05 12:54:27 Completed - Training job completed\n", - "--------!" - ] - } - ], + "outputs": [], "source": [ "scope = \"inference\"\n", "inference_instance_type = \"ml.m4.xlarge\"\n", @@ -2306,7 +1479,6 @@ }, { "cell_type": "markdown", - "id": "9a256546", "metadata": { "papermill": { "duration": 2.366347, @@ -2323,7 +1495,7 @@ "source": [ "## 6. Inference and Model Comparison\n", "\n", - "We will compare model performance both visually and numerically. \n", + "We compare model performance both visually and numerically. \n", "1. Visually, we sample images from the test data, one image from each category, and show the predicted bounding boxes, their predicted categories, and the confidence scores.\n", "2. Numerically, we compute mAP on the pre-allocated test data. This is a fair comparison because we use the same metric and evaluate on the same test data." ] @@ -2331,7 +1503,6 @@ { "cell_type": "code", "execution_count": 22, - "id": "99d9f697", "metadata": { "execution": { "iopub.execute_input": "2022-08-05T14:32:20.182827Z", @@ -2377,7 +1548,6 @@ }, { "cell_type": "markdown", - "id": "09f638aa", "metadata": { "papermill": { "duration": 2.360117, @@ -2398,7 +1568,6 @@ { "cell_type": "code", "execution_count": 23, - "id": "19532281", "metadata": { "execution": { "iopub.execute_input": "2022-08-05T14:32:29.650360Z", @@ -2474,7 +1643,6 @@ }, { "cell_type": "markdown", - "id": "af071369", "metadata": { "papermill": { "duration": 2.402319, @@ -2495,7 +1663,6 @@ { "cell_type": "code", "execution_count": 24, - "id": "c11fcb66", "metadata": { "execution": { "iopub.execute_input": "2022-08-05T14:32:48.589043Z", @@ -2528,8 +1695,7 @@ }, { "cell_type": "code", - "execution_count": 25, - "id": "b8b9dbe9", + "execution_count": 10, "metadata": { "execution": { "iopub.execute_input": "2022-08-05T14:32:53.357762Z", @@ -2549,27 +1715,7 @@ }, "tags": [] }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "endpoint: sagemaker-soln-dfd-c17c2c11-Type1-2022-08-05-08-41-00-890\n", - "output file: results/type1_results.json\n", - "0 / 360 done\n", - "50 / 360 done\n", - "100 / 360 done\n", - "150 / 360 done\n", - "200 / 360 done\n", - "250 / 360 done\n", - "300 / 360 done\n", - "350 / 360 done\n", - "Total predictions for 360 images: 35964\n", - "CPU times: user 5.1 s, sys: 230 ms, total: 5.33 s\n", - "Wall time: 3min 17s\n" - ] - } - ], + "outputs": [], "source": [ "%%time\n", "# 2. Use one endpoint to predict all test images\n", @@ -2645,8 +1791,7 @@ }, { "cell_type": "code", - "execution_count": 26, - "id": "2ba32ffb", + "execution_count": 11, "metadata": { "execution": { "iopub.execute_input": "2022-08-05T14:36:15.996272Z", @@ -2666,39 +1811,7 @@ }, "tags": [] }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "loading annotations into memory...\n", - "Done (t=0.00s)\n", - "creating index...\n", - "index created!\n", - "Loading and preparing results...\n", - "DONE (t=0.36s)\n", - "creating index...\n", - "index created!\n", - "Running per image evaluation...\n", - "Evaluate annotation type *bbox*\n", - "DONE (t=1.74s).\n", - "Accumulating evaluation results...\n", - "DONE (t=0.33s).\n", - " Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.065\n", - " Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.218\n", - " Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.026\n", - " Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.036\n", - " Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.060\n", - " Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.087\n", - " Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.111\n", - " Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.209\n", - " Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.264\n", - " Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.127\n", - " Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.205\n", - " Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.210\n" - ] - } - ], + "outputs": [], "source": [ "# 3. Calculate mean Average Precision (mAP) on the test data\n", "# CoCoeval reports a table of metric values, use the first row result to compare models\n", @@ -2717,7 +1830,6 @@ }, { "cell_type": "markdown", - "id": "0255ceb0", "metadata": { "papermill": { "duration": 2.391113, @@ -2740,7 +1852,6 @@ }, { "cell_type": "markdown", - "id": "c82cc682", "metadata": { "papermill": { "duration": 2.403376, @@ -2765,7 +1876,6 @@ { "cell_type": "code", "execution_count": 27, - "id": "41e7fd6d", "metadata": { "execution": { "iopub.execute_input": "2022-08-05T14:36:32.833309Z", @@ -2805,7 +1915,6 @@ }, { "cell_type": "markdown", - "id": "ffbb292f", "metadata": { "papermill": { "duration": 2.415178, @@ -2834,7 +1943,6 @@ { "cell_type": "code", "execution_count": null, - "id": "7dea0d3e", "metadata": { "papermill": { "duration": 2.382722,