01_perceptron.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "62b217e4",
   "metadata": {},
   "source": [
    "# Perceptron\n",
    "\n",
    "\n",
    "https://en.wikipedia.org/wiki/Perceptron\n",
    "\n",
    "In this notebook, we create a perceptron which is going to help us classify whether an image is a cat or not. Perceptrons cannot be trained to recognize multiple classes since they are only capable of linear classification.\n",
    "\n",
    "This uses logistic regression (binary output) instead of linear regression (continuous output).\n",
    "\n",
    "We don't use backprop in a single perceptron, it is only used for multilayer perceptrons."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "70fcb0de",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import h5py"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b271555d",
   "metadata": {},
   "source": [
    "### Import train dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "631b67b0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<KeysViewHDF5 ['list_classes', 'train_set_x', 'train_set_y']>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_dataset = h5py.File('datasets/train_catvnoncat.h5', 'r')\n",
    "train_dataset.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "0a6508df",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([b'non-cat', b'cat'], dtype='|S7')"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classes = np.array(train_dataset['list_classes'][:])\n",
    "classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "9e6e9f80",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_set_x_orig = np.array(train_dataset['train_set_x'][:])\n",
    "train_set_y_orig = np.array(train_dataset['train_set_y'][:])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2532e243",
   "metadata": {},
   "source": [
    "### Import test dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "7a809c2d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<KeysViewHDF5 ['list_classes', 'test_set_x', 'test_set_y']>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_dataset = h5py.File('datasets/test_catvnoncat.h5', 'r')\n",
    "test_dataset.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "426aea99",
   "metadata": {},
   "outputs": [],
   "source": [
    "test_set_x_orig = np.array(test_dataset['test_set_x'][:])\n",
    "test_set_y_orig = np.array(test_dataset['test_set_y'][:])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45e40022",
   "metadata": {},
   "source": [
    "### Reshape output labels\n",
    "\n",
    "Change label shapes from (50,) to (1,50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "24917049",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_set_y = train_set_y_orig.reshape((1, train_set_y_orig.shape[0]))\n",
    "test_set_y = test_set_y_orig.reshape((1, test_set_y_orig.shape[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df8e1223",
   "metadata": {},
   "source": [
    "### Sample image and label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "915fad2c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction: [0]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD7CAYAAACscuKmAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABDl0lEQVR4nO29aawk13Um+J2IjNzf/uq9WllVXMV9FbVaUlO2TElGawYwjHY3GuoGAf5xN9wYD1pSD2bQHswM7PnRbgM9MEDY7tYPd8uWJVuC4G5bpqS2vIhkUSTFpUSyWEWy1rdUvXxL7hFx50dm5Vney6xkLfnYzvsBD+9m3hs3btyIm3HOPed8h5xz8PDw+PuPYLcH4OHhMRr4xe7hMSbwi93DY0zgF7uHx5jAL3YPjzGBX+weHmOCa1rsRPQ4Eb1ORCeI6MvXa1AeHh7XH3S1dnYiCgG8AeDnAJwB8ByAX3bOvXb9hufh4XG9kLmGYx8FcMI5dxIAiOhrAL4AoO9in5iec3N7b9qxjoi4bOvkNzsX+3zRp49Bhwzoo2+zIY95L8ddVburraMBdYMw4F7oz/1fKLId3ej5GHSCAWjUG73yhbNnVJ1zca+cihdnnKSqXZryZzuKIJACthMl3ZJEuzhOdF332pqtNuI43vFCr2WxHwBwWnw+A+BDgw6Y23sT/venvgcACEKtQYRByIMKbB1/DkIS3+trCki0C1z/OnHTrR4jnwf7cATifOrUZmplHZkTEDlRtv07URbXaRej+GzHL88dyu/NfOg6O4/9xjvcuQAgFMeFEA866XFkxOdgQP/ykTCPh5qDbX3IeyYOtHNP+sarujdefqNX/n/+za+punZyqVdutdq98urGpmpX26r3yhnz7JeK+V45ET8KiXl4cvlyr7y0cknVZfM5AMBrr59EP9zwDToiepKIjhHRsc3K6o0+nYeHRx9cy5v9LIBD4vPB7ncKzrmnADwFAEfvfMhFUQRgh7eJ+NXNmF80/essvjftlCow7C/3ABGWAtuHHFSf7zH4bUh93laAlTjkG9CMo9+YbP9quP2vxb4NpSg54IWnerR9kBz/gLe3etNvE4PEOMTZArPPZKWFfqOU+1ODRHpbs+/g3l55ds8BVXf2HL9ho4iXk+0/ykW9ciHSyy7M8OdASLhhqkX1QvftbcsAEOPKuJY3+3MAbiOio0SUBfCPAHz7Gvrz8PC4gbjqN7tzLiaifwHgz9FR2X7fOffqdRuZh4fHdcW1iPFwzv0ZgD+7TmPx8PC4gbimxf5eQQCisKOTbN8BHrBTKvVjcZw1p/XdLTd9qu4H2Izctq167IiArA65c7kzRlFnVVTVR5/xXqH/vvr8gHNZ05juc2e9eds4BvQxtBluoN2s/3HSRDWsdc36l6hnzpyqNFnqlR/+4EdV3cmvv9wr53PcRz6bVe3imLVqZ66l1W5xWejpWdNHlOGbOFUuqbq1ah1XgneX9fAYE/jF7uExJhitGE/Uc5AZJKoHtk6apJS4b/vfuTyobpsJZpBJqk/dezGvaTVk2DHqdrLLbd5Yw1kYhzYPqrk3fUhHmm3997GobTMB4mpg5OwB7nX93MHtfVftzKDCkK/00Y99WNV97y++wX2kbIZzhUi1q9a4nAm1C1JDOOOsrXPDMGqrdoXidK8sHXEAwHVNdmFo3ZsY/s3u4TEm8Ivdw2NM4Be7h8eYYKQ6O8BusYPcWbcHiIjyIF15gAur3QfY6Rjb/6AAl2CQztvHVGjbDgpiGWgdHNRHn3Y2mGboPYc+fXc+ywAX04fSq/vrw+qz05Fiah6l2+uALmB19D73fZDpbVDY9/4j2l324KHbeuV333m2Vy7mtc4uzWatRPffhHB9JTbRbTW1u+xmjaPvCtMTqm6i3AmSCe0mkYB/s3t4jAn8YvfwGBOM3IPusji93eQlI9sGeGoNMkkpE92AcSi1wMa99/fG0qLvkO3MuQeZzbT3m4wUGz5irZ+5zUaGDZrHfmPa5ikoywOkZ2lu236q/h56SmBXQYDUv9mAMQ7CsBFxhZI2eX3wIx/vld9687leOZ/VfZRLHIu+nuiItTRm9WX+wJ5e+awhylhaW++V56a0B1252PG2s2tHwr/ZPTzGBH6xe3iMCUa7G0/EYsZA4gkrWqsudjzG1m0jpejThxWRg2FFZLkbv525QZT77x0P7eVneui3427bDtpxH5Y/bpBKoh3XrBwvxfMhSU0HyNz9qTwGd6I31q89ial95u595MFe+U+/Ptcrp8mKapcXZBObbb3sZsoc8FIqTPbKtYYObmltshjfaOud+qnLy2rA5Pg3u4fHmMAvdg+PMYFf7B4eY4KR6uwhEkxlOhS7m+mkqlPRVdt04J3NItvNScPpw9KbbDt98ZDRYAMJJ8W5BhE+DNTZxbUMMGttU9GG1PsHeehprzk5H1bn7a+XU1+vuYH+b33hBujbWi/vv4lxNWa4bTA3be+BxV75rnse6ZVffF4TOGWFB10uNc+EoJY+v7LcK990UHvrnX6Xvesqm1VVd2BxsTs870Hn4TH28Ivdw2NMMFIxPkhrKFZfAADUCh8xleyZNNBMNKTpbbDZbOfylet2VhMGkksM6H9Y77ft/PID+lDlq87j9977GNIL7+oxqJcBYxQi+bUb3raPIsrxEvrwz3yiV37xx99X7XIh88wFSUvVheLZX5yb6pUvrq+pdouL+3rluFpRdWk33dQgFcS/2T08xgR+sXt4jAn8YvfwGBOMVGdP23XUznd4tsP9t+q68sFeeZCuPMiUMsg9tF+mz+2c70JfDWydLPeP1ho4jj592H4GZSYdpL72Mx0OnA/bvevTxyAToKnrF3w18FIGWOWoXwScabdNZx1yHIOi3gaa4gRuv+eOXnl2dr+qW1vj7KrFnF52DXFBobALu3ZTnyBiQoyT776rqnK5zl2ME+1GK3HFNzsR/T4RLRPRK+K7WSL6LhG92f0/c6V+PDw8dhfDiPH/CcDj5rsvA3jaOXcbgKe7nz08PN7HuKIY75z7KyI6Yr7+AoBPdctfBfADAF+6Ul/ZbIRDXa+gi3nNiV0h/uxI83f1E9230W0N9IwTzQaZ14b0jNPmqUHt+qsCDoZzTUbEDeASU3MwoO5q0S/903YM8q7r08e2SR3WIDZAZxjyOOmFlxrRXEY/Xi0/3dQse4Xe/7A2Lf/V9073ynljeluvc3RbIHjfXawTMZ8+w6L7xuamqtvc6njUpal+piSudoNu0Tl3vlu+AGBxUGMPD4/dxzXvxrvOT13fnzsiepKIjhHRsUtrG9d6Og8Pj6vE1e7GLxHRPufceSLaB2C5X0Pn3FMAngKAe++62ZHr0OHubz+r2uXbLOasR3qnvpFhXi4KOdD/vXC/oY9oPcgLb6CH3iDJVAWI9B8HzM4pBZkdm233kusvSrZr/IPaFmQHaarP5USn5Zl5VZfNCDUqkeqWPm8UscgZREb1ui7+aqI3cc9sUMx27rq+nYiDdB/pIA66AaK79qrkd+ejH/+4avfs3/y5+LSl6rIbTBHdbPIOvE3lJHfaw0DXra2vb2tjcbVv9m8D+GK3/EUA37rKfjw8PEaEYUxv/wXA3wG4g4jOENETAH4DwM8R0ZsAfrb72cPD432MYXbjf7lP1aev81g8PDxuIEZLOOkcXNrRAV1L6y0Tjs0PRUPWVwv39sob0eFeuZ0zRoBA6vMD+OAHDHFo09XVRs6JA+OmNj+GeZU3qu+pW3U2u2y984aqa7x7old2VbEh2tbnatU5NXA6OavqClmOwoob3K7dbKh22UmO0Crv2avqijPTosz9Z3Kad50EqcP0rPbN6kfE4N5D2ud+nnHO2RsjPfSG09Ft/xJzi3Pqc77EZrl2XRNPlEPu48IWr4NcXs+VTO2UzWZVXWW980wkN0Bn9/Dw+B8MfrF7eIwJRirGO+fQane8guqJ9g6ilE0OuZyum8xyXTld6pXX69OqXb14S6+cFLRYSZQTZfH99igQ0c4EuMjyAK4606H5PMAsJ7KYxk0W55pV7Z9QffdNbveTv1N1ofCgckJkTkNtGnPCVJZsXVJ1SZlTFTWEKSiOtYiYCtNevVLR/QtTXyhMeVbMTInHe9un9DbQwVs5Q2ogRNjtMzpc6qaBKZ6EWL+NaU/ea+td1+dcNh9BXqhGl1a1OuTEvLqWmG9jcy2I+9nIak75qDuSftmKAf9m9/AYG/jF7uExJvCL3cNjTDBa0xuYh7yZaN2nJXJXbW5qHbKQZ/PP3PxCrzyN87rdOn+ubmrTRzzB+p8rMVEGSKfPVa60jYuqLpdnXTaGMPMZzS0V+xFJs6bqkqrQcy/olLzpRoXLW8LVtaFNNUmb9bWoUNB9SN1W8JEj1qa3jIi4y2a062VWXE8Y8XXW23ocbXEtGWMKCoSe7kjopKSjsprCDPXuM3+rx5jhx3Pf0ZtFH8b8Jd2Yh3SlHRS9NlCfNyZdtV8gzjU5O6XaPfDQh3rltYvvqLp6s8LH5biP5S0z3yTcqQM93wkuz7HX2T08xh5+sXt4jAlGL8Z3pYxmQ5sfTglOraWzp1XdZoXF4oM3sTh3/313qHYz0yxmZ1IdiOeEyNlsvt0r1wuHVbuG9DQ7+0NVN3ETp/epVHjqYmN2SjdYDUmN2YxiJi6w0VuBELtDIe5GoRZ9XcrHBalRh4RYvCG85ArGxliSH4xKhZzgOA/5Ol1dc6KtBKwCxVuakKGc4XmcjqVXmE771QKrEPWG7n/rEvOmuyP9kzZT3w9Xh0HkFbCed+hjsiOtGt33KIvxzz/zPVWXCLNz1OY5PX1Re5JeqrHX6f492ns00xXjT59ZQj/4N7uHx5jAL3YPjzHBSMX4aq2KHz33HACgYji06jUW6xOzc3zG3dQrv/RT9iJ6/uTzqt3dh1mMv+tWnQHz0CFOneOavAt+6rX/rtqtLF3olefLeqfeNYRX20XBFdbSv5kqi6vhkpMZal2sRd9EeKshFmJ8rqjaQYi7rqbVhKZwPqwuCBVlU4uERZE+yAaWSO/ANBC76obeLDvHc1oo6TEmLb6HyZmf8jFVQ9zQ5jlIDH/ahTf5uMlFFlsXDx5U7RR/nFWNRPlqJfz3tHPfB4s38fN4+MgHVF3zdVZXApEmamGqrNpVttg6lJrz7pnv3Iso83rfMfg3u4fHmMAvdg+PMYFf7B4eY4KR6uztdhtLF7o68TZdVpgqIh20n8kd7ZXvu/v+XvnCJe1p9xenOBrshbe1l9Ide/jz/XexrpmF1pvTeoWHMalT+Bw6xHsHlSybtS6d0p52kdCn0rr2oNsQaXhrRW2GogIbxErC9Jat6QgnCD03besIQSduaVJn/Thp6euMxRzrHRKAhLktjXjfopnROnUqPBbTLRNVJ9Tcuojqymf0fY+Enh6varORHPGp55/j8Rkz4uLB/aIOI0W/SLrUbHCsXOK9lXseeVTVnXrrhV65Ku51uaj3jGTUW31T79VsZTvjSBJ7Nxn+ze7hMSbwi93DY0wwUjGeiHrBDZbfOsgI80mkTQ6pyBtJwptsenpa9595oFeezGt57tWzzM32cEmY6G65R7Vb3MvmqoxJQxWJoJAssYgcX9Aef9kC929T+LRTnvJgr+bHz4mgluQ8exTGxhSZZlm8a5sUUrJlAUL8z2uPrktCrG+awCNJdCHvks0sFIrglyCjAzMyoo+4xNe8ZtS3OBAEGyaYZkqY0ZpneI5PGBIN90EWi/fedEgPsl862RuMwKgahw6w6XAyr5+rPd9jU2KjyR6QOfPslIqCAKOixfiVrtITm2PUmK40aA8Pj78f8Ivdw2NM4Be7h8eYYKQ6e7FQwEP33wcAOH1OE0+cXWJ3znOJ1tk3iF1pL7zO5jUTJIWFGSYMuGWf1t0CYl38yFE2cZXLWqdL5jmvXKOldcNqTUTOiQgkF5uoNEEskFpGy0AQQq5dUFWtNda3XIv17XaqdfakLXOD6ejBQJh8aEuMy3h8huKyM4ZTPhF7FZNCx7bEEIk8d1ub9qQLbizG1DaKf1WYDpvGbNQq8f2MUh5HwUQSviXMX9bV+sDNbLbNmPxoV4Nh3WMtpJv01Oy0qjt6x3298qU1XhcNE404PcGfL64bYoukM6+DMuwNk/7pEBF9n4heI6JXiehXu9/PEtF3iejN7v+ZK/Xl4eGxexhGjI8B/Jpz7i4AHwbwK0R0F4AvA3jaOXcbgKe7nz08PN6nGCbX23mgQ/bmnNskouMADgD4AoBPdZt9FcAPAHzpCn2h1Y30qjf6i58BtGlicS97SC1dYG+1yckJ1W7fDIv/71zQ5BXLq6wmPHucI7QevkOnKy4J80870WNsCk7vqohQa+QVFQRaQkZOMvpaZGRXWtNjTET/qRSLjYgcCLG1bM4dzYrovryIRMtob6zs2mqv3Nyq6HFMM39fXo7feOutr7NaU6vpKEaZySorrrkQac68UPDfXdpcU3VVwckHwbueWg/LC6wOvfWsTgUeCxXl8O3MQxiG+tEfmBLsOrvlBSYV88cf/2yvfOAIq5uvvfyiHsfz7Gm3elFHD1LaeVYHpa9+Txt0RHQEwIMAngGw2P0hAIALABb7Hefh4bH7GHqxE1EZwDcA/CvnnNohcR0H4R33BojoSSI6RkTH1jdrOzXx8PAYAYZa7EQUobPQ/8A5983u10tEtK9bvw/A8k7HOueecs494px7ZGqiuFMTDw+PEeCKOjt1FJbfA3DcOffvRNW3AXwRwG90/3/rSn3F7RZWznVYYiYK2jWyHnME2NKSMXkdZ0aafMh6eVjSumw4yb9de6a1Pr+xXumVXzjJ+uV/fV5HrH3+YdZGHr1d68OSdaYm3E23Uq3bQ0gwkYl+ygr5R2ueAIn8aNI3NSlrDvxEuKaWjCmrvcj57lZEZFRo90FC1uGDnL7OgnALJqGnJ8bWmeane+XWonb9RUYSVbJuH9Uqqpnk4i8Hel+hLoTFeoX18vU1bbZNxR5GaX1V1Z0U96kt9oluvvde1S4S4x21g+3snnlR/kSv/MBHPqra3XPspV659v/+uqqrVjt7Uqfe7f/+HsbO/jEA/xTAy0T0Yve7f4POIv8jInoCwDsAfmmIvjw8PHYJw+zG/zX6/9h9us/3Hh4e7zOM1IMuCALkuwH5b5w4pepeOcUeQefamiwyyLCY2S6wuO9qWgQ/JUgVMwVNDJE4Fm/OS8+4UO8jBMJEJdMsAZrYMC/E54lEi9Kt0jSPo6n7kOmftkKtyrTFWIplLk/OaPNgKrzEGkI9AYAtYQ5riHO1a9pUk2uxSJsz15kKUshIRNiR4UIPiMfRrJs+ItFWeBSWjXktqotxGE72eonvYSS2l+KL76p2NUE44nJafaNzZ3vlN7dYfauaqLHJeVaVDn/gdlWXlWrNCNkxwoxenrOLnPpswXiIrp7rXE84IMrP+8Z7eIwJ/GL38BgTjFSMr9Vq+MkLnR3FRl3vpF+qc9DDkqFcq9V5h5UCFr/KEzpT5vQEi4RTRe3RNVVmVcDlWDTNmECVcig42Q1ZQ1tkZG2kgm/M7JaXhAdaa0uLi7U2H0c3a+KMnPBWawtCjHhZB8xUKqyuNM3vdSrEYilx5k0QSFsEnZSMaHpRqAIl0X1xala1C8E79XkTrBNGYvdfiOdtk65KetqhpfsoCG77NnjuE8NjF9dYBVw3gUHNEt+L8jLXnXUvq3ZnJX+cCQy67cEHeuVw1CR3AvsP7u2VP/OZz6m6b/7B2wC2k2ZI+De7h8eYwC92D48xgV/sHh5jgpGnbHZdM5U1EcwXWVe+aKLB6jWuSzLscbXW1AH8lYusD+aK06puYpLD7aeF997ClI7Cqq6JqLR53cfyMvOanzgn0jKv6U2GCeHFNRloz7U4FmmZLee78PCKq7znkDR0TEEhK8Zc0EQfKqLKsYmRnCUi5Ou8ZExqNWE2y7XF3Jvou1CYKYuRvs5AfJYc8klOz7cTZsu0ofdZwiKbH0vClLqpt2rgKmwqDM2eQH1T5Ecr8Z5Da0nvg0R5Htf5t7Vp79AHODV40eS0GxRldr0RiTldEDkMAKBY7lxbEPZf0v7N7uExJvCL3cNjTDBa3ngAmS5ZgbVg3DTJIuxCwaRMWmCx8qIQac9v6N+qzZTNa+tVzYW+WmERfO/t7H300O06DH9xgcW502d1CqlXX+YUwktrInjEeMld3GDRsRRq8TaXYTGQ3tYBP5FItZRvsshcM1zrDaECtVffVnUZYQKLJM+cmW/5sd7UIn6Y4TnI5jlcx5mgGxJifWS88MJAqBCBCIox4n5SZVWsbR6KptAu4gabMEuze1W7Yo5VjUuViqprXeSgmUqL1cNMXqs/c+CTrb11UtVVVvl+Fks6aGi3ML9PP7f7b7oZgPZ4tPBvdg+PMYFf7B4eYwK/2D08xgSjNb0RsWnI6Gck9NIw1LphIcuf9xRZt79tRptZGoIgsgEd9bZwgEkdPnQvkzLuX9CRZ0urHI337imdwy0WPN6Te5iPPGkaAvuWMAnG2jzYlLrtxhlVlwqzSSLy3RVtumVhUgvNuYvzrM+2hP5WNb/rRWHymqro6MGM0KsDEZkn0xMDQJqyrr9u0krHwowoU1gX23o/JhS6csYQm9VX2DwWC4KQ5pYeb3bPER6v09fpRCShfOKaGzr6Thp7g4I2r0WZES4TkpNgmd74ChYWdSTkbbd3yDhyxrQp4d/sHh5jAr/YPTzGBCNP2Rxc5gk3RAUQUWRwxhtLiI8kRBvrhXeLICA4dOigqisLjvnEsXnqjTdeV+0urVV4GIaUwgnWOEcyrbFNfcRj3Gjr39P1dRbBD87rJDr5HIu0bWFqWm5q4om8OLflYW/OsVnxvOCMa7X0tRzOS7VJqxpOvANSeZ9MOiInVC83p8kUQuGx6EiaCrV3mls91ytHgX4cc4L0ojbNxA1t62m3LiIL23qMobgWGRHW1IFtSDa4j9PGs/FvfvCDXvmTn9PRZrMz03jPoB2JmLsQz7pRdaUaZZwecc8jDwEACqX+pK7+ze7hMSbwi93DY0ww8kCYy7vwZILsZUDBuZYWCSsxi4QzmUqvfM8BLbLceiuLeuWCvrRzF9iTammZA1XqdUMDLcaRGrF1aZXPfUoEv2zGWqaqtlNR1nWx2H2evrii6o7O8Q78oWn21Jowaa6cE951xnMtOftar5wp8o5tbnKfahfHgmLZBuSEQjwXgTX2nkmLSstw0JEgCEklf5xR0UpCHqWMnquMCLQhEdiUN/yCrc2KGJL2SnSSIlpYLiKblkvc6/qGVhNe/uF/75UDE1D087/IpMp54W24neNCPEsDY2f6Vw7iv7vp9iMAgGzee9B5eIw9/GL38BgT+MXu4TEmGKnOLrM/WuODE/pgmtVRTXfc/sFe+ZJIrfSX57Qn1fE66zQTkTYnSfPMZMA6GZmRpMLc1jLeae06m8Cq4ly5giacDCK266Qtw4XeEl5nde1N9uPzfNzLS3yuPWb/4dAU64ZH9kyruoITemmD9wTiuk6LVBe3vh3rfYtcwvpsPif00ECb76S3WmqILdoiai8QJrrY6KSxYPWMjIdeRngU5sXeQT6vzY21d9h8ms+alNA53geJBd8+GRNdRrz38lm9F1Rc5DwG9zzwoKqLBNEHBdJEbDxExfNtVW/ZVqb0djvnSt25/64ZetB2wBXf7ESUJ6JnieglInqViH69+/1RInqGiE4Q0R8SUfZKfXl4eOwehhHjmwAec87dD+ABAI8T0YcB/CaA33LO3QpgDcATN2yUHh4e14xhcr05AJdlyqj75wA8BuAfd7//KoB/C+B3rtTf5YybVpSJ8mxq2jOl0z/ddtOeXlk6PsWhNjPsn2Lh4p0lTV6xtcaeW3dOsnjrDDl8GrOYHbe1GScUouSdR9iUlYbaFHR6mftPjDmpGLJ4m8mbDKwx87U7YaKq1vRteq7G43rtohbBFws8sbfOswi7f1rPVTESfH2T2gxVXWUVQqaryhkRORIpaXObWk2IW0JFEWI8GVFdqkqZ0LiFiUCbrOi/2TDub7Mis2/N3DOheqWCKz8V6kmnU74vYcsENglPxNSMMQh39tCz1zLIbCY943S57yED++iHYfOzh90MrssAvgvgLQAV53pGxzMADvQ53MPD432AoRa7cy5xzj0A4CCARwF8YNgTENGTRHSMiI5V6+0rH+Dh4XFD8J5Mb865CoDvA/gIgGkiuixfHgRwts8xTznnHnHOPVIqRDs18fDwGAGuqLMT0R4AbedchYgKAH4Onc257wP4RQBfA/BFAN+6Ul+NmPDGakd3dBlt3ggL7BK7IiKhAODHJ9hstiBMTdW6dl0skyQUt/o2lyW/YpoY90pherO53pzggA/F+DM5TWTeFKmMNxKtu9XrbPbLtbSL6XSRf3vnS6yDJU4TLVSkmdJEiq02RGrqM9xH2STQOyz2N+4+rMkL8xFfeHur0isHhlgziPjcGeh7EaXCFCci55yZ75YwNVnahVjMjzRrRYYgk8S9duaJDkm47Yp7Gxi9PE65j6aJyIxX+D32V1//uqor/7Mv9spHbr2V+w/6v0etfi31eXlcagg+h+2jH4axs+8D8FXqJOcOAPyRc+47RPQagK8R0f8F4AUAvzdEXx4eHruEYXbjfwLgwR2+P4mO/u7h4fE/AEbqQZeN8th/8DYAwMSENn04IY7WlnVdZYtFsXaLxZf1TS2KXYjYzLJR0yapdovNY45u7pXzgfZiyxH3EYkyAATEfYbC7BcYAon5OTYVzkWaZ/xdYdbaX9YpfM4tMRNaMcemQ0qMiCY8y3KR9mUKBF97o83nqiR6HG8IzeDEJU0oMZPl8x3Kslh5MKvFyj1iWNmCvmeSxy0QIqblnndZVnNsquRURKY1hKksF2hVYEZw7RWNCD4l5PqzZwVRhjPqW5vvrTN9tMTW1urpE6ruh3/2pzzeX/ifeuUjR25R7QJBtGJF8H5ms4HkFQPq+sH7xnt4jAn8YvfwGBOMVIwPQkKx1BE7N7e0+LwsPi4ta/KAOUEVHIo8RgszOoXPRJHFuS1r0xf0zq2EZdiNVIvBFcfnKgVajJ/Isag0kbBaUIz1b6YT6ZMkFTMARIKQ4cCCJqWo1VktyUVcDlNDR00s0uYnplXdSoPF4iThSZ2ItJg3V+Ad7XZVBxS9uspjfktcyyGT8fZTBRaF97QMCYjIsOsCYZGwu8Zidz4l/ThKEo12vcLfizIAJKJdMq0tC5NZnuOMEKVny/paloQnXxjrZycRhHWVQKtDr5883itvfJPn8UOf/AXV7u57H+6Vc1n9TPQTz4cRzXc6rh/8m93DY0zgF7uHx5jAL3YPjzHBSHX2tY0GvvG9DtGANW/I351iQeuymGZ9cDMW3lI2xa8gUCDDzV2aZnNYJPT0dkOb7yhlXa5Z1Z58GUF40HLs8be8qT262k6kNDL8f7Hwfls3wVWSjBKCHiA0ZAqhMMVFWa1DHp3n61wXHoar61rvnxdTPLOg9z7Oi7RU+8WpZ4y7c1mY0Zp1fTEvrbP327kNLk/ldR+3iv4PTWsefZkXIFvh6LtGUV8zEV9nxpBiOkEQWZ4U99aQSsroxzDQXo9O6PC58qyqy08zqefSCptL//ovv6HaXVxhwtMPfuQxVTc1qT0wL2MYPfy9wL/ZPTzGBH6xe3iMCUZrekOKMjoieWTJAzIiW2hG/wY1qyweFXLcrpC1w2eZOTbiXDtkEa5KLDumJd1uUhwXlnUkb0bwu4UiO2hi+NdczOJX0wRtbNT485lLWtXYSvh6zgiPvzIM97zgWs8FWk/I53hcecFdl8vqds1apVeeKWsOvQfmWORfzLB4u2mIOKQ3nAv0tcQisKlSZdG9FWgxfr+wQiXGgy4S9zojiDPiCS1KT+S4z5K4LgAg8YhHBREU0zbZb2NWDdY3tMoDcZ1lp8dYEKlnc3sO98r1pjYtP/8Mc8/DpC372M88zuMo9M/Ceq3wb3YPjzGBX+weHmMCv9g9PMYEI9XZUwpRDbpmhkCbkxpN1osaa9osQhVh+iiymSgyuo/kGS+ZKKys2AfIC92zkNeui5mIj0uMq2sidP22/J3cxlXO5aJh8s6IYRUM+XZTEFwm4PlYi7U+3Ej43FtVrQNvhWymzAs9PU1Nbr08671RQUei5QI2vaWCu9xksEZLfFE0rp23HmAT4B333s3nivR4199id9P2ps59F0by8eR5DI3ZNiiyHXHz7ElVtzAtCDyKYq8m0Tp7ucQ3ZrOp92DiWIxjo6LqWm+zbh7dxOQVaVabM6nI/ddb+tk8/jpHHd79gSO9cj6n91kkp3wS672guPt5EOGFf7N7eIwJ/GL38BgTjFSMzwQOeyY64sZmS5s3GiKyKAqN55AYZXVtqVe2RAi5EotzF1c0XxqE+So/wR5LNo1ORhAmFI2nVlaIoHlBupA3JqlIfA4zJupNmJOc4Y8rCk+qkKTYavjdRKRYmLGiHoutFzfFHJi52pKegy09/kZ2uleOwe0SYzZrgcXdQqzF4uoai+TZAou05eKCarchTJPThp8uJ65Fcs9HMBGNeaGW7dFRb9RgMTsW4nnGpLyaFKbDdVNXjfjcVUN6Uazzc1xK+LhsXucSaAoSDZur4K03T/XKJ376Kvdd0Kru1pbg8zc6Va2bSuzSRc1XKOHf7B4eYwK/2D08xgQjFeMnozYe29fZ6U2c/p1ppCLdUVvXbTQ5CGIrEZ5wsR7+RoMpp2uRrmuJ3ehGi3f762bnNRQeaJsVvTscSO43IZrGse4jL1JZ5Y1HVCmfF2Ut4ueyPAckVAEr7mfkjnZgveuEuCuoqcmIyCWhXmSLWpU5e4nnZ0uI9JYeebrO811O9Rzkmixybp57h9sZEZbarGo0m1p8LokMqbHwrmusL6l2SaHSK5tNcEQieKcUsJoX17TFpynee+GhParuniY/BydObqi6tQl+XqR6layeV+1Ks9xnSMZTMOI+fvzCsV45YzxEnVBryiZ45nIaqsTvxnt4ePjF7uExJvCL3cNjTDBSnb0aZ/DscifYvxBp3UKSORYyhp98gs0zB0PB6x7odtKhzpk9gZZQWRsiJVPNEE5uNFj33Gjp6am1+Xxb7Qp/b7zHYuHdtLWmdbxNofcTaX0bgnAxEnzwWcMNnxNRXtbLqiTIDPN5rssaksNcxJO1afZIJuf398pOjjc1fOdN9rRzxiwXiH2MEDwHF0+/odplRRRZO9X7CrH0GBO87jbHU3iej6sVdd3FDPefKQjiSBONmCvx3kp7XY9j6qBIfU1an3/9El+bJDmlnJ7vYJOf23PvvKXqyrNHe+WZWY5AXFnRen8g9mfahuAzb8x0O2HoN3s3bfMLRPSd7uejRPQMEZ0goj8kouyV+vDw8Ng9vBcx/lcBHBeffxPAbznnbgWwBuCJ6zkwDw+P64uhxHgiOgjg8wD+bwD/C3XIsR4D8I+7Tb4K4N8C+J1B/cRpgNVmR9wgp01SriVlcCOeC5NPmBGZQw1hghTrI9KiWDbkz4WQxa28MYPkIm53uKx/C4sZFk1zAXuMEfR4Y8FBV4+1qL7eFN5YiZ6DDZGBtRb376OxwXNVNQLVpZBNe06Y4bJZHXxREGa+QqT7zwqzZVaoCZHhRNsQno4ZQ6KREWpDSabQJft+EebGou5jQxznhAkzMaZOEgEus0U9p7OCsCLYZJHbej2mW+x5lqtpEf/Yy3x/a61Lqi43w2bWrEiPVTcebk0RzHTmnDbpTtT4WXKCXzCTaFE9J6KoEkPSUejOXTCAtm7YN/u/B/Cvgd5TPQeg4pjN7wyAA0P25eHhsQu44mInol8AsOyce/5qTkBETxLRMSI61m61rnyAh4fHDcEwYvzHAPxDIvocgDyASQC/DWCaiDLdt/tBAGd3Otg59xSApwBgYmp6+Hw2Hh4e1xXD5Gf/CoCvAAARfQrA/+qc+ydE9HUAvwjgawC+COBbVz6d60VfBUZ3SxPWnZ3JteWEIkIRuwnGpl0s3BUbob40yvFxaYNdNFNjoqu1Ra60UEe9TeVYtyqIPYAo1PpZVkRlZU1dLhR9QEs6c/Osb5YyrK9lzP6DJCgwvBZoCbNiQ+SgqyV6PuoJ67IbTV23vsF6dCTMgwukIwkLWb6WKZMHLpT7LkIF3saFLu5tYO6ZSnOc42upG2KSvJhHaugxLos8AEURFZnJ6HOlgpRiMqd19pV32QSW1nX/2Sy3jVf5vk/vP6ra1RNulxgT4+Y6n7stnseC2VeQxKZtQ855vmsGHSQ9X4tTzZfQ2aw7gY4O/3vX0JeHh8cNxntyqnHO/QDAD7rlkwAevf5D8vDwuBEYqQcdQKBuBBcZfjcSog2F1rNMRG/J44yJzknxyKgJJLyP5LkCYzaT4l2c9I/QSsTUOTKc7EK2ppxWBdIWkylstY3XWZNNSBNCRLbibUaoCYEhtig5jhB8ZI493Gan9RhlcFQz1qL18be4z4sVYboy0XdrIYvFpUTPd1OQkZw5z2Jw3NbjlVpIZDwFbzt6pFcuCBNaYqLjNjfZbBYZwhEnUmAllQqPw6ZDlumhTeqwwrxQHWt6HmUKsglhfjw4raPS4jzf20pNc8pviPTlTaFuLZR1CF9emKDXTdRepnvdkRH9JbxvvIfHmMAvdg+PMcFIxfgEATa7XmNhW++oBsITLDA72DIjq6T1DWCc/wPuw+76Bon04uJzb3M4EuMgE3ABEezhxC41mSAQClh0DGywizhjZPjpUjHmjBhYaNQaqYakpOexLqwaqeC0zpmAGSmo5k2AS63NnmaXllgtmN0zr9plhfdhuH5R1V0SAUA/fIvpnduWXEHQQheN6lUQovbBueleebOu+Qt/epIDSz55r07ZdUIQceSFGpY14r6gsUMambkSc5o3NNZ18Xl1k9tN1HQG4EIg0nll9fNSmOV72GjzM7e+odWVQplTaqXplqrb6loTEkNSIuHf7B4eYwK/2D08xgR+sXt4jAlGm/4pSVDf6JhJDs7MqLoJob7GJlVRLMxtFzZYj1ucMJzswmzWirVuuLbF+tThGTaLJMZ0dXKZTV7Fsu5/Zop11poyIWmdOgxFimKn9bNsXtTVTNRen7TVCRlvQGGCIfN77Rx7UAWiP5t2CUIfzhju+WKJdcWtFkdozSZ6vOkm68NNEzm3VheRXIL0wxJwBsLklTMmxh+fv9Ar/3St0isfXdAEElttHpezqZuECXB5k3X9uVjrtlXh4RaZ9NaNJl9Lo62Pkzp7LFJIBdN6T6BU5j2MrDGP5UTUXpIVxJTGU7Al8oWlW/rZbHTNd86aFAX8m93DY0zgF7uHx5hgxB50zA03ldXiRmudTRWhkURm9x3qlZfWWIxaDLUn0sIsi+fVtu7klTp/PjTNIq3ljT8lxOCi4e0+Os0i20qFzTi1muaZo5jFxWJJi5z7Z1iMXV5bV3WLUyz6HZxkUa9iyBTKgm8sNrewts5z4lLuI2PMd5LPbGJKq1QgJmhoisCSgiFTCMW7opbRYmskuNSStzlLabOmzWapMPtFxpvx/r2sNi3OsLfegjEBvvnWiV45Nl54s4f42bnwEnvaHTHeaeeFqfDWed3/2+cFT31Gm96KYl4rGb6375zXz0RxgsdvyU6iiNWEVnu1V17d0HN1x50P9sqF4pyqu8x7n243Jvfg3+weHmMCv9g9PMYEfrF7eIwJRh/11nXhTA1p4Jsn2OVxVug3AFCcZ3o7GTW1fEHn/FoQet3Guo4K2trk49YrfO5swRAlCrOZg3EjFcSAdZHy+JXjr6t2cZV1w9vueUTVkYg+C81PbUukFyZhoVpb1Wl4S3t4byJf0LpnKjY8ZERWxpi1CiXW+7M5bZZbXWNXTLmnkTNpmbPi8aGGnm/UWNeX0YitWO8/SBNgbFNwCzfpQwWerFKs92pS0UfVkDdk5/dynTC3hSZH3mZLuDgX9ZzWHT9noTFtBTK/W4X17QN7b1HtLon+U0O6Uq3yXEUlwftvIhUv1XgvZbGo91k2uwSuXmf38PDwi93DY1wwUjGeggBRviNunDEieK0pPbq0mLO2xeKjFFOWL2nT1XqV251d0aJvo8Ui0Wsn2DPr9jtuN2MUpBRm/OdWOLLrzAUup3YaQ5bBz6/qMU4XWWQOQyOmbbD4PCn419ar2uQ1KTy1IhhSCklKJ3njTTqiiSmOwkqN+BwLcbchuM7qRkSWEVazE1r0PTTBprhSlkXmlYo2SeUER33ORIPJfAF7J1lFK5R1tKNMbbVV1SL+YWE+TUV/gSFYT4UJLWnqPo7cyZF002VtYtxa5+tZ+slPemXLIZER6bbKE9pshkR46CU8x3FTq001MXfNrI4yLOc6z5y9Lgn/ZvfwGBP4xe7hMSYYrRgP6u2CVi7oDJWtKou7JnYE0vnLCS8uF2vxdrrM4nPa0iJQImiVG5u8azqZv0O1ywgxKGe8pc6/yRaD5YssUklxEwDaNXktOvCjPS/SIhV0Ep1lEVgSimAgSrRaIznoilkTCKPSQfH4J6cnVbucEOtt8ITk/aiJwI+lLT2nDSd2oje1t9dknsXdoigTabUmdiJYxxBDvFNhtea4UNHuOHhEtXvgQx/tlZuGSrqW8iN++J4H+PtJrXbcf8+He+WSsdDcffRwr5wzK6YussFWC7xDHtc0uQS1+LrnFjQ/3eYyq5WbdVapqlVt4ZgSqabW17QYfznAKE2MtUPAv9k9PMYEfrF7eIwJ/GL38BgTjFRnD4IAhWLXbJQzHkzCy8pGaIXSEyxl3a1Z1yaSROorhrxQenHlBJEDGX3bSQ8kw40oU+5INdfqSZK/MWNIIzI5YTYz9hnZNpTlrNbZG4IMIusscYbQ2QV5RWjILaV2bAkt88JUtiHOdWxFm81iCCIHY/K5ed90r/ywiNZarmg9tB3O9sppovXtUxt8fz9ygNMpbZhNnX3z3EdiPMg2tngv4cAcR7NZA1UgSCybJj3Ty8881yvnjYnxzgf52h59lMsrK1qn/tGPftQrzx7QezVOpHxauyS88Ip6n2X/4SM83ow2P4Zdd8ysJSkRGDY/+9sANgEkAGLn3CNENAvgDwEcAfA2gF9yzq3168PDw2N38V7E+H/gnHvAOXfZ2fvLAJ52zt0G4OnuZw8Pj/cprkWM/wKAT3XLX0UnB9yXBh2Qy0W4+cg+AMBb6++ouuKMIHyYmlV1ifDokmJKOLNPtVuPBff8hCYgKEoPpgKbPi4l2iMKIY8jMR5ue+/5UK88lwrVItLTmBX8cTCmt5II1qmta1PWQpGve+IAj/HWQ1qsnCiyKpAaXvpIZHzNtpd75bbhTssLNard0iZMpQEJ8bxt1BoSqoEVby9c5KCNomhHhpOdRIDOpOF+22rwuEgMqjil71nzIovMkel/ch97q5HwiUxtQIswI7bq+r6kKY8jn9PXublV6ZX37r+J+wv0ff/0Zz/XKx+5Q5t7YxFg9e47p3rlhX1a3N8zx0Qo/+E//K6qO7uy1B2PNoFKDPtmdwD+goieJ6Inu98tOucuG8svAFgcsi8PD49dwLBv9o87584S0QKA7xLRT2Wlc84RkXUlBwB0fxyeBIDixOROTTw8PEaAod7szrmz3f/LAP4EnVTNS0S0DwC6/5f7HPuUc+4R59wj+UJppyYeHh4jwBXf7ERUAhA45za75c8A+D8BfBvAFwH8Rvf/t67UV5ICl9NXHXnoZ1Td7cI1NWtYHZaX+Hdk/xzrsvN7P6baydTG+ya1vpMV0We33LK/V05NnrNClckRs3nN23308P29cjzALTEnyCDqRv8rlViX22hp80wsUidLKSgy+vakIJ5omhTIgTCHtes8j1NlrUNmBWmlK2qz3IED7B46PcW88TZ1dCrcWyPDPR+LFM4XqqxHlqb1fkwc8Atg316tCS4IYtDVS2zomTd8/sU8f66sa/Pd/gXe10lTnitLQhEKXb/lKnqMddbZN2q6/603j/fKeTGntYaOELz74Yd65QP7D6q6pogmnJpml9tGXZtc/+ZvXuyV3zyt3c0vrK50+9LHSAwjxi8C+JNuosQMgP/snPtvRPQcgD8ioicAvAPgl4boy8PDY5dwxcXunDsJ4P4dvr8I4NM3YlAeHh7XHyP1oHNwiLsphIqGt1umAQqMiNUkFrGKoSR/MKmSpfObSVUEQdAgHaTiRNuT2i2ujCMtPjdFhFNbcqkZVSAWnPVVQzzRanCf6xcrqs4Jb7gzbwtTkCExWBIeb5mM2XYRaYkLJZ6ffGlBNZMmrzb0vfjZx/g3PJy5i48hPd+pmMjypN58nRW88ZEgpSgVtedXudSfD14GHT77t3/dK++b0+d6UdStLZ9VdYFj9S2+yOVcTXtfxiIKsJbXakKY4fGvndf386aDPK8bFVY3Z6f0fEeOxevKRe17dvKt01w+daZXfuvdM6qdTIOdGMIR5/SzuhO8b7yHx5jAL3YPjzGBX+weHmOCXcj11tE1WsacBJli2ejAtRqbJjIh68pbG5oNRJrRKus6QmtDsJ6cOc26VbOhdbAN4XrZNvp2dZVJMhPB/e0MnzoEs0zS1nVO8OUnbW3GkfztbUGJUi5ps1lBsKw8eO9dqu7woSO98kTxPj7XpNaH5ZZGs2XccQUf4ucfY7JFIh0rlhFEj9mcSfssuNdJcLTn8/paMiLNNkz/75xi19Eo5Lp3T5xQ7dYES07O5K2790Of7JUvvMG+YMnKOdWuLqIFZxb2qrqJKZ6Qn7zyE1UnIyhrG/zM1U10X22T9fTXzmqX1soWP2ex5KE3acezeZ6rIzdp891sdx9j9dwp9IN/s3t4jAn8YvfwGBOMVIxP4gQbXU+oxua6qWPxPGlr76ONdRaBNgSv+znD7J4IcToxkVxS7K6c5O9dos8l0+mGxoMuLLCoNy1E6Wnh1QcAMzMcnTQ3q8XKWUH8OGUixSYF1/qk4CfPmFRFP/oRkylIPnUA2L+HPdTWhSrzu1/9E9XuscfYg3H/IS0Slssskl8Uas2f/9l3VbvPfp4juRb2aVPT20IE/9HfPdsr/+zP/6xqt0ea24wYX62zmvPuSb5pG6vavFaa43Mv7teekyQiF1tZntOq4W6//8Mf75XtfJ87zRGauZw2HbaFqB0L1XSzqlW0Q0fYk+9//vAnVR2JyEVpUkusWViQp8gyAMTd9fPm8ZfQD/7N7uExJvCL3cNjTDBSMT5ubGHl9b/rfHBaRJFpa0LDZxYJcghJXlEoaDG7MMk7veWyFtMmRbqjmRkWrednp1W7PYLPbHZai+cTJcGFLnafI0NeQcID0JnrjMQONhmePMlBJ3etbdDNbXdyyqqFOR1YMr/I4mJW7IjfcrcmTFg8cKhXnjRkIRJZYZGY2rtH1UmrQGqusynUsksiJwBMJLQUW22MdCRSN22KgKLIRE/eeQ9bHR796CdUXUNk2/3hD5/plf/22DOq3d7DbHVoGAKIt99hMf7M+QuqrnCYn6Vp4YW3b3a/apcRxCQ2VVaq5oCffRlY04F8XvRsXeb+zxurSL+jPTw8/h7DL3YPjzGBX+weHmMCsnm+biRm5+bcz3/+swC03gwAc0J33jOv9e15Yb6SJimrn0hvrIzhg5fc85J4MIq0mUWm/5UpiQGAxF6C1Lfzea1DZkSUlOWN1xiQXleafwak4bXmqkBeZ58yoPVEm6tONpWPx3t5VpomvfNlKI85e27Tf3WLvR6/+cd/zMfEeg+jWOL9GEu4uLLC5Buv/vS1XnlZ8LMDwJTYj7EprOWoiiWtR3/wnlt75UfkfkGk2x25+eZeeWZ6WtUFIooxFPMTGDJRfa/NM9Ed5Gd+/nG89NJLOz4w/s3u4TEm8Ivdw2NMMFLT28L8HP7lE/8UABDatEsq+N4EXAjTlhLVDSeaFIFs/7IPKSKHlldNekQZcVEOyw0QkeVvaGyCeqSoGhjTm+IyF6KkTc8kr3ObvCb6sPx6EjLgIgytGM+9KrOQ5VoP+r8r5JhlHw3jWXbhHAcXnT+nzVrHX+PAlRdeZs+wlZUl1U5mem4Z70tt1hLjNyLyek1y1KsqlISIXyppc2xdEKvcesc9vfLktFZTpcl1G2e9mCupHm7TmgaqVFdWsfyb3cNjTOAXu4fHmMAvdg+PMcGIUzYTit08ZZlIk/plBSlhYHRUnTpZKFRGuaIBpiypviaCELJl9HInzW2mu7bk5Ja6lTHVRFF//Uylo3aG813owPK42HDDE7FeGpq0z5Ew9clTJ8mAc5nxy3MrY4+Zb6fcgo37piT4FK60z7+gyR+e/THr4hcvaR79zSqb3lJxXdOLOsdfUxBy5o3uKr1z5R6MM/sZYUaaUrVJd3aO9e8HHrxP1X300Q/2yoUi6/Z2rtT+j9kjcSq6rf8eify4bZ+o+9EN0N39m93DY0zgF7uHx5hgpGI8UYCgSyYgUwcBQJxK3nh9XCL4yaVIu810JcqBFfFF26CPJxygTXu2fynUkhCDraedNGWFA1SL2BwnxTYK+/8Oyyg4OTcAEAtVQ02xmVMpBsaGCEFemzShWROg0qgGmOFkw4cf1vlG7rnvTh6HUalaYlzyvicmwk4eZ82NMt13W8yb5eLPiYi18oQmBJmZYnNbwRCaRH1Ur5aZ04xQU8maRANBWCHvp20W9F+uPbVpgAVuqDc7EU0T0R8T0U+J6DgRfYSIZonou0T0Zvf/zJV78vDw2C0MK8b/NoD/5pz7ADqpoI4D+DKAp51ztwF4uvvZw8PjfYphsrhOAfgEgH8GAM65FoAWEX0BwKe6zb4K4AcAvjSoL+fQS/9kffzbIjvmdpFQ7PoKEYhIi3MQx23rQ5EmCBE21uJWKkRr24dMdyR3va0nmRTnrHeaFLMiExSizifEbGdEdZlN1fXXEpQaYndp1WFx/06kitI0oi8pwhF9ndKiIu+ZDYTJCs/GnAkayovPMrgoMHNqVTYJKeLLlF3b7tkAj8WB90w6PQoKcZg+dLom89yK2yvn2+64S37E7d6LzvzfjmHe7EcBrAD4j0T0AhH9bjd186Jz7nLe2AvoZHv18PB4n2KYxZ4B8BCA33HOPQigCiOyu87O0o4/KUT0JBEdI6Jja5W1nZp4eHiMAMMs9jMAzjjnLpN2/TE6i3+JiPYBQPf/8k4HO+eecs494px7ZGba7+F5eOwWhsnPfoGIThPRHc6519HJyf5a9++LAH6j+/9bV+wLrmda2GbWEvpJYjzGcsLckZLQIY15Q5pBnNF3pC4k9R2r+zihF7UNAYNM3StNPIMikLaRRghvqW1eUMKkJNslxiSVVRFURv8TkGbFbR5dYj7s+DPCK0/eJ7v/IPcwrN4s9xUo4jqrs8v9iNTwpCtPM7Wdocch913stSgzYmbn6EkAaDf5Xqex3iPJCLOc9byT5rxwwD7OtlA6ATlmedxAshDrXdfdhRkU+zasnf1fAvgDIsoCOAngn6Mz/X9ERE8AeAfALw3Zl4eHxy5gqMXunHsRwCM7VH36uo7Gw8PjhmGkHnTOObS6orEVhwIRiNBuavFccYtL0gXD154Isd5t82pj0bRRZ6ICKRJbWGILKTrJ9DvbJDZh2LKeVJq8QlcpUg0hSjcamvBBiqOtVrNvXRSJOTWkDnLQ9VpNVTVFnRR3t5FXhP3VobTJXHBShQiN+Ky8DY25SvHfiXmM8tpER4Jf3nrGydRKsnerGsl5tKKwFM8bJvNuIFUe2b8JLhpEPKE8J5X51ZptBTGJUXkuexUOEv29b7yHx5jAL3YPjzGBX+weHmOCkevsvagso+NJPT1ta31bKjzSLTOuG1IHUWfdGtvY2TwTG/Oa1MFSo9dJgkupx5Hx/c0KUw1M+t+20A1Ts68QCN9LaSq0ueRk/rhtpixFPCHdjA3BhtCPC0UdySW3U3KCiKNl9g6yWSZ5ILMB0WzwvkheRooZE1StyvsFVmcPQhmdKCLKBuxTBCbKMFF7JrI/1QwlEelm50q60tooQyTieZTP0jazqujPmEtlauYk6aO/dw7konl2al2+fGu2lvBvdg+PMYFf7B4eY4KRpn8iohV0HHDmAaxeofmNxvthDIAfh4Ufh8Z7Hcdh59yenSpGuth7JyU65pzbyUlnrMbgx+HHMcpxeDHew2NM4Be7h8eYYLcW+1O7dF6J98MYAD8OCz8Ojes2jl3R2T08PEYPL8Z7eIwJRrrYiehxInqdiE4Q0cjYaIno94lomYheEd+NnAqbiA4R0feJ6DUiepWIfnU3xkJEeSJ6lohe6o7j17vfHyWiZ7r35w+7/AU3HEQUdvkNv7Nb4yCit4noZSJ6kYiOdb/bjWfkhtG2j2yxU8en9P8D8FkAdwH4ZSK6a0Sn/08AHjff7QYVdgzg15xzdwH4MIBf6c7BqMfSBPCYc+5+AA8AeJyIPgzgNwH8lnPuVgBrAJ64weO4jF9Fh578MnZrHP/AOfeAMHXtxjNy42jbnXMj+QPwEQB/Lj5/BcBXRnj+IwBeEZ9fB7CvW94H4PVRjUWM4VsAfm43xwKgCODHAD6EjvNGZqf7dQPPf7D7AD8G4DvoOLDvxjjeBjBvvhvpfQEwBeAUuntp13scoxTjDwA4LT6f6X63W9hVKmwiOgLgQQDP7MZYuqLzi+gQhX4XwFsAKs71UsuO6v78ewD/GkymPrdL43AA/oKInieiJ7vfjfq+3FDadr9Bh8FU2DcCRFQG8A0A/8o5t7EbY3HOJc65B9B5sz4K4AM3+pwWRPQLAJadc8+P+tw74OPOuYfQUTN/hYg+IStHdF+uibb9ShjlYj8L4JD4fLD73W5hKCrs6w0iitBZ6H/gnPvmbo4FAJxzFQDfR0dcniaiy/Gio7g/HwPwD4nobQBfQ0eU/+1dGAecc2e7/5cB/Ak6P4Cjvi/XRNt+JYxysT8H4LbuTmsWwD8C8O0Rnt/i2+hQYANDUmFfK6hDxvZ7AI475/7dbo2FiPYQ0XS3XEBn3+A4Oov+F0c1DufcV5xzB51zR9B5Hr7nnPsnox4HEZWIaOJyGcBnALyCEd8X59wFAKeJ6I7uV5dp26/POG70xofZaPgcgDfQ0Q//txGe978AOA+gjc6v5xPo6IZPA3gTwF8CmB3BOD6Ojgj2EwAvdv8+N+qxALgPwAvdcbwC4P/ofn8zgGcBnADwdQC5Ed6jTwH4zm6Mo3u+l7p/r15+NnfpGXkAwLHuvflTADPXaxzeg87DY0zgN+g8PMYEfrF7eIwJ/GL38BgT+MXu4TEm8Ivdw2NM4Be7h8eYwC92D48xgV/sHh5jgv8fpi9a39ce9uUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.imshow(train_set_x_orig[23])\n",
    "print('Prediction: ' + str(train_set_y[:,23]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "befbf688",
   "metadata": {},
   "source": [
    "### Extract dimensions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "14ca5942",
   "metadata": {},
   "outputs": [],
   "source": [
    "m_train = train_set_x_orig.shape[0]\n",
    "m_test = test_set_x_orig.shape[0]\n",
    "num_px = test_set_x_orig.shape[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ff93269",
   "metadata": {},
   "source": [
    "### Reshape input images\n",
    "\n",
    "Convert images from (m_train, num_px, num_px, 3) to (num_px * num_px * 3, m_train).\n",
    "\n",
    "This can be done using the trick below. We take the last dimension and put it in the first place. Then -1 automatically squishes all other dimensions after it. Lastly, we need to transpose since the pixels dimension comes before the m_train dimension."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "299bfa72",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(209, 64, 64, 3)\n",
      "(12288, 209)\n"
     ]
    }
   ],
   "source": [
    "train_set_x_flatten = train_set_x_orig.reshape(train_set_x_orig.shape[0], -1).T\n",
    "test_set_x_flatten = test_set_x_orig.reshape(test_set_x_orig.shape[0], -1).T\n",
    "print(train_set_x_orig.shape)\n",
    "print(train_set_x_flatten.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbf510e7",
   "metadata": {},
   "source": [
    "### Standardize dataset\n",
    "\n",
    "Divide the entire dataset by 255."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "b20629aa",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_set_x = train_set_x_flatten / 255\n",
    "test_set_x = test_set_x_flatten / 255"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bcddea38",
   "metadata": {},
   "source": [
    "### Activation function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "fd743857",
   "metadata": {},
   "outputs": [],
   "source": [
    "def activation_function(z):\n",
    "    # Sigmoid\n",
    "    s = 1 / (1+np.exp(-z))\n",
    "    return s"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca997685",
   "metadata": {},
   "source": [
    "### Initialize parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "795d3ae0",
   "metadata": {},
   "outputs": [],
   "source": [
    "def weights_and_bias(dim):\n",
    "    w = np.zeros((dim,1))\n",
    "    b = 0.0\n",
    "    return w,b"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ad9de26",
   "metadata": {},
   "source": [
    "### Forward and backward propagation\n",
    "\n",
    "We pass the weights and bias as parameters because we don't want to access them as global variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "39862365",
   "metadata": {},
   "outputs": [],
   "source": [
    "def propagate(w, b, X, Y):\n",
    "    # X.shape = (image_date, no_of_samples)\n",
    "    # Y.shape = (1, no_of_samples)\n",
    "    \n",
    "    # Forward propagation\n",
    "    m = X.shape[1]\n",
    "    Z = np.dot(w.T, X) + b\n",
    "    A = activation_function(Z)\n",
    "    \n",
    "    eps = 0.000001\n",
    "    \n",
    "    # Store cost so that we can draw a graph later on.\n",
    "    cost = (-1/m) * ((np.dot(np.log(A+eps), Y.T)) + np.dot(np.log(1-A+eps), (1-Y).T))\n",
    "    \n",
    "    # Backward propagation\n",
    "    dw = (1/m) * np.dot(X, (A-Y).T)\n",
    "    db = (1/m) * np.sum(A-Y)\n",
    "    cost = np.squeeze(np.array(cost))\n",
    "    \n",
    "    return dw, db, cost"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3695390",
   "metadata": {},
   "source": [
    "### Gradient descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "fc10fb3f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import copy\n",
    "\n",
    "def optimize(w, b, X, Y, num_iterations=100, learning_rate=0.005):\n",
    "    w = copy.deepcopy(w)\n",
    "    b = copy.deepcopy(b)\n",
    "    \n",
    "    costs = []\n",
    "    \n",
    "    for i in range(num_iterations): # Number of iterations\n",
    "        dw, db, cost = propagate(w, b, X, Y)\n",
    "        w = w - learning_rate * dw\n",
    "        b = b - learning_rate * db\n",
    "        costs.append(cost)\n",
    "        #print('Cost after iteration %i: %f' % (i, cost))\n",
    "    \n",
    "    return w, b, dw, db, costs        "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44b42046",
   "metadata": {},
   "source": [
    "### Predict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "65aeb546",
   "metadata": {},
   "outputs": [],
   "source": [
    "def predict(w, b, X):\n",
    "    m = X.shape[1]\n",
    "    Y = np.zeros((1,m))\n",
    "    w = w.reshape((X.shape[0], 1))   # TODO: why?\n",
    "    A = activation_function(np.dot(w.T, X) + b)\n",
    "    \n",
    "    for i in range(A.shape[1]): # m\n",
    "        if A[0, i] > 0.5: Y[0, i] = 1\n",
    "        else: Y[0, i] = 0\n",
    "    \n",
    "    return Y    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "beb66250",
   "metadata": {},
   "source": [
    "### Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "2a4bfac2",
   "metadata": {},
   "outputs": [],
   "source": [
    "def model(X_train, Y_train, X_test, Y_test, num_iterations=100, learning_rate=0.005):\n",
    "    w, b = weights_and_bias(X_train.shape[0])\n",
    "    \n",
    "    w, b, dw, db, costs = optimize(w, b, X_train, Y_train, num_iterations, learning_rate)\n",
    "    \n",
    "    Y_train_preds = predict(w, b, X_train)\n",
    "    Y_test_preds = predict(w, b, X_test)\n",
    "    \n",
    "    print('Train accuracy: {} %'.format(100 - np.mean(np.abs(Y_train_preds - Y_train)) * 100))\n",
    "    print('Test accuracy: {} %'.format(100 - np.mean(np.abs(Y_test_preds - Y_test)) * 100))\n",
    "    \n",
    "    d = {'costs': costs,\n",
    "         'Y_train_preds' : Y_train_preds,\n",
    "         'Y_test_preds': Y_test_preds,\n",
    "         'w' : w,\n",
    "         'b' : b,\n",
    "         'learning_rate' : learning_rate,\n",
    "         'num_iterations': num_iterations}\n",
    "    \n",
    "    return d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "0590f3e4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train accuracy: 96.65071770334929 %\n",
      "Test accuracy: 72.0 %\n"
     ]
    }
   ],
   "source": [
    "logistic_regression_model = model(train_set_x, train_set_y, test_set_x, test_set_y, num_iterations=1000, learning_rate=0.005)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4c801dc",
   "metadata": {},
   "source": [
    "### Cost function plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "6dba831c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAqjUlEQVR4nO3deXhV5bn+8e+TETJDEpA5ICiCCmrqcNSKM9pWbe3pEVtrB0vbU6qn2p6jdrJWW1u1Dj22Pz2ttXbQqnWgVgUnrLMERWWQWYYoEMJMZAg8vz/WStgJOyHZycpO9r4/17Uv9lrrXWs/K1tzZ03va+6OiIikr4xkFyAiIsmlIBARSXMKAhGRNKcgEBFJcwoCEZE0pyAQEUlzCgJJGWZ2opktSHYdIj2NgkA6hZm9b2anJbMGd3/R3Q9OZg0NzGyCma1K0mdfaGbLzWybmT1qZn1baTvezGaZWV347/iYZWZmvzCz2vD1CzOzmOUefsbW8PW7iHdNIqIgkB7DzDKTXQM0/oLslv/vmNlY4E7gIqA/UAf8poW2OcBjwJ+BPsAfgcfC+QCTgfOAccDhwKeArzfbzDh3Lwhfl3Tu3khX6Zb/MUvqMLMMM7vSzJaEf1U+EPsXqpk9aGarzWyTmf0r/EXWsOweM/utmT1hZtuAk8Mjj++a2TvhOn8zs15h+yZ/hbfWNlz+32b2oZl9YGaXhH/hjmxhP2aY2fVm9jLBL9cRZvZlM5tvZlvMbKmZfT1smw88CQyM+Wt54P5+Fp3k88A/3P1f7r4V+CHwGTMrjNN2ApAF3OruO9z9dsCAU8LlFwM3u/sqd68Gbga+1Mn1SjegIJCofZvgr8qTgIHABuCOmOVPAqOAfsCbwF+arX8hcD1QCLwUzvscMBEYTvCX6pda+fy4bc1sInA5cBowkuCX4v5cRPBXciGwHFgLfBIoAr4M3GJmR7r7NuAs4IOYv5Y/aMPPopGZDTWzja28LmyhxrHA2w0T7r4E2Akc1ELbd7xpPzPvhPP32Vb4fixN/SsM8ofNrKKFmqSby0p2AZLyvgFMcfdVAGZ2DbDCzC5y93p3v7uhYbhsg5kVu/umcPZj7v5y+H57eIr69vAXK2b2D2B8K5/fUtvPAX9w97kxn/35/ezLPQ3tQ/+Mef+CmU0HTiQItHha/VnENnT3FUDJfuqJpwDY1GzeJoLwam/b5ss3AQVmZmF4nAS8BuQB1wGPm9n45vsi3Z+OCCRqw4BHGv6SBeYDu4H+ZpZpZjeEp0o2A++H65TFrL8yzjZXx7yvI/iF1ZKW2g5stu14n9NckzZmdpaZvWZm68N9O5umtTfX4s+iDZ/dVlsJjlBiFQFbEmjbfHkRsLXhCCI8/bTT3TcClxEcdR3SoeolKRQEErWVwFnuXhLz6hWec74QOJfg9EwxUBGuYzHrR9U97ofA4JjpIW1Yp7EWM8sF/g7cBPR39xLgCfbWHq/u1n4WTYSnhra28mrp6GUuwcXdhu2MAHKBhS20PdzMYn/eh4fz99lW+D72iKg5p+l3Jz2EgkA6U7aZ9Yp5ZQH/D7jezIYBmFm5mZ0bti8EdgC1BKcXftaFtT4AfNnMDjGzPIKLqu2RQ/ALtgaoN7OzgDNilq8BSs2sOGZeaz+LJtx9Rcz1hXiv5tdSGvwF+JQFz1TkA9cCD7t7vCOCGQRHJJeaWa6ZTQnnPxf+ey9wuZkNMrOBwBXAPWHtYy249TTTzAoILiRXExzlSA+jIJDO9ATwUczrGuA2YCow3cy2EJxTPiZsfy/BRddqYF64rEu4+5PA7cDzwOKYz97RxvW3AJcSBMoGgqObqTHL3wPuA5aGp4IG0vrPolOE1zC+QRAIawnC9j8blpvZk2Z2ddh2J8HF6y8CG4GvAOeF8yG4DfUfwLvAHIJrIneGy/oDfwM2A0sJjuY+6e67OnN/pGuYBqYRATM7hOCXXa4udkq60RGBpC0z+3R4SqQP8AuC++8VApJ2FASSzr5OcPpkCcG58m8mtxyR5NCpIRGRNKcjAhGRNNfjniwuKyvzioqKZJchItKjzJo1a527l8db1uOCoKKigqqqqmSXISLSo5jZ8paW6dSQiEiaUxCIiKQ5BYGISJpTEIiIpLlIg8DMJprZAjNbbGZXxlk+zMyeDUeQmmFmg+NtR0REohNZEFgwvuwdBCM1jQEmmdmYZs1uAu5198MJekn8eVT1iIhIfFEeERwNLHb3pWFvhvcT9D0fawx7u7x9Ps5yERGJWJRBMIimIzqtCufFehv4TPj+00ChmZU235CZTTazKjOrqqmp6VBRLyysYeX6ug5tQ0QklST7YvF3gZPM7C2C8U+rCTr/asLd73L3SnevLC+P+2Bcm1189xuccvOMDm1DRCSVRPlkcTVNh/8bHM5rFA4q/hmAcJSj88PxTyO1a7c62hMRaRDlEcFMYJSZDTezHOACYkZwAjCzMjNrqOEq4O4I6xERkTgiC4JwgI8pwDSCcUwfcPe5ZnatmZ0TNpsALDCzhQRD310fVT0iIhJfpJ3OufsTBOPYxs77Ucz7h4CHoqyhJbt27yE7M9mXSEREki9tfxN+5Z6ZyS5BRKRbSNsgeHHRumSXICLSLaRtEABs+mhXsksQEUm6tAqCWcvXN5n+3oNvJ6kSEZHuI62C4Pzfvtpkevq8NUmqRESk+0irIIhnzx49XCYi6S3tg+CZ+ToqEJH0lvZBMPlPs5JdgohIUqV9EACs2bw92SWIiCSNggD4x9sfJLsEEZGkURAA1/1zfrJLEBFJGgVByF13D4lIelIQhH719MJklyAikhQKgtCvn1uc7BJERJJCQRBDfQ+JSDpSEMR4Z9XGZJcgItLlFAQxLvr9G8kuQUSkyykImtHpIRFJNwqCZn4zQxeNRSS9KAiaufOFpckuQUSkS0UaBGY20cwWmNliM7syzvKhZva8mb1lZu+Y2dlR1tNWO+p3J7sEEZEuE1kQmFkmcAdwFjAGmGRmY5o1+wHwgLsfAVwA/CaqetrjO3+bnewSRES6TJRHBEcDi919qbvvBO4Hzm3WxoGi8H0x0C16f3vi3dXJLkFEpMtEGQSDgJUx06vCebGuAb5gZquAJ4Bvx9uQmU02syozq6qpqYmi1n0sWrOlSz5HRCTZkn2xeBJwj7sPBs4G/mRm+9Tk7ne5e6W7V5aXl3dJYdc+Pq9LPkdEJNmiDIJqYEjM9OBwXqyvAg8AuPurQC+gLMKa2uzFRevUI6mIpIUog2AmMMrMhptZDsHF4KnN2qwATgUws0MIgqBrzv20wf0zV+6/kYhIDxdZELh7PTAFmAbMJ7g7aK6ZXWtm54TNrgC+ZmZvA/cBX/Ju9Gf41Y+8m+wSREQilxXlxt39CYKLwLHzfhTzfh5wfJQ1dET3iSQRkegk+2Jxt3fz9AXJLkFEJFIKgv3QgDUikuoUBG3w5ooNyS5BRCQyCoI2uFxdTohICkurIBh9QGFC671fW0f97j2dXI2ISPeQVkGwpwO3Ad2oi8YikqLSLAgSX1fjFIhIqkqvIOhIEgAbtu3spEpERLqPtAqCLTvqOf/IwQmv/4nbX+zEakREuoe0CYI9e5z123ZyQHEug/v0TmgbH2zaTu3WHZ1cmYhIcqVNEGz6aBe79zil+bl8c8KBCW/nNzOWdGJVIiLJlzZBsL4uOL/fNz+Hzx8zLOHt/P6lZZ1VkohIt5A2QbCzPngOIDcr2OVTR/dLeFuPzW4+rIKISM+VNkFQvzu4Yyg7M9jlS08dlfC2Lrt/dmeUJCLSLaRNEOwMnwzOyjQAxg0p6dD2Nm/f1dGSRES6hbQJgl1hEORk7t3lv3/zuIS3d97/vtzhmkREuoO0CYKGU0NZMUFw1LC+CW9v6bptrNcDZiKSAtImCBqOCLLDU0MNzhs/MOFtXv2whrIUkZ4vDYOg6S5f/+nDEt7mU3NX042GWBYRSUgaBUHTu4Ya5OdmJdw9NcBtzy7qUF0iIskWaRCY2UQzW2Bmi83syjjLbzGz2eFroZltjKqWlk4NAdx50VEJb/fWZxZ1uDM7EZFkiiwIzCwTuAM4CxgDTDKzMbFt3P077j7e3ccDvwYejqqelk4NAQwrzaesICfhbb+4eF3C64qIJFuURwRHA4vdfam77wTuB85tpf0k4L6oimnp1FCDp/7r4wlv++K730h4XRGRZIsyCAYBK2OmV4Xz9mFmw4DhwHMtLJ9sZlVmVlVTU5NQMfV7Wj41BFBWkJvQdhs8M29Nh9YXEUmW7nKx+ALgIXffHW+hu9/l7pXuXlleXp7QBzT0NZTVwhEBwDOXJ35UcMm9VQmvKyKSTFEGQTUwJGZ6cDgvnguI8LQQ7D01lNNKEIzsV0hBblbCn7G8dlvC64qIJEuUQTATGGVmw80sh+CX/dTmjcxsNNAHeDXCWvjy8RW8+cPT6ZXd+i4/d8VJCX/GSTfOSHhdEZFkiSwI3L0emAJMA+YDD7j7XDO71szOiWl6AXC/R/xkVq/sTPrm52AW/xpBg35FvRg3uDjhz1m2TkcFItKzWE97MraystKrqqI9H1+3s54xP5qW8Prv3/CJTqxGRKTjzGyWu1fGW9ZdLhZ3K3k5WXzisAEJr79g9ZZOrEZEJFoKghbcesH4hNc989Z/dV4hIiIRUxC0IDszg7s60PXE2i3bO7EaEZHoKAhaccbYAxJe9+jrn+3ESkREoqMg2I8Xvjch4XXnf7i58woREYmIgmA/hpXmJ9xN9Vm3vdjJ1YiIdD4FQRs8+q3jE1531vL1nViJiEjnUxC0Qa/sTP56yTEJrXv+byN9YFpEpMMUBG30byPLEl73vdW6ViAi3ZeCoB3euPrUhNabeOuLGttYRLotBUE79CvqxUXHDkto3Xteeb9zixER6SQKgna69tyxCa33k3/M09jGItItKQjaycx45cpTElr3pukLOrkaEZGOUxAkYGBJb84c27/d6/1mxhJ21McdhE1EJGkUBAn67ecT64foO3+b3bmFiIh0kIIgQRkZltBoZk+8u5oVtXURVCQikhgFQQeMKC/g6rNHt3u9j9/4fATViIgkRkHQQZM/fmBC682p3tTJlYiIJEZB0Ane+uHp7V7nk79+SQ+ZiUi3oCDoBH3yc/j1pCPavd6fX18RQTUiIu0TaRCY2UQzW2Bmi83syhbafM7M5pnZXDP7a5T1ROlT4wbSJy+7Xev88NE51O2sj6giEZG2iSwIzCwTuAM4CxgDTDKzMc3ajAKuAo5397HAf0VVT1d4LYG+iCbeqjELRCS5ojwiOBpY7O5L3X0ncD9wbrM2XwPucPcNAO6+NsJ6Ipeblckzl3+8XeusWF/HyvW6nVREkifKIBgErIyZXhXOi3UQcJCZvWxmr5nZxHgbMrPJZlZlZlU1NTURlds5RvYr5OZ/H9eudU785fO6cCwiSZPsi8VZwChgAjAJ+D8zK2neyN3vcvdKd68sLy/v2goTcP5Rg9u9zr2vLo+gEhGR/YsyCKqBITHTg8N5sVYBU919l7svAxYSBEOPN//auAc3Lfrx1Lm6cCwiSRFlEMwERpnZcDPLAS4ApjZr8yjB0QBmVkZwqmhphDV1md45mTx52YntWufEX+iJYxHpepEFgbvXA1OAacB84AF3n2tm15rZOWGzaUCtmc0Dnge+5+61UdXU1Q4ZUMTdX6psc/vabTtZUrM1wopERPZlbblIaWb/7u4P7m9eV6isrPSqqqqu/tgOqbzuGdZt3dHm9kt/djYZGRZhRSKSbsxslrvH/cu0rUcEV7VxnsTx6lXtG8jm+4++G1ElIiL7ajUIzOwsM/s1MMjMbo953QPoymYbZWdmtKs/ovveWMnqTdsjrEhEZK/9HRF8AFQB24FZMa+pwJnRlpZa+uTn8M9LT2hz+2N//myE1YiI7NVqELj72+7+R2Cku/8xfD+V4InhDV1SYQoZO7CYL/1bRZvbPz1vTXTFiIiE2nqN4GkzKzKzvsCbBA9+3RJhXSnrmnPGMrJfQZvafu3eKtZv2xlxRSKS7toaBMXuvhn4DHCvux8DtL+HNQFg+n+1vT+iI3/6dISViIi0PQiyzGwA8Dng8QjrSQsZGcbbPzqjze0ffnNVhNWISLpraxBcS/Dw1xJ3n2lmI4BF0ZWV+orzsnn2ipPa1PbyB95my/ZdEVckIumqTUHg7g+6++Hu/s1weqm7nx9taanvwPIC/u+LbXvy+LBrpkdcjYikqzYFgZkNNrNHzGxt+Pq7mbW/i03Zx+lj+nPhMUPb1Pa2Z3QQJiKdr62nhv5AcNvowPD1j3CedIKfffowRh9QuN92tzyzsF1dVYiItEVbg6Dc3f/g7vXh6x6g+w8M0IM8cWnbeiqtvO4ZDWIjIp2qrUFQa2ZfMLPM8PUFIGV6Ce0OMjKMBde1bQyD259dHHE1IpJO2hoEXyG4dXQ18CHwWeBLEdWUtnKzMpn1g9P22+6WZxaqu2oR6TTtuX30Yncvd/d+BMHwk+jKSl+lBbk8/u3990l06s0vsHuPThGJSMe1NQgOj+1byN3XA0dEU5IcOqiYqVOO32+78+54uQuqEZFU19YgyDCzPg0TYZ9DWdGUJACHDy7h6rNHt9rm3epNPDa7+TDQIiLt09YguBl41cx+amY/BV4BfhldWQIw+eMH8s0JB7ba5rL7Z6tjOhHpkLY+WXwvQYdza8LXZ9z9T1EWJoH/mTia40aUttpGHdOJSEe0efB6d5/n7v8bvuZFWZQ0dd/kYykryGm1zXWP6ysRkcS0OQgSYWYTzWyBmS02syvjLP+SmdWY2ezwdUmU9fRkr1/d+m2lv3tpGdPnru6iakQklUQWBGaWCdwBnAWMASaZ2Zg4Tf/m7uPD1++iqqeny8wwlv7s7FbbTP7TLDboeoGItFOURwRHEwxpudTddwL3A+dG+HkpLyPDmPuT1oeKPuKnT6sLChFplyiDYBCwMmZ6VTivufPN7B0ze8jMhsTbkJlNNrMqM6uqqamJotYeIz83i3nXth4G//P3d7qoGhFJBZFeI2iDfwAV7n448DTwx3iN3P0ud69098rycvV1l5eTxczvt3zN4IGqVdz3xoourEhEerIog6AaiP0Lf3A4r5G717p7Q7/KvwOOirCelFJemMvrV7c8bPRVD79L9caPurAiEempogyCmcAoMxtuZjnABQRjGjQKx0FucA4wP8J6Uk7/ol688L0JLS4//obnqN+9p+sKEpEeKbIgcPd6YArBWMfzgQfcfa6ZXWtm54TNLjWzuWb2NnAp6tG03YaV5jPjuxNaXD7y+0/q4rGItMp62i+JyspKr6qqSnYZ3c7y2m2cdOOMuMvOHNufOy9q29jIIpKazGyWu8f9RZDsi8XSSYaV5vP3bx4Xd9m0uWt4banGERKR+BQEKeSoYX2Z/p2Px112wV2v8d7qzV1ckYj0BAqCFHNQ/0Ie+1b8sQwm3voiW7bv6uKKRKS7UxCkoHFDSnjg6/FPEx12zXSNbCYiTSgIUtTRw1s+TXTEtdO7uBoR6c4UBCnsoP6FcYe83Ly9nqsfeTcJFYlId6QgSHGHDy7h8W+fsM/8v76+gjtfWJKEikSku1EQpIFDBxXHfQL550++x+yVG7u8HhHpXhQEaWJYaT7/+t7J+8w/746XmfeBbisVSWcKgjQytDSPl688ZZ/5Z9/+Imu3bE9CRSLSHSgI0sygkt7M+sG+XVgfff2zesZAJE0pCNJQaUEu86+duM/8w66Zzo763UmoSESSSUGQpnrnZMYd6ezgHzzFznp1XS2SThQEaSwvJ4sF1+17ZHDQD9R1tUg6URCkudysTJb9/GxK83OazB/3k+nsUVcUImlBQSCYGbN+eDonjCxrnLd5ez0n3zwjeUWJSJdREEijP19yDFNOHtk4vby2jpNvmpG8gkSkSygIpInvnnkwv/zs4Y3Ty9Zt49O/eVnXDERSmIJA9vG5yiH86atHN06/tWIjp9/yryRWJCJRUhBIXCeOKufZK05qnF68divH/OwZHRmIpKBIg8DMJprZAjNbbGZXttLufDNzM9MI693IgeUFvPXD0xun12zewdgfT2PXbj1nIJJKIgsCM8sE7gDOAsYAk8xsTJx2hcBlwOtR1SKJ65Ofw6Lrz2qcrtu5m1Hff1IPnYmkkCiPCI4GFrv7UnffCdwPnBun3U+BXwDq9aybys7M4P0bPsEZY/o3zjvoB0+yfZe6oxBJBVEGwSBgZcz0qnBeIzM7Ehji7v9sbUNmNtnMqsysqqampvMrlTa564uVXP/pQxunR//wKfVaKpICknax2MwygF8BV+yvrbvf5e6V7l5ZXl4efXHSos8fM4wnLzuxcfro659lTvWmJFYkIh0VZRBUA0NipgeH8xoUAocCM8zsfeBYYKouGHd/hwwo4u0fndE4/clfv8Q/3/kwiRWJSEdEGQQzgVFmNtzMcoALgKkNC919k7uXuXuFu1cArwHnuHtVhDVJJynOy2bpz87m8MHFAHzrr29yzdS5Sa5KRBIRWRC4ez0wBZgGzAcecPe5ZnatmZ0T1edK18nIMKZOOaHxusE9r7zPuJ9oTAORnsZ62gNClZWVXlWlg4buZvHarZz2qxcap1/43gSGleYnsSIRiWVms9w97ql3PVksnWJkvwIWXncW44aUAHDSjTP482vLk1uUiLSJgkA6TU5WBo9963hun3QEAD94dA6T761i2476JFcmIq1REEinO2fcQGb/KBjfYPq8NZx04wyee29NsssSkRYoCCQSJXk5/PmSY7jzoqPYvWcPX7mniil/fZO1m/UAmkh3oyCQSJ059gBe+p9T+PLxFTw5ZzUTbprB/z63iI926s4ike5CQSCRy8/N4sefGssTl57IMcP7ctP0hZz2qxe4/40V6rxOpBvQ7aPS5V5fWst1/5zPu9WbGNynN187cQSfPWow+blZyS5NJGW1dvuogkCSwt2ZsbCG259dxFsrNlLUK4t/rxzCF44dxvAyPX8g0tkUBNKtzVq+gbtfXsa0Oaup3+McPbwv540fxCcOG0BxXnayyxNJCQoC6RHWbt7O32au5JHZ1Syt2UZ2pnHywf0474hBnDK6H72yM5NdokiPpSCQHsXdmVO9mUdnVzP17Q+o2bKDwtwsThvTnzPH9ufjB5WTl6PrCSLtoSCQHmv3HufVJbU8OruaZ+avYWPdLnKzMjhxVBlnjD2AU0f3o7QgN9llinR7rQWB/qySbi0zwzhhVBknjCqjfvceZr6/gWlzV/P0vDU8M38tGQaVFX05Y0x/TjukPxW60CzSbjoikB7J3Zn7wWamz13N9HlreG/1FgBGlOUz4eB+nDK6Hx8b3ofcLF1XEAGdGpI0sKK2jufeW8PzC2p4dWktO+v3kJ+TyfEjyzh5dD9OPrgfBxT3SnaZIkmjIJC0UreznleX1PLce2t5/r21fLAp6N/okAFFnDK6nAkH92P8kBKyM/VgvaQPBYGkLXdn4ZqtQSgsWMus5RvYvccpyM3i2BGlnBhefxhRlo+ZJbtckcgoCERCm+p28cqSdby4eB0vLVrHivV1AAws7sUJo8o4cVQ5x48so29+TpIrFelcCgKRFiyv3caLi4JQeGXJOjZvr8cMxg4s4oSR5Zw4qoyjhvXRw2zS4ykIRNqgfvce3qnexEthMLy5YgP1e5yczAzGDynh2ANLOXZEX44cqmCQnidpQWBmE4HbgEzgd+5+Q7Pl3wC+BewGtgKT3X1ea9tUEEhX2bqjnjeW1fLa0vW8trSWOdWb2OMEwTC0hONGlHLsiFKOGFqiYJBuLylBYGaZwELgdGAVMBOYFPuL3syK3H1z+P4c4D/dfWJr21UQSLJs3r6LmcuCUHht6XrmfhAGQ1YGRwwp4bgDSzlmeCnjh5TQO0fBIN1Lsp4sPhpY7O5LwyLuB84FGoOgIQRC+UDPOk8laaWoVzanHtKfUw/pD8Cmj/YGw6tLa7nt2UW4LyIrwzh0UDEfq+jDUcP6UlnRhzJ1gyHdWJRBMAhYGTO9CjimeSMz+xZwOZADnBJvQ2Y2GZgMMHTo0E4vVCQRxb2zOW1Mf04bEwZD3S5mvr+equUbmLV8PX98ZTn/9+IyIHji+ahhffhYRRAMw3W7qnQjUZ4a+iww0d0vCacvAo5x9ykttL8QONPdL25tuzo1JD3F9l27mVO9iarlG6gKA2Jj3S4ASvNzGoPhyGF9GDuwSNcZJFLJOjVUDQyJmR4czmvJ/cBvI6xHpEv1ys6ksqIvlRV94aQD2bPHWbpuKzPf38DM99cza/kGps9bA0B2pjFmQBHjh5QwfmgJ44f0oaI0T0cN0iWiDIKZwCgzG04QABcAF8Y2MLNR7r4onPwEsAiRFJWRYYzsV8jIfoVMOjo4xbl283beXLGR2Ss3MnvlBh6ctYo/vrocgD552YwbUhKEQ/gqydODbtL5IgsCd683synANILbR+9297lmdi1Q5e5TgSlmdhqwC9gAtHpaSCTV9CvqxcRDD2DioQcAwfgLC9dsCYJhxUbeWrmBFxbW0HAGd0RZfuNRw+GDSxh9QKFOKUmH6YEykW5uy/ZdvLtqE2+t3Mhb4dHDuq07AMjKMA7qX8hhg4o5dHAxhw0qVjhIXHqyWCSFuDvVGz/i3VWbeLc6eM2p3sSG8EK0wkHiURCIpDh3Z9WGj5hT3YZwGFTEmIFFHHxAEQW5GqQwXSgIRNLQ/sIBoKI0j0MGFHHIgCLGDCjikIFFDCzupbuVUpDGLBZJQ2bGkL55DOmbx1mHDQCCcPhw03bmfbCZ+R9uZv7qzcz7YDNPzlnduF5x72wOGVC4NxwGFDGqf4GG/UxhCgKRNGJmDCzpzcCS3o1PREPQwd6C1ZuZ9+GWxpC4/42VfLRrNxCcWhrZr4BDBhRx8AGFHNy/kIMOKNTRQ4pQEIgIBblZHDWsL0cN69s4b/ce5/3abcGRw4fBkcOrS2p55K3qJuuN6l8QBEP/Qg4+oJBR/QsoL8hVQPQgukYgIu2yqW4XC9duYeGaLSxcvYUFa7awYPWWJtce+uRlNwbDQQ0h0b+Q4rzsJFae3nSNQEQ6TXFeNh+r6MvHKvYePbg767buDMIhfC1YvYWH36xm6476xnb9i3IZ1a+Qkf0KOLA8nwP7FTCyvIDyQh1BJJOCQEQ6zMwoL8ylvDCX40eWNc53dz7YtH3v0cPqLSyu2cqDVSvZtnN3Y7vCXllhOBQ0+XdIn95kZWYkY5fSioJARCJjZgwq6c2gkt6cfHC/xvnuzurN21mydhuL1wbhsGTtNl5YWMNDs1Y1tsvJzKCiLG+fgBhRnk9ejn59dRb9JEWky5kZA4p7M6C4NyeMKmuybNNHu1hSs5Ula7c2BsR7q7cwbe5q9sRc0uxflMvwsnyGl+VTUZrf+H5oaZ5udW0nBYGIdCvFvbM5cmgfjhzap8n8HfW7WV5bFwTE2q0sq93G++u2MW3uGtZv29nYLsNgYEnvuCExWKea4lIQiEiPkJuV2XgHUnOb6nY1BsPSdcG/y9Zt45E3q9kSc7E6KyN4yG5vQORRUZbP0L55DCzpTXaahoSCQER6vOK8bMbnBWM2xHJ3arftZFkYDA0BsWzdNl5Zso7tu/Y0ts3MMAaW9GJY33yG9M1jWGkeQ/uGr9I8inql7q2vCgIRSVlmRllBLmUFuU1udwXYs8dZs2U7y2vrWFFbx4r1dSxfH/z71JwPmzwXAcGzEUEo5DO0b+8mgXFAUS8yMnru7a8KAhFJSxkZey9YHzuidJ/lm7fvYkVtHStjAmJFbR1vr9zIE+9+yO6YK9c5mRkM7tuboX3zGNInj8F9ejM4/HdQn96U5ud06+ckFAQiInEU9crm0EHFHDqoeJ9lu3bv4cON21m+fltjQKxYX8fy2jreXL6Bzdvrm7TvlZ2xNxhK9oZEQ1Aku0sOBYGISDtlZ2YwtDS4dhDP5u27qN7wEas2fMSqDXV732+sY/bKjWxsdtopNyuDQeFRRBAUvZscVZQX5EZ66klBICLSyYp6ZVM0IJtDBhTFXb51R30YDnWs2vAR1Rv3vp9TvanJ7bAQnHoaUNKLK844mHPGDez0ehUEIiJdrCA3K+jO+4B9b4UFqNtZH3MUEYTEBxu3U5qfE0k9kQaBmU0EbgMygd+5+w3Nll8OXALUAzXAV9x9eZQ1iYh0d3k5WYzqX8ioOM9MRCGypyfMLBO4AzgLGANMMrMxzZq9BVS6++HAQ8Avo6pHRETii/IxuqOBxe6+1N13AvcD58Y2cPfn3b0unHwNGBxhPSIiEkeUQTAIWBkzvSqc15KvAk/GW2Bmk82sysyqampqOrFEERHpFh1rmNkXgErgxnjL3f0ud69098ry8vKuLU5EJMVFebG4GhgSMz04nNeEmZ0GfB84yd13RFiPiIjEEeURwUxglJkNN7Mc4AJgamwDMzsCuBM4x93XRliLiIi0ILIgcPd6YAowDZgPPODuc83sWjM7J2x2I1AAPGhms81sagubExGRiET6HIG7PwE80Wzej2Lenxbl54uIyP6Zu++/VTdiZjVAog+dlQHrOrGcnkD7nB60z+mhI/s8zN3j3m3T44KgI8ysyt0rk11HV9I+pwftc3qIap+7xe2jIiKSPAoCEZE0l25BcFeyC0gC7XN60D6nh0j2Oa2uEYiIyL7S7YhARESaURCIiKS5tAkCM5toZgvMbLGZXZnsejqLmQ0xs+fNbJ6ZzTWzy8L5fc3saTNbFP7bJ5xvZnZ7+HN4x8yOTO4eJMbMMs3sLTN7PJwebmavh/v1t7BbE8wsN5xeHC6vSGrhCTKzEjN7yMzeM7P5ZnZcGnzH3wn/m55jZveZWa9U/J7N7G4zW2tmc2Lmtfu7NbOLw/aLzOzi9tSQFkHQxkFyeqp64Ap3HwMcC3wr3LcrgWfdfRTwbDgNwc9gVPiaDPy260vuFJcRdF3S4BfALe4+EthA0K054b8bwvm3hO16otuAp9x9NDCOYN9T9js2s0HApQQDVx1KMMrhBaTm93wPMLHZvHZ9t2bWF/gxcAzBWDA/bgiPNnH3lH8BxwHTYqavAq5Kdl0R7etjwOnAAmBAOG8AsCB8fycwKaZ9Y7ue8iLoyfZZ4BTgccAInrbMav59E/R1dVz4PitsZ8neh3bubzGwrHndKf4dN4xn0jf83h4HzkzV7xmoAOYk+t0Ck4A7Y+Y3abe/V1ocEdD+QXJ6pPBw+AjgdaC/u38YLloN9A/fp8LP4lbgv4E94XQpsNGDjg6h6T417m+4fFPYvicZTjCm9x/C02G/M7N8Uvg7dvdq4CZgBfAhwfc2i9T+nmO197vt0HeeLkGQ8sysAPg78F/uvjl2mQd/IqTEfcJm9klgrbvPSnYtXSgLOBL4rbsfAWxj76kCILW+Y4DwtMa5BCE4EMhn39MnaaErvtt0CYI2DZLTU5lZNkEI/MXdHw5nrzGzAeHyAUDDeA89/WdxPHCOmb1PMA72KQTnz0vMrKE33dh9atzfcHkxUNuVBXeCVcAqd389nH6IIBhS9TsGOA1Y5u417r4LeJjgu0/l7zlWe7/bDn3n6RIE+x0kp6cyMwN+D8x391/FLJoKNNw5cDHBtYOG+V8M7z44FtgUcwja7bn7Ve4+2N0rCL7H59z988DzwGfDZs33t+Hn8NmwfY/6y9ndVwMrzezgcNapwDxS9DsOrQCONbO88L/xhn1O2e+5mfZ+t9OAM8ysT3g0dUY4r22SfZGkCy/GnA0sBJYA3092PZ24XycQHDa+A8wOX2cTnB99FlgEPAP0DdsbwR1US4B3Ce7KSPp+JLjvE4DHw/cjgDeAxcCDQG44v1c4vThcPiLZdSe4r+OBqvB7fhTok+rfMfAT4D1gDvAnIDcVv2fgPoLrILsIjv6+msh3C3wl3P/FwJfbU4O6mBARSXPpcmpIRERaoCAQEUlzCgIRkTSnIBARSXMKAhGRNKcgkLRjZq+E/1aY2YWdvO2r432WSHem20clbZnZBOC77v7JdqyT5Xv7uom3fKu7F3RCeSJdRkcEknbMbGv49gbgRDObHfZ9n2lmN5rZzLCv96+H7SeY2YtmNpXg6VbM7FEzmxX2lz85nHcD0Dvc3l9iPyt8EvTGsG/9d83sP2K2PcP2jjXwl/BJWszsBgvGmXjHzG7qyp+RpJes/TcRSVlXEnNEEP5C3+TuHzOzXOBlM5setj0SONTdl4XTX3H39WbWG5hpZn939yvNbIq7j4/zWZ8heDp4HFAWrvOvcNkRwFjgA+Bl4Hgzmw98Ghjt7m5mJZ276yJ76YhAZK8zCPpxmU3QlXcpwQAgAG/EhADApWb2NvAaQWdfo2jdCcB97r7b3dcALwAfi9n2KnffQ9BFSAVBN8rbgd+b2WeAug7um0iLFAQiexnwbXcfH76Gu3vDEcG2xkbBtYXTCAZCGQe8RdDXTaJ2xLzfTTDwSj3BSFMPAZ8EnurA9kVapSCQdLYFKIyZngZ8M+zWGzM7KBwAprligmER68xsNMEQoQ12NazfzIvAf4TXIcqBjxN0jhZXOL5Esbs/AXyH4JSSSCR0jUDS2TvA7vAUzz0E4xpUAG+GF2xrgPPirPcU8I3wPP4CgtNDDe4C3jGzNz3oHrvBIwRDK75N0Fvsf7v76jBI4ikEHjOzXgRHKpcntIcibaDbR0VE0pxODYmIpDkFgYhImlMQiIikOQWBiEiaUxCIiKQ5BYGISJpTEIiIpLn/D8x3VGQG1J+nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "costs = np.squeeze(logistic_regression_model['costs'])\n",
    "plt.plot(costs)\n",
    "plt.ylabel('cost')\n",
    "plt.xlabel('iterations')\n",
    "plt.title('Learning rate = ' + str(logistic_regression_model['learning_rate']))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "379e03fe",
   "metadata": {},
   "source": [
    "### Choice of learning rate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "05d92cb5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training model with learning rate: 0.1\n",
      "Train accuracy: 100.0 %\n",
      "Test accuracy: 70.0 %\n",
      "---\n",
      "Training model with learning rate: 0.01\n",
      "Train accuracy: 99.52153110047847 %\n",
      "Test accuracy: 68.0 %\n",
      "---\n",
      "Training model with learning rate: 0.001\n",
      "Train accuracy: 88.99521531100478 %\n",
      "Test accuracy: 64.0 %\n",
      "---\n",
      "Training model with learning rate: 0.0001\n",
      "Train accuracy: 68.42105263157895 %\n",
      "Test accuracy: 36.0 %\n",
      "---\n",
      "Training model with learning rate: 1e-05\n",
      "Train accuracy: 65.55023923444976 %\n",
      "Test accuracy: 34.0 %\n",
      "---\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEGCAYAAABvtY4XAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA4dUlEQVR4nO2debwU5ZWwn9Pdd2FfDYuA4L4RVNAg+kUEBk3UGBMdFZ2YySgZZ5I4xoRRzJjNGBJN1BjjhDgazSDuCXFFQRgVcWERFJFF9n2HC9ytu97vj6ruW71XL3W7b9/z/H5NV73r6br0eU+fOnVeMcagKIqiVB6BUgugKIqi+IMqeEVRlApFFbyiKEqFogpeURSlQlEFryiKUqGESi2Am969e5vBgweXWgxFUZQ2w8KFC3cZY45IVVdWCn7w4MEsWLCg1GIoiqK0GURkfbo6ddEoiqJUKKrgFUVRKhRV8IqiKBVKWfngFcVNU1MTq1evpr6+vtSilA0dOnTg2GOPpbq6utSiKG0AVfBK2bJ69WpCoRD9+vVDREotTskxxnDw4EFWrlzJKaecotdEyYq6aJSypb6+ns6dO6sicxAROnfuTENDA0899RQNDQ2lFkkpc1TBK2WNKvd4RAQRYePGjbz99tulFkcpcypGwe+oa+DJ9zeg6Y+V9kDnzp3ZsWNHqcVQypyKUPArttVx1i9mc+vzH/Hmql0s33qAhev3lFospQKYM2cO5557LqNGjeKBBx5Iqn/33XcZP348AwcO5MUXX2w1ufSXjeKFilDwF9z3Zux476EmvnT/W3z9ofkllEipBCKRCJMnT2batGnMnTuXGTNmsHLlyrg2Rx55JPfddx+XXXZZiaRUlPRUhIJ388Abq2LHSzftY9PewyWURmnLLF68mMGDB3PUUUdRXV3NpZdeysyZM+PaDBw4kJNPPplAoOK+SkoFUHFhklv2tUQWfOX38wBYN+WiUomjFIl7/28jq3YWNx7+uCM6cPN5A9PWb9u2jf79+8fO+/Xrx6JFi4oqg6L4ScWZHfXNkVKLoCiKUhZUnAWvVCaZLG2/6Nu3L1u2bImdb926lX79+rW6HIqSLxVnwStKsTjttNNYu3YtGzZsoKmpiRkzZjB+/PhSi6UonlEFryhpCIVC/OIXv2DChAmcd955XHLJJZxwwgn8+te/jt1s/fDDDxk+fDgvvPAC//mf/8no0aNLK7SiuGgXLpr31uzmjKN6UBXU9UzJjbFjxzJ27Ni4skmTJsWOTzvtNBYuXNjaYimKJ9qFxrty6rvc/tePSi2GoihKq9IuFDzA659sL7UIiqIorUq7UfCKoijtDVXwiqIoFUq7UfCaY1JRlPZGu1HwzWGLcMQqtRiKoiitRrtR8IeaInFZJxXFC9nSBTc2NvLtb3+bUaNGcdFFF7Fx40YA9uzZw+WXX86xxx7L5MmTW1tsRQHakYIH+GznoVKLoLQhvKQLnj59Ot27d+edd97hhhtu4M477wSgtraWH/7wh9xxxx2lEF1RgHam4BUlF7ykC545cyZXXHEFABdffDFvv/02xhg6duzIF77wBWpqakohuqIA7eRJVqXt03XeXYR2Ly/qmOFeJ3HgnPTuEy/pgt1tQqEQXbt2Zc+ePfTq1auosipKPvhqwYvIzSKyTEQ+FpHpIlLr53yKoihKC75Z8CJyJPA94GRjTL2IPA1cBfzZrzmVyiWTpe0XXtIFR9v079+fcDjMgQMH6NmzZ2uLqigp8dsHHwI6iEgI6AhsydLedxau31tqEZQ2gpd0wePHj+eZZ54B4MUXX+Tcc8/VDbGVssE3C94Ys1lE7gE2APXAa8aY1xLbichEYCLAoEGD/BInxtcfeke38FM84U4XHIlEuOqqq2LpgocNG8YFF1zA1Vdfzfe+9z1GjRpF9+7deeihh2L9zzrrLA4ePEhTUxMzZ85k+vTpHH/88SX8REp7w08XTQ/gUmAIsA94RkSuNcb8r7udMWYqMBVgxIgR+sCpUlZkSxdcW1vL1KlTU/Z9//33fZVNUbLhp4tmHLDWGLPTGNMMPA+M8nE+RVEUxYWfCn4DMFJEOortlBwLFDfOTVEURUmLbwreGPMe8CywCPjImSv1b1lFURSl6Pj6oJMx5sfAj/2cQ1EURUmNpipQFEWpUNqlgr/pycU0a+pgRVEqnHap4Gd8uIUlG/eVWgylDZBvumCABx54gFGjRnHuuecyd+7cWPnNN9/M0KFDOf/881vjIyjtmHap4EF3eFKyU0i64JUrVzJjxgzmzJnDE088wW233UYkEgHgyiuvZNq0aa3+eZT2R7tV8IqSjULSBc+cOZNLL72UmpoaBg0axODBg1m8eDEAI0eOpEePHq3+eZT2R7tNF2yKZMIbY7hv1iouHz6AgT07FmdQJYmHPn2Iz+o+K+qYx3Q5hhtPvDFtfSHpgrdu3crw4cPj+m7btq2o8itKNtSCT8O9r6/kuYWbsrZbs+sQ989exbf/srAVpFIURfFOO7bgM5vw989eBcDXhw/wNE5DOFIcwZSUZLK0/aKQdMH9+vVL6tu3b99Wk11RQC34IqCpYSuVQtIFjx8/nhkzZtDY2MiGDRtYu3Ytp59+eik+htKOab8WfNkPqJSaQtIFn3DCCVxyySWMHj2aYDDIXXfdRTAYBODGG29k/vz57Nmzh+HDh3PLLbcwYcKEUn5UpUJptwq+WOjeDpVNIemCb7rpJm666aakcnfOeEXxk3brovnb4s00NHv3m6/ffYi5K3b4KJGiKEpxabcK/skPNjLllU89tz/v7rl889EPfJRIURSluLRbBQ+w/UBD0caKuuBfW7aNhev3FG1cRVGUfFEffIEkuuAnOvHwuu+roiilpl1b8Is27I07r2toZt/hprzGyhZXryiK0tq0awW//UBj3PnIu2Zz2s9eL5E0iqIoxaVdK/hEDjX59zTqqu11PPXBBt/GV/zBj3TB6cZ85JFHGDVqFP3792f37t2+fi6lfaAK3iFi5ediEScQfmddY8Z2/3Dvm/zncx/lNYdSGvxIF5xpzDPPPJOnnnqKAQMyp8dQFK+ognd4+K01BfU/1BTJKa5eKX/8SBecacyhQ4cycODAVv+cSuXS7qNomsIW1aEAG/YcjpXtPpjZGnfjjqJZt/tQESVT3Bz+/e+JrC5uuuDgscfQ8TvfSVvvV7rgbGMqSrFo9xb8vzxmP7zkTjlw47T8vnAX3vdWUtm+w02c/cvZSeX1TRF++9oKmsK6N6yiKP7Q7i34t1btAmD+Zy03tbL5073w8FtrGH3C51i1vY6t+5MfqPrD3NU88MZqenaq5vwTP0ffbrXUhIIFz1upZLK0/cKvdMHZxlSUYtHuLfgon+1sca8kxrTn82TqnS8t57IH56VMRnagoZl6J2Jn7+Fmzrt7LpOeXZrzHIq/+JEu2MuYilIsVMEDy7bsjztPjKdZu6vFP3/HjI/j6jJlk6xrDCeV7ahr4PM/eY2H314LELsxG/0lsXzrAV5YsiWpn9L6uNMFn3feeVxyySWxdMHRG6NXX301e/fuZdSoUUydOpXJkycD8emCJ0yYEEsXnG5MgIcffpjhw4ezdetWxo0bxy233FKyz65UBu3eRQNw0e/e9tz28fnr+dmlp+YwevwKsH1/Y8rq6K+GL91v+/EvGdYfpfT4kS441ZgA119/Pddff32BEitKC2rBpyCXrAOSZUcnr/niNdGBoijFRhW8z+h+IIqilApV8CkwLnt6fYGx7ZJgwptEW905TfzVcO/rKxn6k/iHahRFUXJBFXwK3Mr2ifeKmz/Gq/vn/tmrqGsIs7++uajzK4rSflAFnwK3Es6mj7P52BOr042XLt3wsJ++lkUCRVGU1KiCz0Khed6LcZP16qnvFiSDoijtE1XwBZJN/ycq+HwWjPlrNHVsqbj55psZOnQo559/fs59ly5dypgxYxg1ahQ/+tGPYn/7e+65hzPOOINx48Yxbtw4Zs9OTmWhKMVAFXwK3Eq42Bs1JQ5n5TBBQ3OEB2avojmi+WtaiyuvvJJp06bl1ffWW2/l7rvvZt68eaxdu5Y5c+bE6m644QZmzZrFrFmzUsbEK0oxUAWfApPmOHXbzC2yxcnH0tB70PMPzlnNb15fyZMfbMzeWCkKI0eOpEePHnFl69atY8KECVxwwQV89atfZdWqVUn9tm/fTl1dHcOHD0dEuPzyy3n11VdbS2xFAXx+klVEugMPA6diq7BvGWPm+zlnsSm6BZ8YJRkNk/TQ97CTv6bBx52nypUlr25j/7bCk8C56da3hmEX9s2536RJk5gyZQpHH300ixYtYvLkybF8NFG2bdsWl0Ssf//+sXTBAI8++ijPPvssn//85/nxj39M9+7d8/4cipIOvy34+4FXjTEnAsOA5T7PVxTio2gyq96sC0CSAW8SzrytIINvfclTO8VfDh06xIIFC5g4cSLjxo1j0qRJbN++PacxrrvuOubPn8/rr79Onz59+OlPf+qTtEp7xzcLXkS6AV8EvglgjGkCmvyazy8enbeOe64YFld2oKGZrrVVnvonhUmms+A9/FQIRPPWtMPEBvlY2n5gWRZdu3Zl1qxZceWRSIQLLrgAsDNMXnfddWzdujVWv2XLlli64COOOCJWfs011/CNb3yjFSRX2iN+WvBDgJ3AoyKyWEQeFpFOiY1EZKKILBCRBTt37vRRHO9sO5Ccv93NovV7Y8fZ4+Qz++BziaqJjmUMNEcsBt/6Eg/N/Szv/WSV3OnSpQsDBw7khRdeAOy/37JlywgGg7GbppMmTaJPnz506dKFhQsXYozh2WefjS0Abov/lVdeiWWTVJRi46eCDwFnAA8ZY04HDgG3JjYyxkw1xowwxoxwWzblTCF++eQomtTlqec1sbb1TprhX736KcdMfpnZy7ezYfdh3vg0N3eBkpkbb7yRSy65hM8++4zhw4fzxBNP8OCDDzJ9+nTGjRvH6NGjk/ZpjfLLX/6SH/zgB4waNYqjjjqKMWPGAHDnnXcyZswYxo4dy7x589RFo/iGnzdZNwGbjDHvOefPkkLBt0XcLpJsFng2F0228lS8t2Y313xhUFzZrOXbmfiXhUQsw7opF3kfTMnIQw89lLL8iSeeyNp32LBhcaGRUR544IGC5VIUL/hmwRtjtgEbRST6+3Ms8Ilf87UmBVnwJr+brO5556zYyYtLtybVuV014YjFLU8vYc3Og/kLqyhKm8bvKJrvAtNEZClwGnCXz/O1CoXkqknnovE0r+t4w57DcXWJD0wt2bSf5xZt4vtPL/E+QYGs332I//frN9iR5R6Goiitg68K3hjzoeNf/7wx5qvGmL3Ze5U/uTx9mvigU/o4+OxjuvsmypA4rsQibmx+8MwSRt41m3+btpC9h/wJZnrsnfVs3FPP34u45WChuYAqDWOMXhPFM/okax7EPemaay6axDj46I1TD9/ZeN9/epnA5ft3Gj67cBPbDjTw8kfb+NNba2Lt9tc3M/ruOUn70uZDbFFxhPl02wE2JvzSyIUOHTpQV1enCs3BGENdXR3NzZpCWvGG7smaBzmFNiacu0Ms7bFymbflODE0MtmCd0IqU4zj7vrO6l2s232Y381exR//aYR3YVIQ/azRhejC++z9ZfO96XvssceycOFC6urqsoabtgeMMTQ3N7N27VosyyIYDJZaJKXMUQWfB4UYlPe8tjLuPBd3T6Z+ib8MYso2xfDutu7Y+kIJOE9iFSssv7q6GmMMb7zxBt26dSMQ0B+cYD9sVVdXx1lnnVVqUZQyRxV8HiSq1oxkMTxzyDUW98vBymrBR8dNHtndNtquGEo506KSLyNHjsQYw6pVqwiHw8UbuA1TW1vL6NGjGT58eKlFUcocVfB5kIsCy5ZN0uSg4d1NIkk3WeNj86PzppLVvTi0SFc8DV/MVAqBQIBzzjmHc845p2hjKkp7QX/z5oEVp0wzt83mOs7Fnx8fRZO5XeINz7h613GgiC6aTItKlNeWbeP7T39Y+GSKomRFFXweFDOmo8WA9xAmSQYXjevYMiYpTJKE+igtLprCP1XAw33QiX9ZyPOLNhc8l6Io2VEF74Fp762PO49zh2Tpm03n5RsCmCmKxjJuazqzDz6QIdomV2KLhSY/U5SyQBW8BxZv2Bd3XsybiLFkY1588O4wyYQObgs8zoJP5aKJu8saL0c+vLlyJzc9ubhlUUmofzrNDlTGGH7z2gq27KvPf3JFUdKiCj4PMj1wlEjWdMEJ717appo3sS5TFI2VyoIvYNX6xiPvM+PDLWkXlUnPLWXl9rqkfiu21/HAG6u5cdqitGN/8ddz+Mv8dXnLpijtGVXweWDlsOe115usEcvw53lrs7R1yZBBw1vZomjcPvgUY+dLprQL9QnbDLrdOPVN6cMfN+w5zH/NWFa4cIrSDlEFnwdx1nLWTbezjOXqPuXVTz3PnOSDd9Wt3XUoo1zumkyWfq5E3Uap3D2JLqWwZQg6k+uGJYriD6rg86CYuVHiI2NykSH+3N334gfe5sE5nznjp+rbUlrMMMmoVf672auoa2hOWRc7N6boT74qihKPKvg8yOQLz3msDDdOE3Er8UwWPMD8z3YDsHrHQRau35N2nOgvjELCJKPhkW6Z/jxvXfycCcNH0ljwj85by3enL85bFkVRWlAFnwdFteAz+dUT27qt/SzpgsMuLf71h+bH1T21YCMTH18AFCcXTTCFJd6cwmJ3EzEm1s+t4H/6wie8UMR0w4rSnlEFnwdxG35kUYxN4cx+l1wictwkWsSJXSORzIO99om9d2umcEqvRN08biUeSfA3JbloLHcop/poFMUPVMHnQS4+4482Z86zntOOThms/UQd2ezRoZ+Y4jcfAilcLeFIssXuJuxqm801pShKfqiCz4M4qzuLYmxozmLB55KLxnWc/CRrgkL1uHKItLhX9h5qYvCtLzHjw9xSCcRcLS4ZmhMUfKI4lmVii1I6WdWyV5TC0GySeZCL1d0YjmSsz3fDj+R88PGE8whN2bjX3n3p4bfWculpR2Ztf/1jC2gMR1K6WsJZXDSROHdOOgXvSWxFUdLgyYIXkSu8lLUbcsgm2ZjVB58fWW/Iehy4ZctAQ4cqe4eg+ubMi1KUWcu389aqXSlvliZa8NE6d8RNJgvemGImHVaU9olXF81tHsvaBbkonuwWfC4umvQx8/m6M6K9LAM1IUfBN3lT8FFafPAtZeFIggVvTFxby2r5PKkM+MQFQlGU3MnoohGRLwFfBo4Ukd+5qroC7XZ7nTtmLGPzvnp6dKxOyjSZSGMWH3wueixTzHy+6rAlvUBLRE22RSmRVFE0iS6ieAVviBgTu8GbyoJvjlgERG8RKUohZPPBbwEWAF8BFrrK64Cb/RKqLfDH/1vjqd0zCzdlrH9z5U7Pc85avj12nPxkqOdhYixcv5fHnURexpiYDz5XCz5VTvlEBR816MXlokl1czZKc8SiOqQKXlEKIaOCN8YsAZaIyBPGmGYAEekBDDTG7G0NAZUW6hpafjQlqsT31+7OebyvP/RO7HjtrkNM+NN7QO5hi6maJ+bgybQBSap8PcboTVZFKRSvJtLrItJVRHoCi4A/ici9PsqlZCHR554tHDMb7sUj2z6yKaRxZErfIjaiaTlI/Azuh8IMxd3bVVHaI14VfDdjzAHga8DjxpgvAGP9E0vJRjmpvqie/nuGFAMi9qLU5Lr5+rfF8fH2jyfkfVcLXlEKw6uCD4lIP+AfgRd9lEfxSOIuU6XE4yNVvL16V0sfA797Y7Vd45j3hxojrnrV7opSKF4V/M+AmcBnxpgPRORoYJV/YuVGj45VpRahXeNVGR9qzBx4lRQZpDpeUQrC05OsxphngGdc52uAr/slVK4M6d2JvWVk0bZ1su1ClUjKG6eSfO6O3Xf3SbVBuPrgFaVwvD7JOkBE/ioiO5zXcyIywG/hvKJqoLR4fmo2S73u7KQoxcWri+ZR4O9Af+f1glNWFuhP+dLixUWTGPboPm6Jo0/fXlGU3PGq4I8wxjxqjAk7rz8DR/gol9KGKJYeTlwoVL8rSmF4VfC7ReRaEQk6r2uB3J+s8QlVBMUl1yj4VH+A5DFM1jTLbheNSREnryhKbnhV8N/CDpHcBmwFLge+6ZNMSgWQSjVn2wkr2y5ViqLkhtd88D8DroumJ3CeaL0HW/GXHrX0SoqXq+/lKde4FMjqg1eUgvFqwX/enXvGGLMHON1LR8els1hE9AGpCiWVKyVlfpm4Psn1STnuVcErSkF4VfABJ8kYELPgvVr/NwHLcxUsF1QPlBZPFjzxC8G+w02xY0mRbljj4BWlcLwq+N8A80Xk5yLyc+Ad4NfZOjmx8hcBD+cvYnb0p3xpyef6r99zOKkskrSJSZ4CKYoCeH+S9XERWQCMcYq+Zoz5xEPX+4BJQJd0DURkIjARYNCgQV7EUcoML5Z2orKO5oKPb+N9K0RFUbLjedNtR6F7UeoAiMjFwA5jzEIRGZ1h3KnAVIARI0bk9bXWn/LFRXLMVZAyH3yKMdztginqs20krihKbvi5Zc45wFdEZB3wJDBGRP7Xx/mUEuHNBx8fBx8Ktij4lq37EtqrGa8oBeGbgjfG3GaMGWCMGQxcBbxhjLnWn7n8GFXxjNdcNK52gZQWvlrwilJMdNNLpWC8+uDTLsQp9nTVXDSKUjieffCFYIyZC8z1b3y/Rm6f5LxhXx7ZJJNi3hPqNUxSUQqnIix4VQOlxWscfNy5O5tkIQMripKWilDwSmlJZY2nfJLV1S6lBW/i26p+V5TCqAgFr9EWpcXL5U9U2F729tA/q6IURkUoeKXtkdUHr8pdUQpGFbySTM4J4T2S4IKJTZfmwSp10ihKYaiCV/whhc52K2x10SiK/1SEgk9UBF89rX9pBFHSkhjXbsVZ8NE2+qCTohSTylDwCaqgc22rhPcrBZDNgrcXBFXxilIIFaHgleLihwveJCzDcT74WJuEPqrfFaUgKkLBqyJoG8S5aLKY8HqDVVEKpyIUvFL+GKM3WRWltakIBV9lGrkmOIuuHARAfIvzU7yS+DfIdJO1pVFCe7XiFaUgKkLB/2vDI/yi6hGW1k6kIw2lFkdJgz7IpCitS0Uo+M9ZO2LHn9R+q4SSKOlI1OdWlged7E26/ZVJUSqdilDwkqA+hu6dVSJJ2g95hTCazD74RJeM6ndFKYyKVPD/uP7H1NBUImnaPl72ZM2m3xOHSE42lhwmmdReTXhFKYiKVPAANwRfKoEk7YeUN0mzkJgOOFM9qAWvKIVSIQo+mR9UPcMpso5zAx+1ujztgVyVb7IPPnt7NeAVpTAq4pl+wUpZ/lLNZAAGNzzRmuK0C/Kz4DNv+JGiR85zKIrSQsVa8Er+eHDBZ/fBJ3VI3JM1eb54F45a8IpSKBWh4L1ogloamRKaSjfnYSilMLJd8mx/ES83UFW/K0phVISCT+eiidKfXVwTnMVVobnMqP4vRgY+aSXJKpdcXTQGk+FJVom1cfdQFKUwKsQHn5l3ar8XOx4c2M6T1XeqX75AsqnflKGPrmPNRaMo/lMhFrxqgmLi5Z5GNgs+VerfdDdZU/n8NReNohROhVjwuSuCY2Qzk0JPEcBwQ/MtPkjVdvEU35LZK5bdR5/qSdbEOHjV74pSEO1Wwc+u+WHLSXMRhWkn5GpdJ8XBp/DRJHrg8wnFVBSlhYpQ8IXekFtXOyF2fFPTvzHDOrdQgdo03lw0uY8bf5M1eT614BWluFSGD76ImuD+6j/w/wJLizZepZItzDEpcVjShh/ZffCKohRGZSj4It+M+0v1FM0rn4VCLfj4PVlT/2ZQF42iFIYq+DQEs8TWt3dyzfSYuOl2qidZ3a62xLh5RVFyp0IUfPGJ6pa+7GZ1zbWcLOt8mKU88ZQuuMAG2dIFe5pDUZSMVIiCL761/Y3ga9wd+m/GBhcTEotvh14kSKTo87RV8ouDd/dvOY4uKIm5aNRFoyiFUSEKvviKYFLV01wRejM29qXBd/hN1UNU08zvq+5nkGwv+pxtibx88G4XjJdcNKrfFaUgKkTB+zl2i5b5avAdRgU+5uLge/ws9GcfZy1/skbRJNQn5nf39CSranhFKYgKUfD+3RDtQGPCXDYG+2nYSnTbeFkwC80mmTJdcI5jKIqSGd8edBKRgcDjQB/s7+pUY8z9vszlx6AOk6umx50HnMVkgOxids0PCZsAIbEY0vC/mMpYLz2RcyqChAIrS5ikRtEoSuH4+SRrGLjFGLNIRLoAC0XkdWOMD7l6W18THBfYDEBIbIUfwqK5HSn4XG+yQrzLxaSw4HOdQ1GUzPimkYwxW40xi5zjOmA5cKQfcwVaMWY93a+F42UjJ8iGVpWllGRTvon1yXuyJodJJi4Aqt8VpTBaJReNiAwGTgfeS1E3EZgIMGjQoPzGb0VF8HD1b1KWv1Rze+z42003M9M6s7VEKgn5xMFnC5NMbq8aXlEKwXefgoh0Bp4D/sMYcyCx3hgz1Rgzwhgz4ogjjshzlvJSBH+svpdnqn/CFwNLSi2Kb+SXi6aFlBZ80hiKohSCrwpeRKqwlfs0Y8zzvs1ThqrgzMBKHq/+Fc9X30Ev9pdanKKTLQ7eSuGpSpeLJhXqolGUwvFNwYv9u/t/gOXGmN/6NQ+Up4KPckZgNQtrb+Tfg38rtSie8ZCpwEOYZKIFb+Ks9rgFQG+yKoov+GnBnwP8EzBGRD50Xl/2Y6JyVvBRflj1NOtqJ3CCbCi1KEUh+03W5LKsLhq3ha8b9ilKwfh2k9UY8zb+hqjHaJVJisTMmlt51zqJq5tub9Nx87la1/aTrO588C116W6yqgWvKIXRdjWMCz+fZPWDkYHlrK29luNkU6lFyZt8dG/6fPBOWWJb1e+KUhAVouDbJq/XTOJfgi+VXex8wEu64ByVb2J2yGy5aCD3fV8VRYmnIhR8W96c47+qprGm9lqqCJdalBieFHweyjfthh8pUxWkjsRRFMU7FaLg237Cr1W136Avu0stBgABDz+JsoVJJm2gTXYLPlUGSkVR8qciFHy5uTjy5d3a73KqrPHcvkNV0Bc5vOzolDUXTar6hA09svXXm6yKUhiq4MuMF2t+xNmBZSWVoRhx8JFEazwxDj7Ogk+XqiC7HIqipKciFHyoAlw0bqZX/4IrgnNLNr+3m6x5xMGn2/AjRZlx/asoSn5UhIKvJAs+yt1VU7k08HbGNl4s7Xwojg8+uUHam6wxH3xucyiKkhlV8GXM/dV/4AuyPG29X+GhxbHgk+vjLPQUN1nj69VFoyiFUhEKvi2HSWbjqZqfc4asbNU5vfwyyDXZWKLCThUmmTimxsErSmFUiIKvLB98Is/X/IRTZF1SuZdol3zwMm425ZvKgo9YqW+yxsaMKzPqolGUAqkQBV/5muClmsmcJOvjyvzaEMOLDz7XPVkNJi6yJpUPPsmCVx+NohRERSj49sIrNbfFWfJhn0xcLz74XLfsA7Dc8maLolEfvKIUjCr4NsZLNZP5x+AcAli+PQjkyUWTzQefFAcfXxYXJy/qg1cUP1AF3wb5ddWfWFN7LceZdb6M7y1MMvc4+Ijrxqv7JmzKTbdRC15RCkUVfBvm5erbWFc7gbtCf6KGpqKNW4xskqnyymTPRRM/ht5kVZTCUAVfAUwIzWFF7TdZVzuBfw6+QqFPgHq6yZo1iia5zB1FE+ehifVJ9MGrhleUQlAFX2H8uOovrKu9hnW1E7gs8FZ+m6F4ucmaZdhUPvhIGgs+SsRK7qMoSv6ogq9g7q1+iLW117KudgJfCczzrOy9WfCZSZ2LJlnBnzWkZ1JZtG1iwjJFUXJDFXw74XfVD8aU/bXB1zOmdyhKmGSiNY5JeNAJqoJCTSgQWywSwz6j514WHEVRkqkIBd9s/MmLXqncWfUoaxxlf3vof6mlMecx8slFE7Eg6GjrprBFKBBIqI+Pook4YTdB1fCKkhcVoeB313dmXu+LSy1Gm+SG0Mt8WvvPrKudwF+r7+BIdnq6uZlvHHxUWYcti1BQEJHYWOFIagteFbyi5Eeo1AIUSn19HX/d+0es/VV8HPwmwchBqqSRkDQRkkbn5RwHGgnRTEAiBCRCUJxjIgQIO+VhgoTtMgm3lGMBFgGxEDtK2y5zzgXLeRlEEs9dx9G2Ylun0bHsY2jxbhtXtkjjtHPqnYp0fSXOQ27izkVSjw+Gk9jKG9W3wTZovAPka3+C48eDY2m7H4AKN1mEMih5iRiC7vqIwQpbVCFEDETChppAgIBlEMsQCVtUByDg9LHChnDYQgyEEKxoEH2C+yhJ9ScU+JWvR1HaAlJOoWgjRowwCxYsyKmP1djI/Tf/iN47oGt9NSZQTSRYheW8RwLVWIEqrGC1cxzCSBATCGJJMHZsRN087ZLEBSFrfbYOuffJNmfOi1pSvWQ6TU2unyHbmK39GVLUt/pnyDKgu7q2cxVf+8HwLAKkG0cWGmNGpKpr8xZ8oKaGb/zmVl5Z+wrr9m6mad16mreuxWxeQ5MRmiwhbIRQRAhF7L9BwAIxIMYQsCAQhqARqi2h2gpRZQlVJki1CRAyQYImQNBIy7slBAgQMEIACBj7WAggFvY7gkTLEMQEHBs+AM67bUMLmKgNHX3iR1zHELPJjUDMpREto2UcnPLYmu0aJ0N5DImz9VvqJN7eT018vcmutbKMBybnb3GuMmSuz/szROUWwUhUrdh/u7hvdfRcXGO524jzN4jWJ7YDkIBLDPd4toIxCf0kYUwjjuJzzWGiqtCRxcSq0siTJL/72jjn4j5v+X+VVJb0uV3nidfXPT7uMZPbGuc4WmOSxk/dL3ZsXGO75U5s45bBtLQxuBS+6/sUbVoV9sfAbPMKHqBXh15ce/K19sk5TuFPusXqI8DBQID9zutAMMAhEQ4FAhwOCIclwKGAfb7LVX5IWuobAkKDCI0iRPL82V9jWVQbqMJQbQw1xlBl7ONqA9XOeU2szFDllFdjqO47jOo+p9IU6kGH2q50q6qiqmE/dZtX0/3Aerru+oia5nqqDYSwxwo584WcsULGUIVzbBlC2IsdOH716H9KZ9GJeoFiC0y0XaxO+MgazM+br2Wb6YUY+79uj9oQBw432kuJgW+ePYgPN+xhw65D1NU3ExL4XOdqBnSroa6hmSmXncp/PLmI+oYwB+qbuP1LJ/Lp5n3M+HAzXWtC/O6qYXbojbHAGEzEahHYyUxmLBNXZiyrJeeBsfua2DEtY0XLkvq3jJ2uzJ7DuOZJmMOyWvq654j1d50bgzGWI1eG8aJ905V5msMlsyOj3dc1njH2A21J/VONR1xZi9yuOaLjmdznwLKc/4Mm6ZVreTkS7N0bbh1X9HErQsGn5I69sOo1mH4lQaCbZdEt29M5HjBAGGgQiSn9BgnQKEJ9wF4AGqIv57zeqW8UoUmEJoFGEZpj5/arWeBQIGAf45TXdqEpEKLZhGlsWI+1bm164Xp3Bbrm/JlCxrUApFsY3OXGXhiqov3Yx6nmAU536j+1BvOxnMzB5iowQSDIvK77WNungd2dwxxssDAmwP4uHdnbqQOHGi3mH7Gfjwc20hQW9h6CZUNCLKuuZfHejnSrrWHL5/sRCoQISpBgIEhVoCp2HBTn3DkOSlB970pGkhaQbItF1NpJKjfRAVMsUAl9LCvtOAT9seDbvA8+K+EmaDpov2q6QtMhWP8OPH99cecpJuPvhGFXQ6feSVVhK0xTpIlmq5mmSBNNVhONkUaaIy3nC9fv5FczP+GakUeyYsc+Fq7fBRJBJAISASJx5yJhussBBgS2MSiwla6Bg4SBZhHCIjS7jwXC2IuT+zgsTpu4tqVTsiEJxRR+KBCKWxxCknAeCMXaZ2oXlCABCRAKhAhIILaYxI5dbeLaSyipzFN7p03K9tG5A97aR9sEJKCLX4WRyQdf+Qo+Hc31sHIm7PgE3votWM2p2x05Aja7ZPrOAvh9ymuZH0O+CMeMsV/9hhVvXIfdBxsZfucshvTuxNpdhzz1GTawOzP+/ZyWgsY6WPsmLJkOy1/wPLfBdo81ixABIs4C8OPwPzHTOoMhfbvSq3OIvYcbuO+qz3PVn96hR4cgG/bWcftFJ7Bww25eXbaFLh2Ee64YStiEiVgRwlaYiLHfo8cRK0LYpDm3It7aW+HYHLH2CXNaxoq1jx1Hy51+EVPeO4wlLjLuBSO6aLgXr1AghIjEzgMEYm0EscfAXjwCAbs++isq1ifhlao8pz7OfMGAI0OavtnmSeyTaZxYeYrPH5Uh+vlTze/jDmyVe5M1b6o6wClftV/nT7bLIs2wayVsWQyDzrYt/dOvhYfHwuaFcMKXofdxqcc783pY9jf7uPkw9DoGOveBTkdA589Bx97Qd6hd1uMoqOqYcLPIH3p1rmHdlIti5x9t2s/MZdv4/ZzVSW2fu/Fs7nxpedJTqNR0gRMvsl8OD85Zzd0zV9CDA4wLLuLa4CyGBdbEdRPs/2Ah989Y4PeBxyDwGOwCdsFjHa/jxE5fovlQf7p26kykfj8ndD+dDVu2EznUhQDVjDuq+P5JP3ErfMtYhE0Yy0pYEJyFItbGCmMZq6V9wqKRz5hJ7RPKktq7Fi93+6hcFnYby7KP3fJGTAQ7xUTLe6yf80osy9rHmaMSECTlwhUgQK8OvXjhMu/Gk1far4JPRbAK+pxiv8BW0gA3vBHf7qaltvukqqN9vnct9DwaLvpN68maJ0MHdOPEfl0QgcuHD2DT3nreX7uHZVv2M/yonnTvUMWcFTvZc6iJnp2q047T2GxbqXvpyjOR0TwTGR1XX0MTZwc+4brgTM4PLsko03WHH4NfPsayALALPq4ejNl6J03NAwAop1+ZXglIgEAwQBVVpRalzWOMiVP2qRaHbAtJpj7ZFpmIidhzuxa1iBVxtqHMXYbYZ3HGM8bQIdTBl2vXfl00SkoeeXstP3vxE7p3rOKOi0/m0tOOTPkk6V0vL2fqm2tSjJCam8cdz+Pz13FKnxo6rJvNv4Re4azACu+C9RkK502CY8dBdUfv/RSlwlEXjeKZb507hJP6deWul5fz/aeX8Ie5n/HdMcfy5aH9qAq2ZLY4UJ/6nsV3xxzLA28ku38ADjdFqKruwEzrLGY2ncXXzxjA/vomth1o4NGrjmfKvb/llk4v0795Q3Ln7R/B0//Uci5BOGqUfQ+j17H2L6ieQ+wb6XoTUVEAVfBKCs4+phcz/v0cXl22jXtfX8lNT37IlFc+5dqRR3HFiAF8rkstuw6mTlB2y/gTmPHhFjbsORxXvnnfYeqbIwzo0fJTtH/3WvbXN2EMvLslwnPWFzn+3In88pVPmfK1odz6/EdUEeac2nX8+dSl8PGzLQOaCKx7y3656dADOveFzkfY9z069oLabtChu/1e3QmqO9vutepO9r2YQBUEQxAI2ceBEASCtssuVlYRaZuUdoYqeCUlgYDw5aH9uPCUvsxZsYNH563j7pkruOe1FZw1uCcfbtyX1OfI7rbybgzb/vnLTj+SK4YPYMLD7/H0gk0AnDmkJ4/NXw/A2JP6sHxrHcbAvbNWAnB8ny4A3Pr8RwA0E2KxnASX38Thr/yR4T+fxbCB3Zh2/UiCAmxfZt8DObgdDu+FA5vh4A44vAu2LrHfGw5AwRt4i63sY0rf9cq6OATT9A06T6IGnSdAA/YrVu6lLtBSn3OdU5+2LmAvbJ7qnBcSP6/7Cd609UVqryThq4IXkQuB+4Eg8LAxZoqf8ynFJxAQxp7Uh7En9WH1jjpeWLKVF5duQQQmfvFopr65hqvPGkTvztVcfdYgAC47fQDvrd3NpAtPoG/XWgb17MiGPYc59ciunHNMS2z/sAHdqKkK8MnWAwCMO+lzdO0Q/1/yurOP4vlFm2mOWPzwmaXUN0d4d80ejpn8MpcM68/N445jyImnZA5BsyxoqoOG/dB02Hku4pAd7dR8GKwIWGE7isoKx78izU69qy4SrY+WRTL3DTeCdTC5r7GcJzrdr0jLcaa6CoksKSpFX0DE43hOuoi85xf7l+fljxT/kvh1k1VEgsBK4B+ATcAHwNXGmE/S9dGbrG2HiGWn/t19sJGenaozKtjGcIQV2+o4qV9XqoIBGpoj1FbZT+5t2VfPM451/2/nH0NDc4T/+tvHDO7die+NOY57XlvBH+Z+Rk0oQGM4vVIb3Ksj40/py6hjetGrUw1VIaEqGKA6GKA6FCAYEIIiBERs41OEgNjvIrTUCW3jQSBXSoDkhSF6bDLURevzqUt4Resw8fNCwnlifSHtjYfx8m2PR3njUzPk/PncY3ToAde6XJA5UJIHnUTkbOAnxpgLnPPbAIwxv0zXRxW8kkh9U4RZy7ezeMM+aqsCfGfMsXSsDvHx5v3c89oK1u8+zJ5DTViWoa4xXJQ5RVoWAIm+x5J1OW1ibSXunHT1acrjy+JLkvtEz7OPGT9ednJq6yFRXK7j5rKk5rIA57RU53QNcmjrQd6eHat5+l/PzmHUuPFLEkVzJLDRdb4J+EJiIxGZCEwEGDRokI/iKG2RDtVBLhnWn0uG9Y8rP/XIbvz5n8+KK9tf38ynWw9Q1xCmKWLRHLFoCls0Rwxhy8KyDJaxNx4xznv0PL6u5TgSy0PSEo8fO4eE8/h6SOiX0D5Vn+Q2CfVp5koniyf8aer5+YXcxsyhbU7jem+dk0nssXGXWn9UcclvshpjpgJTwbbgSyyO0obp1qGKLxzdq9RiKErZ4Gfs12ZgoOt8gFOmKIqitAJ+KvgPgONEZIiIVANXAX/3cT5FURTFhW8uGmNMWES+A8zEDpN8xBizzK/5FEVRlHh89cEbY14GXvZzDkVRFCU1+vy1oihKhaIKXlEUpUJRBa8oilKhqIJXFEWpUMpqww8R2Qmsz7N7b+xN4MqVcpcPVMZiUO7yQfnLWO7yQXnJeJQx5ohUFWWl4AtBRBaky8dQDpS7fKAyFoNylw/KX8Zylw/ahoygLhpFUZSKRRW8oihKhVJJCn5qqQXIQrnLBypjMSh3+aD8ZSx3+aBtyFg5PnhFURQlnkqy4BVFURQXquAVRVEqlDav4EXkQhFZISKrReTWEsoxUETmiMgnIrJMRG5yynuKyOsissp57+GUi4j8zpF7qYic0UpyBkVksYi86JwPEZH3HDmeclI7IyI1zvlqp35wK8nXXUSeFZFPRWS5iJxdTtdQRG52/r4fi8h0Eakt9TUUkUdEZIeIfOwqy/maich1TvtVInJdK8h4t/N3XioifxWR7q662xwZV4jIBa5y377vqWR01d0iIkZEejvnJbmOOWOcLcra4gs7DfFnwNFANbAEOLlEsvQDznCOu2BvOH4y8GvgVqf8VuBXzvGXgVewt3ccCbzXSnJ+H3gCeNE5fxq4yjn+b+BG5/jfgP92jq8Cnmol+R4DrneOq4Hu5XINsbehXAt0cF27b5b6GgJfBM4APnaV5XTNgJ7AGue9h3Pcw2cZxwMh5/hXLhlPdr7LNcAQ5zse9Pv7nkpGp3wgdtrz9UDvUl7HnD9TqSYu0h/kbGCm6/w24LZSy+XIMgP4B2AF0M8p6wescI7/CFztah9r56NMA4DZwBjgRec/5y7Xlyx2PZ3/0Gc7xyGnnfgsXzdHgUpCeVlcQ1r2Ge7pXJMXgQvK4RoCgxOUZ07XDLga+KOrPK6dHzIm1F0GTHOO477H0evYGt/3VDICzwLDgHW0KPiSXcdcXm3dRZNqY+8jSyRLDOen+OnAe0AfY8xWp2ob0Mc5LoXs9wGTAMs57wXsM8aEU8gQk8+p3++095MhwE7gUceN9LCIdKJMrqExZjNwD7AB2Ip9TRZSXtcwSq7XrNTfpW9hW8RkkKXVZRSRS4HNxpglCVVlI2Mm2rqCLztEpDPwHPAfxpgD7jpjL+kliUsVkYuBHcaYhaWY3yMh7J/IDxljTgcOYbsXYpT4GvYALsVeiPoDnYALSyFLLpTymnlBRG4HwsC0UsviRkQ6ApOBO0otS760dQVfVht7i0gVtnKfZox53ineLiL9nPp+wA6nvLVlPwf4ioisA57EdtPcD3QXkejOXm4ZYvI59d2A3T7KB7a1s8kY855z/iy2wi+XazgOWGuM2WmMaQaex76u5XQNo+R6zUryXRKRbwIXA9c4C1E5yXgM9mK+xPneDAAWiUjfMpIxI21dwZfNxt4iIsD/AMuNMb91Vf0diN5Jvw7bNx8t/4ZzN34ksN/1k7roGGNuM8YMMMYMxr5ObxhjrgHmAJenkS8q9+VOe1+tQGPMNmCjiJzgFI0FPqFMriG2a2akiHR0/t5R+crmGrrI9ZrNBMaLSA/nl8p4p8w3RORCbJfhV4wxhxNkv8qJQhoCHAe8Tyt/340xHxljPmeMGex8bzZhB1Jso4yuY0ZK5fwv1gv7bvZK7Lvrt5dQjnOxfwYvBT50Xl/G9rnOBlYBs4CeTnsBHnTk/ggY0YqyjqYliuZo7C/PauAZoMYpr3XOVzv1R7eSbKcBC5zr+DfsSISyuYbAT4FPgY+Bv2BHepT0GgLTse8JNGMroX/J55ph+8FXO69/bgUZV2P7q6Pfl/92tb/dkXEF8CVXuW/f91QyJtSvo+Uma0muY64vTVWgKIpSobR1F42iKIqSBlXwiqIoFYoqeEVRlApFFbyiKEqFogpeURSlQlEFr1QMIvKO8z5YRCYUeezJqeZSlHJGwySVikNERgM/MMZcnEOfkGnJJ5Oq/qAxpnMRxFOUVkMteKViEJGDzuEU4P+JyIdi528POrnHP3Byd3/baT9aRN4Skb9jP5GKiPxNRBaKnfN9olM2BejgjDfNPZfzJOPdYueH/0hErnSNPVdacttPc55+RUSmiL1vwFIRuac1r5HSvghlb6IobY5bcVnwjqLeb4w5U0RqgHki8prT9gzgVGPMWuf8W8aYPSLSAfhARJ4zxtwqIt8xxpyWYq6vYT99Owzo7fR506k7HTgF2ALMA84RkeXYqXFPNMYYcW1yoSjFRi14pT0wHjtvyIfYKZx7Yec3AXjfpdwBviciS4B3sZNGHUdmzgWmG2MixpjtwP8BZ7rG3mSMsbAfxR+MnTK4AfgfEfkacDh5SEUpDqrglfaAAN81xpzmvIYYY6IW/KFYI9t3Pw57k45hwGLsfDL50ug6jmBvChIGzsLOlHkx8GoB4ytKRlTBK5VIHfa2iVFmAjc66ZwRkeOdjUQS6QbsNcYcFpETsbdii9Ic7Z/AW8CVjp//COxt395PJ5jY+wV0M8a8DNyM7dpRFF9QH7xSiSwFIo6r5c/Yee8HY+fyFuxdo76aot+rwL86fvIV2G6aKFOBpSKyyNhplqP8FXsruSXY2UQnGWO2OQtEKroAM0SkFvuXxffz+oSK4gENk1QURalQ1EWjKIpSoaiCVxRFqVBUwSuKolQoquAVRVEqFFXwiqIoFYoqeEVRlApFFbyiKEqF8v8BqVLCWxWL8ogAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learning_rates = [0.1, 0.01, 0.001, 0.0001, 0.00001]\n",
    "models = {}\n",
    "\n",
    "for lr in learning_rates:\n",
    "    print ('Training model with learning rate: ' + str(lr))\n",
    "    models[str(lr)] = model(train_set_x, train_set_y, test_set_x, test_set_y, num_iterations=1500, learning_rate=lr)\n",
    "    print('---')\n",
    "\n",
    "for lr in learning_rates:\n",
    "    plt.plot(np.squeeze(models[str(lr)]['costs']), label=str(models[str(lr)]['learning_rate']))\n",
    "\n",
    "plt.ylabel('cost')\n",
    "plt.xlabel('iterations')\n",
    "\n",
    "legend = plt.legend(loc='upper center', shadow=True)\n",
    "frame = legend.get_frame()\n",
    "frame.set_facecolor('0.90')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}