Single_Layer_Perceptron.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Single-Layer Perceptron\n",
    "\n",
    "### Joseph Melby, 2023"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook contains an implementation of a single layer Perceptron classifier. This algorithm\n",
    "serves as a simple case for the more general framework for implementing machine learning algorithms\n",
    "from scratch (not using Scikit-Learn or any other ML libraries, except for some data splitting,\n",
    "etc.). This particular implementation only uses NumPy, pandas, and some visualization tools.\n",
    "\n",
    "This framework was introduced to me through this Towards Data Science blog post:\n",
    "https://towardsdatascience.com/6-steps-to-write-any-machine-learning-algorithm-from-scratch-perceptron-case-study-335f638a70f3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Basics\n",
    "\n",
    "A single layer perceptron is the most basic neural network and is used primarily for binary\n",
    "classification. It uses a linear decision boundary, so classification problems with nonlinear\n",
    "decision boundaries are not a great fit for this classifier. \n",
    "\n",
    "Typical uses: loan default prediction, sentiment analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Toy example for the input features:\n",
    "X = np.array([[0, 1, 0],\n",
    "              [0, 0, 1],\n",
    "              [0, 1, 1]])\n",
    "\n",
    "# Output:\n",
    "y = np.array([1,\n",
    "              1,\n",
    "              0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0. 0. 0.]\n"
     ]
    }
   ],
   "source": [
    "# Initializing a weight vector with rows corresponding to three features\n",
    "w = np.zeros(X.shape[0])\n",
    "print(w)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n"
     ]
    }
   ],
   "source": [
    "# Multiply the weights by the features\n",
    "f = np.dot(X[0], w)\n",
    "print(f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compare this value against a threshold\n",
    "\n",
    "#Threshold:\n",
    "z = 0\n",
    "\n",
    "def activation(f, z = 0):\n",
    "    if f > z:\n",
    "        y_hat = 1\n",
    "    else:\n",
    "        y_hat = 0\n",
    "    return y_hat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n"
     ]
    }
   ],
   "source": [
    "y_hat = activation(f)\n",
    "print(y_hat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since our prediction of y_hat[0][0] = 0 and the actual value was y0 = 1, we will need to update our\n",
    "weights. To update the weights, we will use the equation \n",
    "\n",
    "w_new = w + eta*(y - y_hat)x\n",
    "\n",
    "Here, eta is the learning rate. We can iterate this process until we get the desired accuracy in our classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Choose eta (this can be adjusted later as needed)\n",
    "eta = 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.  0.1 0. ]\n"
     ]
    }
   ],
   "source": [
    "# Adjusting the weights \n",
    "w = w + eta*(y[0] - y_hat)*X[0]\n",
    "print(w)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.1\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "# Test the threshold again:\n",
    "f = np.dot(X[0], w)\n",
    "print(f)\n",
    "\n",
    "y_hat = activation(f)\n",
    "print(y_hat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looks like we got the correct prediction for the first entry y[0] = 1, but we still have the other\n",
    "values to predict correctly. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Perceptron Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Total Perceptron function using Sum-of-squared error\n",
    "\n",
    "def perceptron(X, y, z = 0.0, eta = 0.1, num_it = 50):\n",
    "    \"\"\"_summary_\n",
    "\n",
    "    Args:\n",
    "        X (2darray): input data set\n",
    "        y (1darray): output\n",
    "        z (int, optional): activation function threshold. Defaults to 0.\n",
    "        eta (float, optional): learning rate. Defaults to 0.1.\n",
    "        num_it (int, optional): number of iterations. Defaults to 10.\n",
    "\n",
    "    Returns:\n",
    "        1darray, list: array of weights and SSE vector\n",
    "    \"\"\"\n",
    "\n",
    "    # Initalize the weights\n",
    "    w = np.zeros(len(X[0]))\n",
    "    n = 0  #Iteration number\n",
    "\n",
    "    # Initialize other parameters to compute SSE\n",
    "    y_hat_vec = np.ones(len(y))   # predictions\n",
    "    errors = np.ones(len(y))   # errors = actual - predictions\n",
    "    J = []   # vector for SSE cost function\n",
    "\n",
    "    while n < num_it:\n",
    "        for i in range(0, X.shape[0]):\n",
    "            f = np.dot(X[i], w)\n",
    "\n",
    "            y_hat = activation(f)\n",
    "            y_hat_vec[i] = y_hat\n",
    "\n",
    "            # Update weights\n",
    "            for j in range(len(w)):\n",
    "                w[j] = w[j] + eta*(y[i] - y_hat)*X[i][j]\n",
    "            \n",
    "        n += 1\n",
    "\n",
    "        # Computing SSE\n",
    "        for i in range(len(y)):\n",
    "            errors[i] = (y[i] - y_hat_vec[i])**2\n",
    "\n",
    "        J.append(0.5* np.sum(errors))\n",
    "\n",
    "    return w, J, y_hat_vec\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Simple accuracy function to use for our classifier\n",
    "def accuracy(true, predicted):\n",
    "    return np.sum(true==predicted)/len(predicted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Toy Example\n",
    "\n",
    "Let's consider the NAND gate as an example to see this a little more formally."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set up the input and output\n",
    "# Note that x0 is a dummy variable fo the bias term\n",
    "\n",
    "#             x0, x1, x2\n",
    "X = np.array([[1, 0, 0],\n",
    "              [1, 0, 1],\n",
    "              [1, 1, 0],\n",
    "              [1, 1, 1]])\n",
    "\n",
    "y = np.array([1,\n",
    "              1,\n",
    "              1,\n",
    "              0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The weights are:\n",
      "[ 0.2 -0.2 -0.1]\n",
      "The errors are:\n",
      "[1.0, 1.5, 1.5, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]\n"
     ]
    }
   ],
   "source": [
    "w, J, pred = perceptron(X, y, z = 0.0, eta = 0.1, num_it = 50)\n",
    "\n",
    "print(\"The weights are:\")\n",
    "print(w)\n",
    "\n",
    "print(\"The errors are:\")\n",
    "print(J)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "accuracy(y, pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bigger Synthetic Data Set\n",
    "\n",
    "Let's try this out on a slightly larger example data set, which is generated below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Generate the random data\n",
    "np.random.seed(440)\n",
    "n_rows = 100\n",
    "diff = .1\n",
    "X = np.random.random((n_rows,2))\n",
    "X_prime = X[(X[:,1] - X[:,0]) <= -diff,:]\n",
    "X_2prime = X[(X[:,1] - X[:,0]) >= diff,:]\n",
    "\n",
    "del X\n",
    "X = np.append(X_prime,X_2prime,axis = 0)\n",
    "\n",
    "y = np.empty(np.shape(X)[0])\n",
    "y[(X[:,1] - X[:,0]) <= -diff] = 1\n",
    "y[((X[:,1] - X[:,0]) >= diff)] = 0\n",
    "\n",
    "X[1,:] = [.8,.1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(75, 2)\n",
      "(75,)\n"
     ]
    }
   ],
   "source": [
    "print(X.shape)\n",
    "print(y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmcAAAJbCAYAAAChCjPHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAABNW0lEQVR4nO3de3RU9b3//9dkMtzETPCGoICRmICSaA2VmOhq7bFYbLU0bcV6VgNSz+/QoigcrSIexUuL9vZVOYC2iHBW0XKqgdPTRT1mnbbKRWmhgGnDKg1ewApFMDe0hTDZvz+2ExIy12Rm78/s/XysNStmZid8JhOTV96fz+f9CViWZQkAAABGyHN7AAAAADiBcAYAAGAQwhkAAIBBCGcAAAAGIZwBAAAYhHAGAABgEMIZAACAQfLdHoBbOjs79d577+nUU09VIBBwezgAAMDjLMtSe3u7Ro4cqby8+PUx34az9957T6NGjXJ7GAAAwGf27dunc889N+7jvg1np556qiT7C1RQUODyaAAAgNe1tbVp1KhRXRkkHt+Gs+hUZkFBAeEMAAA4JtlyKjYEAAAAGIRwBgAAYBDCGQAAgEEIZwAAAAbx7YYAAADQP5FIRB0dHW4Pw3WhUEjBYDBjn49wBgAA0mJZlg4cOKDW1lZZluX2cFwXCAQUDod19tlnZ6SxPeEMAACkpbW1VS0tLTrzzDN1yimn+PqkHcuy9OGHH+r999/X4MGDVVhY2O/PSTgDAAApsyxLBw8eVEFBgc444wy3h2OEwYMH6+jRozp48KDC4XC/wyobAgAAQMoikYgikQgN3E9SUFDQ9bXpL8IZAABI2fHjxyVJ+flMvnUX/XpEvz79QTgDAABp8/M6s1gy+fUgnAEAABiEcAYAAGAQwhkAAMDHjhw5ojvuuEMjR47UoEGDdMkll+hnP/uZo2MwIpy9+uqruu666zRy5EgFAgGtW7cu6ce88sorqqio0KBBg3T++efrqaeeyv5AAQBAVjQ0SA8+KM2da79taHBnHDU1NVq1apUeeOAB/epXv9InP/lJfe1rX9Nzzz3n2BiM2Grx4Ycf6uKLL9bNN9+sL3/5y0mvf+utt3TttdfqX/7lX/TTn/5UmzZt0re+9S2deeaZKX08AAAwQ1OTNH26tHmzFAxKeXlSZ6e0cKFUXS2tXCkVFzszlvXr16u+vl7PPfecvva1r0mSrrrqKr3zzju66667NG3atIwe0xSPEeFsypQpmjJlSsrXP/XUUxo9erQef/xxSdL48eO1detW/eAHPyCcwVUNDVJdndTSIhUWSjU1UlmZ26MCADM1NUmTJkmtrfb7kYh9i3r9dfvxLVucCWhr167V0KFD9dWvfrXH/TfffLNuuukmbdmyRVVVVVkfhxHTmul67bXXNHny5B73XXPNNdq6dWvcA1iPHj2qtra2HjcgU5qa7L/wysulhx+Wliyx35aXS1dcYT8OAOhp+nQ7mMXr2xqJ2I/PmOHMeP74xz9q/PjxvXq4lZeXdz3uhJwMZwcOHNDw4cN73Dd8+HAdP35chw4divkxixYtUjgc7rqNGjXKiaHCB6J/+W3ZYr8fiUgdHSd+2ET/8iOgwW2mrOkBJPv7b/Pm+MEsKhKRNm1y5vv18OHDOu2003rdH73v8OHD2R+EDJnW7IuTm71ZlhXz/qj58+dr3rx5Xe+3tbUR0JAR6fzlt3Gjo0MDJJm1psdPWOaQWF2d/f2YymlHwaC0dq0zX79EzWSdarybk+Hs7LPP1oEDB3rcd/DgQeXn5+v000+P+TEDBw7UwIEDnRgefCT6l18y3f/y44cznGTamh4/IAynpqXF/tqkEs7y8qTm5qwPSaeffnrM6tgHH3wgSTGratmQk9Oal19+uerr63vc9/LLL2vixIkKhUIujQp+FP3LLxXRv/wAJ5m2psfrWOaQusJCO7SmorNTGjYsq8ORJJWVlWnXrl29zsds+HhOdcKECdkfhAwJZ0eOHNGOHTu0Y8cOSXarjB07dmjv3r2S7CnJ2trarutnzZqld955R/PmzdOuXbu0YsUKPfPMM7rzzjvdGD58LPqXXyqc+ssPiDJxTY/XEYZTV1OTWtVMsq+rqcnueCTpS1/6ko4cOaIXX3yxx/2rVq3SyJEjNWnSpOwPQoZMa27dulVXXXVV1/vRtWHTp0/XypUrtX///q6gJklFRUVav3695s6dqyVLlmjkyJF68sknaaMBx5n4lx8QZeqaHq9imUN6ysqkqiq7ypjoezQYlCorJSeKVlOmTNFnP/tZffOb31RbW5uKi4v1/PPP66WXXtJPf/pTR3qcSYaEs09/+tNdC/pjWblyZa/7PvWpT+kPf/hDFkcFJFdTY68jSYVTf/kBUSau6fEywnD6Vq06sSYy1tctGJTCYXudnlPq6uq0YMEC3X///frggw80btw4Pf/887rxxhsdG4MR05pAror+5Zfsj6lg0F4I7NByBUASlV2nscwhfcXFduWsstJ+PxiUQqETP1MrK53frDJ06FA98cQT2r9/v44ePaqdO3c6GswkQypnQC4z8S8/QKKy6zTCcN8UF9tthhoa7Gpic7P9tamp8e8ftIQzoJ+if/nNmGGvI+m+dT4Ssf/yY+s83GDimh4vIwz3T1kZ07xRhDMYJVebNvKXH0xFZdc5hGFkCuEMRkjUtHHUKOmf/kk67zzzwxp/+cE0VHadRRhGJhDO4LpkHcz37bN/kOXl0WEb6Asqu84hDCMTCGdwXbKmjVHRhbYcNwP0DZVdZxCG0V+EM7gq1aaN3XGQOIBcQBhGX9HnDK5K52zK7jhuBgDgVYQzuCqdpo0n4yBxAIAXEc7gqnSaNp6MDtsAAC8inMFVNTWpnUMXCx22AQCZ1t7erm9/+9uaPHmyzjzzTAUCAS1MtbtwhrAhAK5KtWljLHTYBoAc1vyGtGe5JCuFiwPS2FukYeXZHpUOHz6sH//4x7r44os1depULV++POv/5skIZ3BdsqaNsdBhGwBy3PsbpN2LpUBQiSfyOiUrIhWMcyScjRkzRs3NzQoEAjp06JAr4YxpTbgu2rSxstJ+P9nuTTpsA4AHFNVKoUI7eFkdCW4R+7qiWkeGFQgEFAgEHPm34iGcwQjRpo1vvCHdf7/dw2zUKPuxYFAKhU6EtspKGtACQM4LnSpdeLekZEEoIF10jxQa6sSojMC0JoxyctNGOmwDgIeVzJYaH5M6WuJfEwpLF8x2bEgmIJzBaHTYBgAPi1bPdt6r2BsD/Fc1k5jWBAAAbiqZbVfHYvFh1UwinAEAADfFXXvmz6qZRDgDAABui1U982nVTGLNGQAAcFuvtWfuVs1+9atf6cMPP1R7e7skqbGxUS+88IIk6dprr9WQIUOy+u8HLMtKpTWv57S1tSkcDqu1tVUFBQVuDwcAgJzwj3/8Q2+99ZaKioo0aNCgzH3ijnZp3Wh752aoUJq6z7Vwdt555+mdd96J+dhbb72l8847r9f9qXxdUs0eTGsCAAD3da09k+trzd5++21ZlhXzFiuYZRrTmgZqaJDq6qSWFqmw0O7tRTsJAIDnlc6R8odKY2e6PRJXEc4M0tQkTZ8ubd5sd8PPy5M6O6WFC6Xqavu4IrriAwA8K3+IVHqr26NwHdOahmhqsg//3rLFfj8SkTo6ThwE/vrr9uNNTe6NEQAAZB/hzBDTp0utrSfC2MkiEfvxGTMcHRYAAHAY4cwADQ32VGa8YBYViUibNtnXAwAAbyKcGaCuzl5jlopg0D4IHAAAN/m0E1dcmfx6EM4M0NJiL/5PRV6e1Nyc1eEAABBXKBSSJH300Ucuj8Qs0a9H9OvTH+zWNEBhob0rMxWdndKwYVkdDgAAcQWDQRUWFurgwYOSpCFDhigQOPlcTP+wLEsfffSRDh48qMLCQgVTnQpLgHBmgJoau11GKiIR+3oAANxy9tlnS1JXQINUWFjY9XXpL8KZAcrKpKoqu41Gok0BwaBUWSlNmODc2AAAOFkgENCIESN01llnqaOjw+3huC4UCmWkYhZFODPEqlV2H7N47TSCQSkcthvRZgOnEgAA0hUMBjMaSmBjQ4Ahiovtylllpf1+MCiFQid2cVZW2o9n+oSApib79IHycunhh6UlS+y35eXSFVfQ9BYAAKcFLJ/uhU31ZHg3NDTY7TKam+3F/zU12ZnKjJ5KkKxal41QCACA36SaPZjWNFBZmTNTiumcSrBxY/bHAwAAmNb0LU4lAADATIQzn+JUAgAAzEQ48ylOJQAAwEyEM5/iVAIAAMxEOPOpmprk682iOJUAAADnEM58KnoqQbJ1Z8Gg3QeNUwkAAHAGrTSyJBc67rt9KgEAAOiNylmG5VLHfbdOJQAAAPFxQkAGTwjI5Y77Tp1KAACAX3FCgAtyueO+U6cSAACAxJjWzBA67gMAgEwgnGUIHfcBAEAmEM4yhI77AAAgEwhnGULHfQAAkAmEswyh4z4AAMgEwlmG0HEfAABkAuEsg1atsvuYxQtodNwHAADJEM4yiI77AACgv2hCm2HFxXaDWTruAwCAviCcZQkd95FtDQ12f72WFnu3cE0N33MA4AWEMyDHNDXZR4Vt3mxPmefl2e1ZFi60N5usXMnUOQDkMtacATmkqUmaNMleuyjZbVk6Ok60cXn9dfvxpib3xggA6B/CGZBDpk+XWlvj99SLROzHZ8xwdFgAgAxiWhPIEQ0N9lRmMpGItGmTfT1r0JBtqax9ZH0kkB7CGZAj6ursNWapnEQRDNq7hfkFiGxJZe2jxPpIoC8IZ0COaGmxf7mlEs7y8uw2LkA2RNc+trba70ciPb8vX39dmjjR/u8jR+JfE10/SUADemLNGZAjCgvtqkMqOjvt/npANqS69pH1kUDfEM6AHFFTk1rVTLKvq6nJ7njgT9G1j6l+LybSfX0kgBMIZ0COKCuTqqrin90aFQza63k4kQLZEF37mCnR9ZEATmDNGZBDVq06sdYnVuUiGJTC4ROLsdE/7DLsLZ21j6lgfSTQG5UzIIcUF9sLqCsr7feDQSkUOlHJqKxkgXUmNDXZ1cfycunhh6UlS+y35eXSFVf4u8lvOmsfU8H6SKC3gGVZltuDcENbW5vC4bBaW1tVUFDg9nB64S92JNPQYE8HNTfbv9xqapjKzITuOxETVSf9GoIbGuyQmunPyfcu/CDV7EE4MyycxesdFInQFwhwQnW1HbwSTdsFg3aVcuNG58ZlklS+Rqnw+9cR/kM4S8LEcMZf7IC70q0KvfFGdiraplfOm5rsPmbRPmexnHqq/cflkSP8PAOiUs0erDkzCOcmAu5KZydiNnYZemmtW16e9OKLrI8E+oLdmobg3ETAfW6ewpBK131TOupPn36i8388R45IDzxgT1myPhJID+HMEJybCLjPzVMY0qmcu7lGq69/SPLzCkgd05qGiP7Fngr6AgHZ4dYpDKl23U+no35Dg/Tgg9LcufbbTHXhd3vqF/ADKmeG4NxEwH3RUxhS3a2Zqam5TFbO4+34XrgwMzu+3Zz6BfyCypkhODcRMMOqVfYuwnjVoWycwpCpynl03dqWLfb7kYjU0XHiZ0t03Vp/NhbwhySQfYQzQ3BuImAGN05hyFTgcWLHN39IAtlHnzP6nAGIw6ldhun2V4vVUd/JHm006gX6JtXswZozg0T/Yp8xw170e/IJAZWVnBAAOMmpXYaZWOvm5I7vVatS+0Myk1O/gJ8QzgxTXExfIMCP+ht4nFyozx+SQHYRzgxFXyDAX/obeJxeqM8fkkD2sObMoDVnACD1LfBkYt0agOxizRkA5Ki+VM7d6tEGIPNopQEAHuFGjzYAmUc4AwCPcKNHG4DMY1oTADyEhfpA7iOcAYAHseMbyF3GTGsuXbpURUVFGjRokCoqKrRhw4aE169evVoXX3yxhgwZohEjRujmm2/W4cOHHRotAABAdhgRztasWaM77rhDCxYs0Pbt23XllVdqypQp2rt3b8zrN27cqNraWn3jG9/Qn/70J/385z/X73//e91yyy0OjxwAACCzjOhzNmnSJF166aVatmxZ133jx4/X1KlTtWjRol7X/+AHP9CyZcu0Z8+ervsWL16s733ve9q3b19K/yZ9zgAAcEHzG9Ke5ZJSiR8Baewt0rA0mvgZLGf6nB07dkzbtm3TPffc0+P+yZMna/PmzTE/pqqqSgsWLND69es1ZcoUHTx4UC+88II+//nPx/13jh49qqNHj3a939bWlpknAAAAUvf+Bmn3YikQVOIJvE7JikgF4zwTzlLl+rTmoUOHFIlENHz48B73Dx8+XAcOHIj5MVVVVVq9erWmTZumAQMG6Oyzz1ZhYaEWL14c999ZtGiRwuFw123UqFEZfR4AACAFRbVSqNAOXlZHglvEvq6o1u0RO871cBYVCAR6vG9ZVq/7ohobGzVnzhzdf//92rZtm1566SW99dZbmjVrVtzPP3/+fLW2tnbdUp3+BAAAGRQ6Vbrwbkmxf8efEJAuukcKDXViVEZxfVrzjDPOUDAY7FUlO3jwYK9qWtSiRYtUXV2tu+66S5JUXl6uU045RVdeeaUeeeQRjRgxotfHDBw4UAMHDsz8EwAAAOkpmS01PiZ1tMS/JhSWLpjt2JBM4nrlbMCAAaqoqFB9fX2P++vr61VVVRXzYz766CPl5fUcevDjFtgG7G8AAACJJK2e+bdqJhkQziRp3rx5Wr58uVasWKFdu3Zp7ty52rt3b9c05fz581Vbe2LO+brrrlNdXZ2WLVumN998U5s2bdKcOXN02WWXaeTIkW49DQAAkKqS2XZ1LBYfV80kA6Y1JWnatGk6fPiwHnroIe3fv18TJkzQ+vXrNWbMGEnS/v37e/Q8mzFjhtrb2/Uf//Ef+rd/+zcVFhbqM5/5jB577DG3ngIAAEhHtHq28171bKvh76qZZEifMzfQ5wwAAJd1tEvrRvdcexYqlKbu82Q4SzV7GDGtCQAAfKjX2jOqZpIh05oAAMBnoicFdB6V8kJS5zH77ZE3pa23nXSxt04KSIZwBgAAnNf9pIDoCqvO49KeZ0+6MAsnBRh+hBThDAAAOK+oVtp530m9zjolq7P3tZk+KcDwI6RYcwYAAJzn5kkBhh8hReUMALppaJDq6qSWFqmwUKqpkcrK3B4V4FFunRQQt43HydzZoEDlDAAkNTVJ1dVSebn08MPSkiX22/Jy6Yor7McBZJibJwUkaoIb5VIzXMIZAN9rapImTZK2bLHfj0Skjg77rSS9/rr9OAENyAK3Tgow+AgpwhkA35s+XWptPRHGThaJ2I/PmOHosAB/iBuSHAhHhh4hRTgD4GsNDdLmzfGDWVQkIm3aZF8PIMNihSQnwpGbwTABwhkAX6urk4LB1K4NBqW1a7M7HsCX3DwpwK1gmADhDICvtbRIeSn+JMzLk5qbszocwL+6hyQnw5GBR0gRzgD4WmGh1Bmj52UsnZ3SsGFZHQ7gX10hSc6HI7eCYRz0OQPQxY89vmpqpIULU7s2ErGvB5AlpXOk/KHS2JnO/rtdfc/mu141k6SAZVmpHCzlOW1tbQqHw2ptbVVBQYHbwwFc1dRk71jcvNleV5WXZ1eJIhG799fKlVJxsdujzJ7qaruNRqJNAcGgVFkpbdzo3LgAOOj4R9KeFXYwzB+SlX8i1exBOCOcweeiPb7itZIIBqVw2A4vXg1ofA0AOCHV7MGaM8Dn6PFlB64tW+zKmGSHsVDoxC7OykqCGQDnsOYM8LFoj69kuvf48uoatOJie8qyocFul9HcbC/+r6mRJkxwe3QA/IRwBvhYtMdXsgas0okeX14NZ1FlZd5/jgDMxrQm4GP0+AIA8xDOAB+jxxcAmIdwBvhYTU1qU5oSPb4AwCmEM8DHysqkqqrkZ0sGg3YvMBbGA0D2Ec4An1u1yu7hFS+gRXt8rVzp6LAAwLcIZ4DP0eMLAMxCKw0A9PgCkFua35D2LJeUyiFHAWnsLdKw8myPKmMIZwC60OMLQE54f4O0e7EUCCrxJGCnZEWkgnE5Fc6Y1gQAALmlqFYKFdrBy+pIcIvY1xXVuj3itBDOAABAbgmdKl14t6RAkgsD0kX3SKGhTowqYwhnAAAg95TMlkLhxNeEwtIFs50ZTwYRzgAAQO5JWj3LzaqZRDgDAAC5KlH1LEerZhLhDAAA5Kq41bPcrZpJtNIA0tbQINXVSS0t9sHhNTW0nwAA15TMlhofkzpaTtyXw1UziXAGpKypSZo+Xdq82e6en5cndXZKCxfa506uXEkXfQBwXLR6tvNe2U1pc7tqJjGtCaSkqUmaNMk+xkiSIhGpo8N+K0mvv24/3tTk3hgBwLe6rz3L8aqZRDgDUjJ9utTaeiKMnSwSsR+fMcPRYQEApG5rz5TzVTOJaU0gqYYGeyozmUhE2rTJvp41aADgsNI5Uv5QaexMt0fSb4Qz9JvXF8jX1dlrzOJVzboLBu2Dw730/AEgJ+QPkUpvdXsUGUE4Q5/5ZYF8S4v93FIJZ3l5UnNz1ocEAPAw1pyhT/y0QL6w0A6dqejslIYNy+pwAAAeRzhDn/hpgXxNTWpVM8m+rqYmu+MBAHgb4Qxpiy6QTxZYui+Qz2VlZVJVlT11m0gwaE/nTpjgzLgAAN5EOEPaogvkUxFdIJ/rVq2SwuH4zzsYtB9fudLRYQEAPIhwhrRFF8inwisL5IuL7fV1lZX2+8GgFAqdCGuVlfbjXtgAAQBwF7s1kTa/LpAvLpY2brSnadeutUPnsGH2GjOmMgEAmRKwLMtyexBuaGtrUzgcVmtrqwoKCtweTk5paJDKy9O7nvACAPC7VLMH05pIGwvkAQDIHsIZ+oQF8gAAZAfhDH3CAnkAALKDDQHoMxbIAwCQeYQz9FtZGQd9AwCQKUxrAgAAGIRwBgAAYBDCGQAAgEEIZwAAAAYhnAEAABiEcAYAAGAQwhkAAIBBCGcAAAAGIZwBAAAYhHAGAABgEMIZAACAQQhnAAAABiGcAQAAGIRwBgAAYBDCGQAAgEEIZwAAAAYhnAEAABgk3+0BAACAOJrfkPYsl2SlcHFAGnuLNKw826NClhHOAABIhRtB6f0N0u7FUiCoxJNdnZIVkQrGEc48gHAGAEAq3AhKRbXSzvukjhZJkcTXhgrt65HzWHMGAEAqimrtAGRFJKsjwS2SuaAUOlW68G5JgSQXBqSL7pFCQ/v/b8J1hDMAAFLhVlAqmS2FwknGFpYumJ2Zfw+uY1oTMFxDg1RXJ7W0SIWFUk2NVFbm9qgAnyqZLTU+9vE0YxyZDkrRULjzXsVe70bVzGuonAGGamqSqqul8nLp4YelJUvst+Xl0hVX2I8DcFjS6lmWglKi6hlVM88hnAEGamqSJk2Stmyx349EpI4O+60kvf66/TgBDXCBG0EpbiikauZFhDPAQNOnS62tJ8LYySIR+/EZMxwdFgDJvaAUKxRSNfMkwhlgmIYGafPm+MEsKhKRNm2yrwfgMDeCUq9QSNXMqwhngGHq6qRgMLVrg0Fp7drsjgdADG4Fpe6hkKqZZxHOAMO0tEh5Kf6fmZcnNTdndTgA4nEjKHWFQlE18zDCGWCYwkKpszO1azs7pWHDsjocAPG4FZRK50gVi6WS25z59+A4whlgmJqa5OvNoiIR+3oALnEjKOUPkUpvtd/CkwhngGHKyqSqquTrzoJBuw/ahAnOjAtADAQlZAHhDDDQqlVSOBw/oAWD9uMrVzo6LACAAwhngIGKi+0GtJWV9vvBoBQKnQhrlZX248XF7o0RAJAdnK0JGKq4WNq40e5jtnatvStz2DB7jRlTmfC95jekPcsV+6zJkwWksbdIw8qzPSogIwhngOHKyjjoHOjl/Q3S7sVSIKjEk0CdkhWRCsYRzpAzmNYEAOSeolopVGgHL6sjwS1iX1dU6/aIgZQRzgAAuSfu+ZYn44gj5B5jwtnSpUtVVFSkQYMGqaKiQhs2bEh4/dGjR7VgwQKNGTNGAwcO1NixY7VixQqHRgsAcF2s8y1PxhFHyEFGrDlbs2aN7rjjDi1dulTV1dV6+umnNWXKFDU2Nmr06NExP+aGG27Q3/72Nz3zzDMqLi7WwYMHdfz4cYdHDgBwTbR6tvNexd4YQNUMuSlgWVYqW12yatKkSbr00ku1bNmyrvvGjx+vqVOnatGiRb2uf+mll3TjjTfqzTff1GmnnZbSv3H06FEdPXq06/22tjaNGjVKra2tKigo6P+TAAA4r6NdWjda6mjp/VioUJq6j3AGY7S1tSkcDifNHq5Pax47dkzbtm3T5MmTe9w/efJkbd68OebH/OIXv9DEiRP1ve99T+ecc45KSkp055136u9//3vcf2fRokUKh8Ndt1GjRmX0eQAAXBB37RlVM+Qu18PZoUOHFIlENHz48B73Dx8+XAcOHIj5MW+++aY2btyoP/7xj1q7dq0ef/xxvfDCC5o9O/66gvnz56u1tbXrtm/fvow+DwCAS2KtPWOtGXKYEWvOJCkQ6PlXj2VZve6L6uzsVCAQ0OrVqxUO2/9D/uhHP9JXvvIVLVmyRIMHD+71MQMHDtTAgQMzP3AAgLt6rT2jaobc5nrl7IwzzlAwGOxVJTt48GCvalrUiBEjdM4553QFM8leo2ZZlt59992sjhcAYKDu1TOqZshxroezAQMGqKKiQvX19T3ur6+vV1VVVcyPqa6u1nvvvacjR4503bd7927l5eXp3HPPzep4AQAG6lp7JqpmyHmuhzNJmjdvnpYvX64VK1Zo165dmjt3rvbu3atZs2ZJsteL1dae6O5800036fTTT9fNN9+sxsZGvfrqq7rrrrs0c+bMmFOaAAAfKJ0jVSyWSm5zeyRAvxix5mzatGk6fPiwHnroIe3fv18TJkzQ+vXrNWbMGEnS/v37tXfv3q7rhw4dqvr6et12222aOHGiTj/9dN1www165JFH3HoKAAC35Q+RSm91exRAvxnR58wNqfYaAQAAyISc6XMGAACAEwhnAAAABiGcAQAAGMSIDQEAzNfQINXVSS0tUmGhVFMjlZW5PSoA8B7CGYCEmpqk6dOlzZulYFDKy5M6O6WFC6XqamnlSqm42O1RAoB3EM4AF5lejWpqkiZNklpb7fcjEfsW9frr9uNbthDQACBTaKVBKw24IF41KhIxqxpVXW0Hr+6B7GTBoFRZKW3c6Ny4ACAXpZo9qJz5mOlVG6/KlWpUQ4MdHpOJRKRNm+zr+f4BgP4jnPkQa4jcNX26HcziVaMiEfvxGTPcrUbV1dnfH4mqZlHBoLR2LeEMADKBcOYzuVK18apcqka1tNjBPZVwlpcnNTdnfUhA3zS/Ie1ZLimVVTwBaewt0rDybI8KiItw5jO5UrXxqlyqRhUW2hXVVHR2SsOGZXU4QN+9v0HavVgKBJW4vWenZEWkgnGEM7iKcOYjuVS18apcqkbV1NhT3amIROzrgYzJZLWrqFbaeZ/U0SIpyf98oUL7esBFhDMfyaWqjVflUjWqrEyqqkp9t+aECc6NDT6QyWpX6FTpwrulnfcqcdgLSBfdI4WG9n3cQAZwfJOPRKs2qXC7auNVNTWphWPJjGrUqlVSOGwHsFiCQfvxlSsdHRb8oKjWrmJZEcnqSHCLpFbtKpkthcKJrwmFpQtmZ+oZAH1GOPORXKraeFW0GhUv7EQFg/bOWberUcXFduWssvLEuEKhE+OvrGTzCLIkWu1SIMmFKVa7kn4+qmYwB01ofdSEtqFBKk9jjWtDg/vhwIu675iNVUWLVqNMCz0NDfZUd3OzHdxravj+QJZ1tEvrRn+8ViyOUKE0dV9qoSrR50vn8wB9RBNa9MIaIjNEq1EzZtgbL04+IaCy0sxec2VlrEGEw5KuFUuz2hX381E1g1monPmociblbtXGq6hGAUlkutoV6/NRNYNDqJwhplyt2ngV1SggiUxXu3p9PqpmMA+VM59VzrqjagMgJ2S62tX981E1g4OonCEpqjYAckKmq11dn28+VTMYiVYaAADzde9Tlol+ZKVzpIrFUslt/R8bkGGEMwCA+br6lCkz1a78IVLprfZbwDBMawIAckPpHCl/qDR2ptsjAbKKcAYAyA3RahfgcUxrAgAAGITKGYzU0CDV1dmHtRcW2m0+2FkKAPADwhmM0tQkTZ8ubd7cs0HuwoX2QeA0yAUAeB3TmjBG9GipLVvs9yMRqaPjxDFTr79uP97U5N4YAQDINsIZjDF9evwzPyX7/tZW++gpAAC8imlNGKGhwZ7KTCYSsc8EbWhgDRoAAzS/Ie1Zrp7nfsYTkMbeIg0rz/aokOMIZzBCXZ29xixe1ay7YNA+E5RwBsB172+Qdi+WAkElnozqlKyIVDCOcIakCGcwQkuLvfg/lXCWl2cf1u4GdpEC6KGoVtp538eHsif5ARYqtK/PJVQGXUE4gxEKC+1dmano7JSGDcvqcHphFymAmHodyh5PPw9rdwuVQVf0eUPAP/7xDzU0NOijjz7q9dimTZv6NSj4T01NalUzyb6upia74+mOXaQAEup+KHs8mTis3Q1FtXbFz4pIVkeCWyQ3K4OG6lM4e+211zRq1Ch9+tOf1plnnqlHH320x+NTpkzJyODgH2VlUlWVXZVKJBi0K1UTJjgzLoldpACS6DqUPRDnghytmkkpPLeoHH6OBupTOPu3f/s3/fCHP9Thw4e1bds21dXVaebMmer8eF7KslKZmwZ6WrVKCofjB7Rg0H585UrnxhTdRZqsqtd9FykAH0pUPcvVqlmUlyuDhupTOGtsbFRtrV26HDdunF555RUdPHhQX/nKV3Ts2LGMDhD+UVxsTx1WVtrvB4NSKHQirFVW2o87ubYruos0FdFdpAB8KG6FyQMVJS9XBg3Vp3BWUFCgv/71r13vDx48WOvWrdOgQYP0uc99rquCBqSruFjauFF64w3p/vul2bPttw0N9v1OL7qP7iJNhZu7SAEYIFaFySsVJS9XBg3Up3B29dVX69lnn+1xX35+vlavXq2xY8fq73//e0YGB/8qK7ND2f/7f/ZbJ9eYdWf6LlIABulVYfJQRcnLlUEDBaw+LBA7duyYjh8/riFDhsR8fO/evRo9enS/B5dNbW1tCofDam1tVUFBgdvDgaEaGqTyNHaFNzS4FyQBGKCjXVo32u57FiqUpu7zTnDp/tyivPYcsyzV7JFW5ezf//3fJUkDBgyIG8wkGR/MgFSZvIsUgIG6KkzyXkXJy5VBw6QVzh577DHNnTs37uN79+7t94AA05i4ixSAwUrnSBWLpZLb3B5J5nVfe8Zas6xJ64SAuro63XDDDTpy5Ih+/OMfKxCw03N7e7u+853v6Mknn4zZlBbIZdFdpDNm2O0yup8QEInYu0g5IQBAl/whUumtqV2ba8cjdZ2IMJ+qWRalFc6+8IUvaP369friF7+oDz/8UM8++6xWrFihhQsX6oMPPtDMmTOzNU7AVdFdpA0NdruM5mZ78X9NDVOZAPohF49HKp0j5Q+VxvI7P1v6tCFg69atuuqqq5SXl6cjR47o+uuv16OPPqrS0tJsjDEr2BAAAHBdrEX28bD4PudlZUOAJG3fvl333nuvPvzwQ7W3t6u6ulovvPBCTgUzAACMwPFIiCGtcHbTTTfpk5/8pBobG7VixQpt2LBBf/zjHzV16lQdPXo0W2MEAMC7OB4JJ0krnP3P//yPHnjgAf3lL3/RjBkzVF1drV//+tf6/e9/r8997nM6cuRItsYJAIA3cTwSTpLWmrMDBw7o7LPP7nX/n//8Z1199dUaMWKEfve732V0gNnCmjMAgDESrT1jrZlnZGXNWaxgJkmlpaXasGGDmjlYEACA9HE8Errp09masZx33nnasGFDpj4dAAD+4uWD05GWjIUzKX5lDQAAJMHxSPhYRsMZAADoB45HgghnAACYw8sHpyNlaR3fBGc1NEh1dVJLi1RYaB8VVFbm9qgAAFnF8Ui+16fjm7zA5FYaTU3S9OnS5s29D9muruaQbQAAclHWjm9CdjU1SZMmSVu22O9HIlJHh/1Wkl5/3X68qcm9MQIAgOxhWtMw06dLra0nwtjJIhH78RkzpI0bHR0aAMA0zW9Ie5ZLSmUSLCCNvUUaVp7tUaGfCGcGaWiwpzKTiUSkTZvs61mDBgA+9v4GafdiKRBU4smwTsmKSAXjCGc5gGlNg9TV2WvMUhEMSmvXZnc8AADDFdXaxztZEcnqSHCL2NcV1bo9YqSAyplBWlrsxf/xpjS7y8uTOC0LABxk4hRitPXGznuTjIuGtrmEcGaQwkJ7V2YqOjulYcOyOhzj0WoEgKNMnUIsmS01Phb70PSovJB05E1p620JPhFr0kxBKw2DWmk0NEjlafw/0dAgTZiQvfGYilYjAFzR0S6tG504BEWFCqWp+5yrVP3p0STVs4AUSFSP+ThQTlwilXwrCwOERCuNnFRWJlVVJV93FgzaIcSvwYxWIwBc0evsy3hcmEKMdWh6DxZr0nII4cwwq1ZJ4XD8gBYM2o+vXOnosIyRTqsRAMi4pCFI7pyJGTc4BqQRn4tx/8lYk2YSwplhiovtqlBlpf1+MCiFQifCWmWl/bgfp+2irUaSbZiIthr5+telBx+0Pw4AMiJp9czFkBMrOIbCUuUKMwMl4iKcGai42G4w+8Yb0v33S7Nn228bGuz7/RjMpPRajUjSc89JDz9sr+O74gqmOgFkSKLqmZshp1dw/DgoDh5hbqBETOzWNFhZGbsPu0un1YjUc+drdC2aX6uOADIobvsKA0JO952b3YNioh2dVM2MQ+UMOSOdViMnYy0agIyKN4Xodsjpqp6pZ1BMtCbN7UCJXghnyBk1NalXzWLpfuwVAPRLvClEE0JO6RypYrFUclJPM1MDJXohnCFnpNpqJBGOvQKQMd3DjkkhJ3+IVHqr/bY7kwMleiCcIackazWSDMdeAciYeFOIJjM1UKIHwhlySqxWI3lpfBdz7BWAjIo3hWiqXAyUPsRuTeScaKuRhgZ7ivIvf5F++tPUPjYSsdeuAUBGRKcQc0npHCl/qDR2ptsjQRyEM+Ss7q1G3nzTrqgl2jAQDNoVNz8eewUAXXIxUPoM05rwBI69AgB4BeEMnsCxVwAAr2BaE55x8lq05mZ78X9NDVOZAIDcQTiD53DsFQAglzGtCQAAYBAqZwAAoLfmN6Q9y9XzcPd4AtLYW6Rh5dkelS8QzgAAQG/vb5B2L5YCQSWeaOuUrIhUMI5wliGEMwAAnJYLVamiWmnnfVJHi6QETSQlKVRoX4+MIJwBAOC0XKhKRY962nmvEodIDlDPNDYEAADgtKJau9pkRSSrI8Et4m5VqvtB6fFwgHrGEc4AAHBa1wHkgSQXulyVSjpOqmbZQDgDAMANuVKVSjROE8bnQaw5AwDADUnXdPWxKpXpzQZxx0nVLFuMCWdLly7V97//fe3fv18XXXSRHn/8cV155ZVJP27Tpk361Kc+pQkTJmjHjh3ZHygAAJlSMlv603el4+29H8sLSUfelLbe1u3OFMJUNjYblMyWGh/7eOfmx6iaZY0R4WzNmjW64447tHTpUlVXV+vpp5/WlClT1NjYqNGjR8f9uNbWVtXW1uqf/umf9Le//c3BEQMAcp4J7SxCp0rDr5L++ovej3Uel/Y82/2O1MJUNlpg9KqeUTXLpoBlWal8V2bVpEmTdOmll2rZsmVd940fP15Tp07VokWL4n7cjTfeqAsuuEDBYFDr1q1Lq3LW1tamcDis1tZWFRQU9Gf4AIBctHuJtPXW1CtME5dIJd/K/Dg+ek9ad05q14YKpan7koeiPz2aWguMSxZ9vOA/BR3t0rrRduhLdRzoIdXs4fqGgGPHjmnbtm2aPHlyj/snT56szZs3x/24Z599Vnv27NEDDzyQ0r9z9OhRtbW19bgBAHzMlHYWQ0ZKZ1+TwoVpVKuysdmga+emqJplmevh7NChQ4pEIho+fHiP+4cPH64DBw7E/Ji//OUvuueee7R69Wrl56c2M7to0SKFw+Gu26hRo/o9dgBADjOpnUXliuTXpBOmstUCo3SOVLFYKrkt+bXoM9fDWVQg0PMbyLKsXvdJUiQS0U033aQHH3xQJSUlKX/++fPnq7W1teu2b9++fo8ZAJDjTGlnkbR61ocwlY0WGPlDpNJb7bfIGtfD2RlnnKFgMNirSnbw4MFe1TRJam9v19atW3XrrbcqPz9f+fn5euihh7Rz507l5+fr17/+dcx/Z+DAgSooKOhxAwD4nElNVi9fJeUNjv1YX8JU3OfGYn7TuR7OBgwYoIqKCtXX1/e4v76+XlVVVb2uLygoUENDg3bs2NF1mzVrlkpLS7Vjxw5NmjTJqaEDALzAlCarg4dLZfcro2Eq1nOjBYbxXA9nkjRv3jwtX75cK1as0K5duzR37lzt3btXs2bNkmRPSdbW2gsx8/LyNGHChB63s846S4MGDdKECRN0yimnuPlUAAC5xqQKU6bDVK/nRtUsFxgRzqZNm6bHH39cDz30kC655BK9+uqrWr9+vcaMGSNJ2r9/v/bu3evyKAEAnmVKhSkbYar7c6NqlhOM6HPmBvqcAQB66NEbLM0eYJmUjX5if3pU2jlfuuRRd54TJOVQnzMAAIxgSoUpG/3EaIGRU4w4vgkAANd1HVE03/11WaVzpPyh0tiZmfl80RYYyAmEMwAAojIdivqKMOVrhDMAAKIIRTAAa84AAAAMQjgDAAAwCOEMAADAIIQzAAAAgxDOAAAADEI4AwAAMAjhDAAAwCCEMwAAAIMQzgAAAAxCOAMAADAIxzcBAJCq5jekPcslWSlcHJDG3iINK8/2qOAxhDMAAFL1/gZp92IpEFTiyadOyYpIBeMIZ0gb05oAAKSqqFYKFdrBy+pIcIvY1xXVuj1i5CDCGQAAqQqdKl14t6RAkgsD0kX3SKGhTowKHkM4AwAgHSWzpVA48TWhsHTBbGfGA89hzRnS1tAg1dVJLS1SYaFUUyOVlbk9KgBwSLR6tvNexd4YQNUM/ROwLCuVLSee09bWpnA4rNbWVhUUFLg9nJzQ1CRNny5t3iwFg1JentTZKUUiUnW1tHKlVFzs9igBwAEd7dK60VJHS+/HQoXS1H2EM/SSavZgWhMpaWqSJk2Stmyx349EpI4O+60kvf66/XhTk3tjBADHxF17RtUM/Uc4Q0qmT5daW0+EsZNFIvbjM2Y4OiwAcE+stWesNUMGEM6QVEODPZUZL5hFRSLSpk329QDgeb2qZ1TNkBmEMyRVV2evMUtFMCitXZvd8QCAMbpXz6iaIUMIZ0iqpcVe/J+KvDypuTmrwwEAc3RVz0TVDBlDKw0kVVho78pMRWenNGxYVocDAGYpnSPlD5XGznR7JPAIKmdIqqYm+XqzqEjEvh4AfCN/iFR6q/0WyAAqZ0iqrEyqqrLbaCQKacGgVFkpTZjQ/3+TRrcAAL+iCS1NaFMS7XMWr51GMCiFw3aA608jWhrdAgC8iia0yKjiYjt4VVba7weDUih0YhdnZWVmghmNbgEAfse0JlJWXCxt3GhPOa5da+/KHDbMnnLMxFRmOo1uN27s/78HQFLzG9Ke5Yp9RuTJAtLYW6Rh5dkeFeBrhDOkraws8+u/oo1uk+ne6JY1aEAGvL9B2r1YCgSVeDKlU7IiUsE4whmQZUxrwgg0ugVcUlRrH9RtRSSrI8EtYl9XVOv2iAHPI5zBCDS6BVwS9wDvk3E0EeAUwhmMQKNbwEWxDvA+GUcTAY4hnMEINLoFXJS0ekbVDHAS4QxGiDa6TbbuLBi0+51lYncogG4SVc+omgGOIpzBGKtW2Y1s4wW0aKPblSsdHRbgD3GrZ1TNAKcRzmAMJxrdAkggVvWMqhngOPqcwSjZbnQLIIFo9WznvbKb0lI1A9zA2Zo+PVuTg8UBxNTRLq0bLXW02H3Npu4jnAEZkmr2oHLmM/EOFl+4kIPFAahb9Ww+VTPAJYQzH4keLN7aar8fifRsXxE9WJx1XYDPlc6R8odKY2e6PRLAlwhnPsLB4vAcDu3OjvwhUumtbo8C8C3CmU9wsDg8iUO7nUUYBhxBOPOJ6MHiqXThjx4sTjiD8YpqpZ332YvXleSbm0O7+48wDDiCPmc+wcHi8CQO7XZWUa0dcq2IZHUkuEUIw0A/EM58goPF4Vkc2u0cwjDgCMKZT3CwODyLQ7udRRgGso5w5hMcLA5P49Bu5xCGgawjnPkIB4vDszi021mEYSCrCGc+wsHi8DQO7XYOYRjIKlpp+AwHi8OzOLTbWSWzpcbHPm5j8jHCMJARhDOfKiujjxk8qHtgIChkF2EYyBqmNQF4R9d0mwgKTug+lUwYBjKGcAbAW0rnSBWLpZLb3B6J9xGGgaxgWhOAt3Bot7NK50j5Q6WxM90eCeAZhDMAQN8RhoGMY1oTAADAIFTOAMBrmt+Q9iyXvYsymYA09hZpWHm2RwUgRYQzAPCa9zdIuxdLgaAST5B0SlZEKhhHOAMMwrQmAHhNUa0UKrSDl9WR4BaxryuqdXvEALohnAGA1yQ9nDyKxrGAiQhnAOBFiQ4nj6JxLGAkwhkAeFHS6hlVM8BUhDMA8KpE1TOqZoCxCGcA4FVxq2dUzQCTEc4AwMtiVc+omgFGI5wBgJf1qp5RNQNMRzgDAK/rXj2jagYYj3AGAF7XVT0TVTMgB3B8EwD4QekcKX+oNHam2yPJHlPOFDVlHMhZhDMA8IP8IVLprW6PIrtMOVPUlHEgZzGtCQDwBlPOFDVlHMhZhDMAgDeYcqaoKeNAziKcAQC8w5QzRU0ZB3IS4QwA4B2mnClqyjiQkwhnAABvMeVMUVPGgZxDOAMAeIspZ4qaMg7kHMIZAMB7TDlT1JRxIKcQzgAA3mPKmaKmjAM5hXAGAPAmU84UNWUcyBmcEAAA8KZo1WrnfHerVaaMIxGOnDIK4QwA4F2mnClqyjji4cgpowQsy0olJntOW1ubwuGwWltbVVBQ4PZwAABwT0e7tG601NGS/NpQoTR1n5kVQMOlmj1YcwYAgN9x5JRRCGcAAIAjpwxCOAMAABw5ZRBjNgQsXbpU3//+97V//35ddNFFevzxx3XllVfGvLaurk7Lli3Tjh07dPToUV100UVauHChrrnmGodHDSBnsBsNSK5kttT4WOy1Z1TNHGNEOFuzZo3uuOMOLV26VNXV1Xr66ac1ZcoUNTY2avTo0b2uf/XVV/XZz35W3/3ud1VYWKhnn31W1113nbZs2aJPfOITLjwDAMZjNxqQXFfbj3vV8w8ZqmZOMmK35qRJk3TppZdq2bJlXfeNHz9eU6dO1aJFi1L6HBdddJGmTZum+++/P6Xr2a0J+Ay70YDUxPp/hf8nMiJndmseO3ZM27Zt0+TJk3vcP3nyZG3evDmlz9HZ2an29naddtppca85evSo2traetwA+Ai70YDUcOSU61wPZ4cOHVIkEtHw4cN73D98+HAdOHAgpc/xwx/+UB9++KFuuOGGuNcsWrRI4XC46zZq1Kh+jRtADmI3GpAajpxylevhLCoQ6PnXrGVZve6L5fnnn9fChQu1Zs0anXXWWXGvmz9/vlpbW7tu+/bt6/eYAeQYdqMBqen6f0X8P+EC1zcEnHHGGQoGg72qZAcPHuxVTTvZmjVr9I1vfEM///nPdfXVVye8duDAgRo4cGC/xwsgx7EbDUiN6UdOeZjr4WzAgAGqqKhQfX29vvSlL3XdX19fry9+8YtxP+7555/XzJkz9fzzz+vzn/+8E0MFYKp022QU1do7N9mNBsSXP0QqvdXtUfiS6+FMkubNm6evf/3rmjhxoi6//HL9+Mc/1t69ezVr1ixJ9pTkX//6V/3nf/6nJDuY1dbW6oknnlBlZWVX1W3w4MEKh5OsJwHgPem2yfjED+0qWY/daFTNAJjBiDVn06ZN0+OPP66HHnpIl1xyiV599VWtX79eY8aMkSTt379fe/fu7br+6aef1vHjxzV79myNGDGi63b77be79RQAuKmo1t7qb0UkqyPBLWJfV/z/sRsNgLGM6HPmBvqcAR7zp0djNM48WUC6ZJEdzLr3cqKHEwAH5EyfMwDIiHTbZLAbDYChCGcAvKEvbTJK50gVi6WS25wYIQCkhHAGwDsSVc9iLfiP7kbLH5L9sQFAighnALwjbvWMBf8AcgfhDIC3xKqe0SYDQA4hnAHwFg5tBpDjCGcAvIdDmwHkMCNOCADwsXSPIRp7izSsPNujyj3R6tnO+VTNAOQcwhlgknSPISoYRziLx41DmwnXfcPXDeiBcAaYpKhW2nnfx2c+RhJfGyq0r0dsbhzaTLjuG75uQA+sOQNMkrSRahSL3I2U7hmfhGsbXzegB8IZYJp0jyGCOQjXfcPXDeiBcAaYpi/HEMEchOu+4esGdCGcASZK9xgimINw3Td83YAuhDPARBxDlNsI133D1w2QRDgDzMUxRLmLcN03fN0ASYQzwFwcQ5TbCNd9w9cNIJwBRuMYotxFuO4bvm4A4QwwWtcvKvELKhcRrvuGrxt8jnAGmK50jlSxWCq5ze2RIF2E677h6waf4/gmwHRuHEOEzHHjjE8v4OsGHyOcAUA2Ea77hq8bfIxpTQAAAIMQzgAAAAxCOAMAADAI4QwAAMAghDMAAACDEM4AAAAMQjgDAAAwCH3OAADe0fyGtGe5JCuFiwPS2FukYeXZHhWQFsIZAMA73t8g7V4sBYJKPDnUKVkRqWAc4QzGIZwByByqFnBbUa208z6po0VSJPG1oUL7esAwhDMAmUPVov8IuP0TPTR9571K/DUMcKg6jEU4A5A5VC36j4DbfyWzpcbHPv4+jCMUli6Y7diQgHSwWxNA5kSrFgokuZCqRVxFtXZwtSKS1ZHgFiHgxpP0+5DvP5iNcAYgs0pm21WJRKhaxEfAzYxE34d8/8FwhDMAmUXVov8IuP0X9/uQ7z+YjzVnADIv0ZofQkVySRe1EzBSEuv70JTvPzZ+IAHCGYDMixsuCBUpI+D2X6/vQ4O+/9j4gQSY1gSQHbGm5ggVqWNaLjO6fx+a9P3Hxg8kQDgDkB29wgWhIm0E3P7r+j6UWd9/bPxAAkxrAsie7lNzhIr0mTwtl0tK50j5Q6WxM90eSU9e7cfGerp+I5wByJ6ucDGfUNFXBNz+yx8ild7q9ih68+rGD9bT9RvTmgCyq3SOVLFYKrnN7ZHkJlOn5ZAZXuzHxnq6fqNyBiC7TK1a5BJTp+XQf4mqZ+EL7apzUoZNDXK+ab8FLMtKZVLYc9ra2hQOh9Xa2qqCggK3hwMA8KuOdmnd6J5rz/IGS51/T31qcOISqeRbWR5oGmI9p5OFCqWp+3wVzlLNHkxrAgDgprg7mwtzd2qQk0L6hXAGAIDbTu7HNm5e7rfa8OJ6OocQzgAAcFusjR+5fsYqjZT7jHAGAIAJTt7Z7IWpQRop9wnhDAAAE0R3NucPOXFfrk8NclJInxDOAAAwlRemBk0939RghDMAAEyW61ODNFJOG+EMAACTeWFqkJNC0kI4AwDAdLk+NRhrPR3iIpwBAGA6pgZ9hbM1AQDIBZyx6huEMwAAckF0ahCex7QmAACAQQhnAAAABiGcAQAAGIRwBgAAYBDCGQAAgEEIZwAAAAYhnAEAABiEcAYAAGAQmtACANAfzW9Ie5ZLslK4OCCNvUUaVp7tUSGHEc4AAOiP9zdIuxdLgaAST0h1SlZEKhhHOENCTGsCANAfRbVSqNAOXlZHglvEvq6o1u0Rw3CEMwAA+iN0qnTh3ZICSS4MSBfdI4WGOjEq5DCmNQEA6K+S2VLjY1JHS/xrQmHpgtmODanPWEPnOsIZAAD9Fa2e7bxXsUNNDlXNWEPnOqY1AQDIhJLZdnUsllypmkmsoTMA4QwAgEyIu/Ysh6pmEmvoDEA4AwAgU2JVz3KpahaVqAoYlYvPK0cQzgAAyJReVaccrS4lrZ7l6PPKEYQzAAAyqXvVKZerS15ZQ5eDCGcAAGRSV9VJuV1d8soauhxEOAMAINNK50gVi6WS29weSf94ZQ1djqHPGQAAmZY/RCq91e1R9F+v/m0uV8180iCXcAbAe3zyAxxwRPfTD9yumvmkQS7hDID3+OQHOOCIrurZfPfXmhXVSjvv+/iYrEjia3O4QS5rzgB4Dx3OgcwyZQ2dTxrkEs4AeI9PfoADjomuocsf4vZIfNEgl3AGwJt88AMc8CUfNMglnAHwJh/8AAd8y+MNcglnALzL4z/AAd/yeINcwhkA7/L4D3DA1zzcIJdwBsDbPPwDHPA1rxwyHwPhDIC3efgHOOB7Xjlk/iTGhLOlS5eqqKhIgwYNUkVFhTZs2JDw+ldeeUUVFRUaNGiQzj//fD311FMOjRRAzvHoD3DA97xyyPxJjAhna9as0R133KEFCxZo+/btuvLKKzVlyhTt3bs35vVvvfWWrr32Wl155ZXavn277r33Xs2ZM0cvvviiwyMHkBM8+gMcgMxpkJtBAcuyUjl8LqsmTZqkSy+9VMuWLeu6b/z48Zo6daoWLVrU6/q7775bv/jFL7Rr166u+2bNmqWdO3fqtddei/lvHD16VEePHu16v62tTaNGjVJra6sKCgoy+GwAGOn4R9KeFdLYmWY00gTgO21tbQqHw0mzh+uVs2PHjmnbtm2aPHlyj/snT56szZs3x/yY1157rdf111xzjbZu3aqOjo6YH7No0SKFw+Gu26hRozLzBADkBpM6nANAAq6Hs0OHDikSiWj48OE97h8+fLgOHDgQ82MOHDgQ8/rjx4/r0KFDMT9m/vz5am1t7brt27cvM08AAAAgg/LdHkBUINCzD5FlWb3uS3Z9rPujBg4cqIEDB/ZzlAAAANnleuXsjDPOUDAY7FUlO3jwYK/qWNTZZ58d8/r8/HydfvrpWRsrAABAtrkezgYMGKCKigrV19f3uL++vl5VVVUxP+byyy/vdf3LL7+siRMnKhQKZW2sAAAA2eZ6OJOkefPmafny5VqxYoV27dqluXPnau/evZo1a5Yke71YbW1t1/WzZs3SO++8o3nz5mnXrl1asWKFnnnmGd15551uPQUAAICMMGLN2bRp03T48GE99NBD2r9/vyZMmKD169drzJgxkqT9+/f36HlWVFSk9evXa+7cuVqyZIlGjhypJ598Ul/+8pfdegoAAAAZYUSfMzek2msEAAAgE3KmzxkAAABOIJwBAAAYhHAGAABgEMIZAACAQQhnAAAABiGcAQAAGIRwBgAAYBDCGQAAgEGMOCHADdHeu21tbS6PBAAA+EE0cyTr/+/bcNbe3i5JGjVqlMsjAQAAftLe3q5wOBz3cd8e39TZ2an33ntPp556qgKBQEY+Z1tbm0aNGqV9+/ZxJJQBeD3MwuthHl4Ts/B6mCUbr4dlWWpvb9fIkSOVlxd/ZZlvK2d5eXk699xzs/K5CwoK+B/LILweZuH1MA+viVl4PcyS6dcjUcUsig0BAAAABiGcAQAAGIRwlkEDBw7UAw88oIEDB7o9FIjXwzS8HubhNTELr4dZ3Hw9fLshAAAAwERUzgAAAAxCOAMAADAI4QwAAMAghDMAAACDEM4AAAAMQjhL09KlS1VUVKRBgwapoqJCGzZsSHj9K6+8ooqKCg0aNEjnn3++nnrqKYdG6g/pvB51dXX67Gc/qzPPPFMFBQW6/PLL9b//+78Ojtb70v3/I2rTpk3Kz8/XJZdckt0B+ky6r8fRo0e1YMECjRkzRgMHDtTYsWO1YsUKh0brD+m+JqtXr9bFF1+sIUOGaMSIEbr55pt1+PBhh0brXa+++qquu+46jRw5UoFAQOvWrUv6MY7+PreQsp/97GdWKBSyfvKTn1iNjY3W7bffbp1yyinWO++8E/P6N9980xoyZIh1++23W42NjdZPfvITKxQKWS+88ILDI/emdF+P22+/3Xrssces3/3ud9bu3but+fPnW6FQyPrDH/7g8Mi9Kd3XI6qlpcU6//zzrcmTJ1sXX3yxM4P1gb68Htdff701adIkq76+3nrrrbesLVu2WJs2bXJw1N6W7muyYcMGKy8vz3riiSesN99809qwYYN10UUXWVOnTnV45N6zfv16a8GCBdaLL75oSbLWrl2b8Hqnf58TztJw2WWXWbNmzepx37hx46x77rkn5vXf/va3rXHjxvW471//9V+tysrKrI3RT9J9PWK58MILrQcffDDTQ/Olvr4e06ZNs+677z7rgQceIJxlULqvx69+9SsrHA5bhw8fdmJ4vpTua/L973/fOv/883vc9+STT1rnnntu1sboR6mEM6d/nzOtmaJjx45p27Ztmjx5co/7J0+erM2bN8f8mNdee63X9ddcc422bt2qjo6OrI3VD/ryepyss7NT7e3tOu2007IxRF/p6+vx7LPPas+ePXrggQeyPURf6cvr8Ytf/EITJ07U9773PZ1zzjkqKSnRnXfeqb///e9ODNnz+vKaVFVV6d1339X69etlWZb+9re/6YUXXtDnP/95J4aMbpz+fZ6f8c/oUYcOHVIkEtHw4cN73D98+HAdOHAg5sccOHAg5vXHjx/XoUOHNGLEiKyN1+v68nqc7Ic//KE+/PBD3XDDDdkYoq/05fX4y1/+onvuuUcbNmxQfj4/ijKpL6/Hm2++qY0bN2rQoEFau3atDh06pG9961v64IMPWHeWAX15TaqqqrR69WpNmzZN//jHP3T8+HFdf/31Wrx4sRNDRjdO/z6ncpamQCDQ433Lsnrdl+z6WPejb9J9PaKef/55LVy4UGvWrNFZZ52VreH5TqqvRyQS0U033aQHH3xQJSUlTg3Pd9L5/6Ozs1OBQECrV6/WZZddpmuvvVY/+tGPtHLlSqpnGZTOa9LY2Kg5c+bo/vvv17Zt2/TSSy/prbfe0qxZs5wYKk7i5O9z/lxN0RlnnKFgMNjrL5yDBw/2StNRZ599dszr8/Pzdfrpp2dtrH7Ql9cjas2aNfrGN76hn//857r66quzOUzfSPf1aG9v19atW7V9+3bdeuutkuxwYFmW8vPz9fLLL+szn/mMI2P3or78/zFixAidc845CofDXfeNHz9elmXp3Xff1QUXXJDVMXtdX16TRYsWqbq6WnfddZckqby8XKeccoquvPJKPfLII8y+OMjp3+dUzlI0YMAAVVRUqL6+vsf99fX1qqqqivkxl19+ea/rX375ZU2cOFGhUChrY/WDvrwekl0xmzFjhp577jnWbWRQuq9HQUGBGhoatGPHjq7brFmzVFpaqh07dmjSpElODd2T+vL/R3V1td577z0dOXKk677du3crLy9P5557blbH6wd9eU0++ugj5eX1/DUdDAYlnajawBmO/z7PyjYDj4pug37mmWesxsZG64477rBOOeUU6+2337Ysy7Luuece6+tf/3rX9dGtt3PnzrUaGxutZ555hlYaGZTu6/Hcc89Z+fn51pIlS6z9+/d33VpaWtx6Cp6S7utxMnZrZla6r0d7e7t17rnnWl/5ylesP/3pT9Yrr7xiXXDBBdYtt9zi1lPwnHRfk2effdbKz8+3li5dau3Zs8fauHGjNXHiROuyyy5z6yl4Rnt7u7V9+3Zr+/btliTrRz/6kbV9+/autiZu/z4nnKVpyZIl1pgxY6wBAwZYl156qfXKK690PTZ9+nTrU5/6VI/rf/vb31qf+MQnrAEDBljnnXeetWzZModH7G3pvB6f+tSnLEm9btOnT3d+4B6V7v8f3RHOMi/d12PXrl3W1VdfbQ0ePNg699xzrXnz5lkfffSRw6P2tnRfkyeffNK68MILrcGDB1sjRoyw/vmf/9l69913HR619/zmN79J+PvA7d/nAcuiNgoAAGAK1pwBAAAYhHAGAABgEMIZAACAQQhnAAAABiGcAQAAGIRwBgAAYBDCGQAAgEEIZwAAAAYhnAEAABiEcAYAMezfv19Dhw7VjTfe2OP+X/7ylwqFQlqwYIFLIwPgdYQzAIhhxIgR+va3v63/+q//0rZt2yRJv/3tb/XVr35V3/zmN/Wd73zH5REC8CrCGQDEceedd2rEiBG6++679fvf/17XX3+9vva1r+mJJ57oumbZsmW69NJLFQqFtHDhQvcGC8AzCGcAEMeQIUP0yCOP6P/+7/901VVXacqUKfrJT36iQCDQdc2IESP04IMPaurUqe4NFICn5Ls9AAAwWUlJiSQpEAho5cqVCgaDPR6PhrL//u//dnpoADyKyhkAxLFjxw594QtfUHV1tY4cOaIVK1a4PSQAPkA4A4AY/vznP+uaa67R5Zdfrt/85jf64he/qIULF6q1tdXtoQHwOMIZAJzk7bff1tVXX63S0lK9+OKLCoVCevTRR9Xc3Kzvfve7bg8PgMcRzgCgm/379+vqq6/WWWedpV/+8pcaPHiwJGncuHGaOXOmnnjiCb399tvuDhKApwUsy7LcHgQA5Krjx4/r+PHj+uY3v6lzzjlH9913n0KhUK+NAwCQKsIZAPTDwoUL9eCDD/a479lnn9WMGTPcGRCAnEc4AwAAMAhrzgAAAAxCOAMAADAI4QwAAMAghDMAAACDEM4AAAAMQjgDAAAwCOEMAADAIIQzAAAAgxDOAAAADEI4AwAAMAjhDAAAwCD/P//C2paOdujUAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 700x700 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## Plot the data\n",
    "plt.figure(figsize = (7,7))\n",
    "\n",
    "plt.scatter(X[y == 0,0],\n",
    "            X[y == 0,1],\n",
    "            c = \"blue\",\n",
    "            s = 60,\n",
    "            label=\"0\")\n",
    "plt.scatter(X[y == 1,0],\n",
    "            X[y == 1,1],\n",
    "            c = \"orange\",\n",
    "            marker = 'v',\n",
    "            s = 60,\n",
    "            label=\"1\")\n",
    "\n",
    "plt.legend(fontsize = 12)\n",
    "plt.xlabel(\"$x_1$\",fontsize = 12)\n",
    "plt.ylabel(\"$x_2$\",fontsize = 12)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Clear linear decision boundary, so our perceptron should do just fine here. Let's split our data\n",
    "into train and test sets so that we can see how the model does."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y,\n",
    "                                    shuffle=True,\n",
    "                                    random_state=435,\n",
    "                                    test_size=.3,\n",
    "                                    stratify=y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Train the perceptron"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The weights are:\n",
      "[ 0.04586024 -0.04772532]\n",
      "The errors are:\n",
      "[2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]\n",
      "The accuracy is:\n",
      "1.0\n"
     ]
    }
   ],
   "source": [
    "w, J, pred_train = perceptron(X_train, y_train, z = 0.0, eta = 0.1, num_it = 10)\n",
    "\n",
    "print(\"The weights are:\")\n",
    "print(w)\n",
    "\n",
    "print(\"The errors are:\")\n",
    "print(J)\n",
    "\n",
    "print(\"The accuracy is:\")\n",
    "print(accuracy(y_train, pred_train))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Test it on our test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "def perceptron_test(X, w, z = 0.0):\n",
    "    y_pred = []\n",
    "    for i in range(X.shape[0]):\n",
    "        f = np.dot(X[i], w)\n",
    "\n",
    "        y_hat = activation(f, z)\n",
    "        y_pred.append(y_hat)\n",
    "    return y_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The accuracy is:\n",
      "1.0\n"
     ]
    }
   ],
   "source": [
    "y_pred = perceptron_test(X_test, w, z = 0.0)\n",
    "\n",
    "print(\"The accuracy is:\")\n",
    "print(accuracy(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected, it does very well on the test set."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Messier Data Set\n",
    "\n",
    "Let's try this same process on a modified version of this data set with less clear separation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Generate the random data\n",
    "np.random.seed(440)\n",
    "n_rows = 100\n",
    "diff = .1\n",
    "X = np.random.random((n_rows,2))\n",
    "X_prime = X[(X[:,1] - X[:,0]) <= -diff,:]\n",
    "X_2prime = X[(X[:,1] - X[:,0]) >= diff,:]\n",
    "X_3prime = [[.4,.9],[.6,.45],[.7,.9],[.3,.19],[.1,.4]]\n",
    "\n",
    "del X\n",
    "X = np.append(X_prime,np.append(X_2prime,X_3prime,axis = 0),axis=0)\n",
    "\n",
    "y = np.empty(np.shape(X)[0])\n",
    "y[(X[:,1] - X[:,0]) <= -diff] = 1\n",
    "y[((X[:,1] - X[:,0]) >= diff)] = 0\n",
    "y[-5] = 1\n",
    "y[-4] = -1\n",
    "y[-3] = 1\n",
    "y[-2] = -1\n",
    "y[-1] = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmcAAAJbCAYAAAChCjPHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAABOjUlEQVR4nO3de3RU9b3//9dkMtzETPCGoICRSEBJtIZKTHRZW4vFVkvTVqxnNSD1/A4tikK1ingULy3a21flANoiwllFy6kGTk8X9Zh12ioXoYUCThtWafACViiCuaEthMn+/bGdkJCZyUwyM/uz934+1poVMrMTPnNJ8pr357Pfn4BlWZYAAABghDynBwAAAIATCGcAAAAGIZwBAAAYhHAGAABgEMIZAACAQQhnAAAABiGcAQAAGCTf6QE4pb29Xe+9955OPfVUBQIBp4cDAAA8zrIstba2avjw4crLS1wf8204e++99zRixAinhwEAAHxm3759OvfccxPe7ttwduqpp0qyH6CCggKHRwMAALyupaVFI0aM6Mggifg2nMWmMgsKCghnAAAgZ3paTsUJAQAAAAYhnAEAABiEcAYAAGAQwhkAAIBBfHtCAAAA6JtoNKq2tjanh+G4UCikYDCYse9HOAMAAGmxLEsHDhxQc3OzLMtyejiOCwQCCofDOvvsszPS2J5wBgAA0tLc3KympiadeeaZOuWUU3y9045lWfrwww/1/vvva+DAgSosLOzz9yScAQCAlFmWpYMHD6qgoEBnnHGG08MxwsCBA3X06FEdPHhQ4XC4z2GVEwIAAEDKotGootEoDdxPUlBQ0PHY9BXhDAAApOz48eOSpPx8Jt86iz0escenLwhnAAAgbX5eZxZPJh8PwhkAAIBBCGcAAAAGIZwBAAB87MiRI7rzzjs1fPhwDRgwQJdccol+/vOf53QMRoSz1157Tddff72GDx+uQCCgtWvX9vg1r776qsrLyzVgwACdf/75evrpp7M/UAAAkBWRiPTQQ9KcOfbHSMSZcVRXV2vlypV68MEH9etf/1qf/OQn9bWvfU3PP/98zsZgxKkWH374oS6++GLdcsst+vKXv9zj8W+99Zauu+46/eu//qt+9rOfaePGjfrWt76lM888M6WvBwAAZmhokKZNkzZtkoJBKS9Pam+XFiyQqqqkFSuk4uLcjGXdunWqq6vT888/r6997WuSpKuvvlrvvPOO7r77bk2dOjWj2zQlYkQ4mzx5siZPnpzy8U8//bRGjhypJ554QpI0btw4bd26VT/84Q8JZ3BUJCLV1kpNTVJhoVRdLZWWOj0qADBTQ4M0caLU3Gx/Ho3al5jNm+3bt2zJTUBbs2aNBg8erK9+9atdrr/lllt08803a8uWLaqsrMz6OIyY1kzX66+/rkmTJnW57tprr9XWrVsTbsB69OhRtbS0dLkAmdLQYL/DKyuTHnlEWrzY/lhWJl1xhX07AKCradPsYJaob2s0at8+fXpuxvOnP/1J48aN69bDraysrOP2XHBlODtw4ICGDh3a5bqhQ4fq+PHjOnToUNyvWbhwocLhcMdlxIgRuRgqfCD2zm/LFvvzaFRqazvxyyb2zo+ABqeZsqYHkOzX36ZNiYNZTDQqbdyYm9fr4cOHddppp3W7Pnbd4cOHsz8IGTKt2RsnN3uzLCvu9THz5s3T3LlzOz5vaWkhoCEj0nnnt2FDTocGSDJrTY+fsMwhudpa+/WYym5HwaC0Zk1uHr9kzWRz1XjXleHs7LPP1oEDB7pcd/DgQeXn5+v000+P+zX9+/dX//79czE8+EjsnV9POr/z45czcsm0NT1+QBhOTVOT/dikEs7y8qTGxqwPSaeffnrc6tgHH3wgSXGratngymnNyy+/XHV1dV2ue+WVVzRhwgSFQiGHRgU/ir3zS0XsnR+QS6at6fE6ljmkrrDQDq2paG+XhgzJ6nAkSaWlpdq1a1e3/TEjH8+pjh8/PvuDkCHh7MiRI9qxY4d27NghyW6VsWPHDu3du1eSPSVZU1PTcfzMmTP1zjvvaO7cudq1a5eWL1+uZ599VnfddZcTw4ePxd75pSJX7/yAGBPX9HgdYTh11dWpVc0k+7jq6uyOR5K+9KUv6ciRI3rppZe6XL9y5UoNHz5cEydOzP4gZMi05tatW3X11Vd3fB5bGzZt2jStWLFC+/fv7whqklRUVKR169Zpzpw5Wrx4sYYPH66nnnqKNhrIORPf+QExpq7p8SqWOaSntFSqrLSrjMleo8GgVFEh5aJoNXnyZH32s5/VN7/5TbW0tKi4uFgvvPCCXn75Zf3sZz/LSY8zyZBw9qlPfapjQX88K1as6HbdVVddpT/+8Y9ZHBXQs+pqex1JKnL1zg+IMXFNj5cRhtO3cuWJNZHxHrdgUAqH7XV6uVJbW6v58+frgQce0AcffKCxY8fqhRde0E033ZSzMRgxrQm4VeydX09vpoJBeyFwjpYrAJKo7OYayxzSV1xsV84qKuzPg0EpFDrxO7WiIvcnqwwePFhPPvmk9u/fr6NHj2rnzp05DWaSIZUzwM1MfOcHSFR2c40w3DvFxXaboUjEriY2NtqPTXW1f9/QEs6APoq985s+3V5H0vnU+WjUfufHqfNwgolreryMMNw3paVM88YQzmAUtzZt5J0fTEVlN3cIw8gUwhmMkKxp44gR0mc+I513nvlhjXd+MA2V3dwiDCMTCGdwXE8dzPfts3+R5eXRYRvoDSq7uUMYRiYQzuC4npo2xsQW2rLdDNA7VHZzgzCMviKcwVGpNm3sjI3EAbgBYRi9RZ8zOCqdvSk7Y7sZAIBXEc7gqHSaNp6MjcQBAF5EOIOj0mnaeDI6bAMAvIhwBkdVV6e2D108dNgGAGRaa2urvvOd72jSpEk688wzFQgEtCDV7sIZwgkBcFSqTRvjocM2ALhY4xvSnmWSrBQODkijb5WGlGV7VDp8+LB+8pOf6OKLL9aUKVO0bNmyrP+fJyOcwXE9NW2Mhw7bAOBy76+Xdi+SAkEln8hrl6yoVDA2J+Fs1KhRamxsVCAQ0KFDhxwJZ0xrwnGxpo0VFfbnPZ29SYdtAPCAohopVGgHL6stySVqH1dUk5NhBQIBBQKBnPxfiRDOYIRY08Y33pAeeMDuYTZihH1bMCiFQidCW0UFDWgBwPVCp0oX3iOppyAUkC66VwoNzsWojMC0JoxyctNGOmwDgIeNmSXVPy61NSU+JhSWLpiVsyGZgHAGo9FhGwA8LFY923mf4p8Y4L+qmcS0JgAAcNKYWXZ1LB4fVs0kwhkAAHBSwrVn/qyaSYQzAADgtHjVM59WzSTWnAEAAKd1W3vmbNXs17/+tT788EO1trZKkurr6/Xiiy9Kkq677joNGjQoq/9/wLKsVFrzek5LS4vC4bCam5tVUFDg9HAAAHCFf/7zn3rrrbdUVFSkAQMGZO4bt7VKa0faZ26GCqUp+xwLZ+edd57eeeeduLe99dZbOu+887pdn8rjkmr2YFoTAAA4r2PtmRxfa/b222/Lsqy4l3jBLNOY1jRQJCLV1kpNTVJhod3bi3YSAADPK5kt5Q+WRs9weiSOIpwZpKFBmjZN2rTJ7oaflye1t0sLFkhVVfZ2RXTFBwB4Vv4gqeQ2p0fhOKY1DdHQYG/+vWWL/Xk0KrW1ndgIfPNm+/aGBufGCAAAso9wZohp06Tm5hNh7GTRqH379Ok5HRYAAMgxwpkBIhF7KjNRMIuJRqWNG+3jAQCANxHODFBba68xS0UwaG8EDgCAk3zaiSuhTD4ehDMDNDXZi/9TkZcnNTZmdTgAACQUCoUkSR999JHDIzFL7PGIPT59wdmaBigstM/KTEV7uzRkSFaHAwBAQsFgUIWFhTp48KAkadCgQQoETt4X0z8sy9JHH32kgwcPqrCwUMFUp8KSIJwZoLrabpeRimjUPh4AAKecffbZktQR0CAVFhZ2PC59RTgzQGmpVFlpt9FIdlJAMChVVEjjx+dubAAAnCwQCGjYsGE666yz1NbW5vRwHBcKhTJSMYshnBli5Uq7j1midhrBoBQO241os4FdCQAA6QoGgxkNJbBxQoAhiovtyllFhf15MCiFQifO4qyosG/P9A4BDQ327gNlZdIjj0iLF9sfy8qkK66g6S0AALkWsHx6LmyqO8M7IRKx22U0NtqL/6urszOVGduVoKdqXTZCIQAAfpNq9mBa00ClpbmZUkxnV4ING7I/HnhI4xvSnmWSUnnvF5BG3yoNKcv2qOAFvLbgA4Qzn4rtStCTzrsSsAYNKXt/vbR7kRQIKvnqiXbJikoFY/kDitTw2oIPsObMp9iVAFlVVCOFCu0/jlZbkkvUPq6oxukRwy14bcEHCGc+xa4EyKrQqdKF90jqqTFlQLroXik0OBejghfw2oIPEM58il0JkHVjZkmhcPJjQmHpglm5GQ+8g9cWPI5w5lPV1ckb3nbGrgTolR4rHFQ20Eu8tuBxhDOfiu1K0NO6s2DQ7oPGrgTolWQVDiob6AteW/AwwlmWRCLSQw9Jc+bYHyMRp0fU3cqVdh+zRAEt27sSwAcSVjiobKCPeG3Bw2hCm+EmtA0Ndv+wTZvscJOXZ6/ZikbtCtSKFWY1dG1osPuYbdzojvHChdpapbUjpbamE9eFCqUp+/gDir7htQWXoQmtAzp33JfsgNN5XdfmzfbtJnXcLy62G8zmalcC+FCswrHzPtmNQ6lsIEN4bcGjqJxlsHJWVWUHr2QL7YNBe59MOu7DVzpXOKhsIJN4bcFFUs0erDnLkFjH/Z7OgOzccR/wjY71QaKygczitQUPYlozQ2Id91NpTxHruM92SPCVktlS/mBp9AynRwKv4bUFjyGcZUis434q4YyO+/Cl/EFSyW1OjwJexGsLHsO0ZobQcR8AAGQC4SxD6LgPAAAygXCWIXTcBwAAmUA4yyA67gMAgL4inGVQcbHd56yiwv48GJRCoRNhraLCrAa0AADAPJytmWF03AcAAH1BOMuS0lL6mCG7IhG7v15Tk322cHU1rzkA8ALCGeAyDQ3StGn2jhSdN6tfsIDN6gHAC1hzBrhIQ4M0caK9dlGy27K0tZ1o47J5s317Q4NzYwQA9A3hDHCRadOk5ubEPfWiUfv26dNzOiwAQAYxrQm4RCRiT2X2JBqVNm60j2cNGrItlbWPrI8E0kM4A1yittZeY5bKThTBoH22MH8AkS2prH2UWB8J9AbhDHCJpib7j1sq4Swvz27jAmRDbO1jc7P9eTTa9XW5ebM0YYL97yNHEh8TWz9JQAO6Ys0Z4BKFhXbVIRXt7XZ/PSAbUl37yPpIoHcIZ4BLVFenVjWT7OOqq7M7HvhTbO1jqq/FZDqvjwRwAuEMcInSUqmyMvHerTHBoL2ehx0pkA2xtY+ZElsfCeAE1pwBLrJy5Ym1PvEqF8GgFA6fWIyNvuEsw+7SWfuYCtZHAt1ROQNcpLjYXkBdUWF/HgxKodCJSkZFBQusM6Ghwa4+lpVJjzwiLV5sfywrk664wt9NftNZ+5gK1kcC3QUsy7KcHoQTWlpaFA6H1dzcrIKCAqeH0w3v2NGTSMSeDmpstP+4VVczlZkJnc9ETFad9GsIjkTskJrp78lrF36QavYgnBkWzhL1DopG6QsE5EJVlR28kk3bBYN2lXLDhtyNyySpPEap8PvjCP8hnPXAxHDGO3bAWelWhd54IzsVbdMr5w0Ndh+zWJ+zeE491X5zeeQIv8+AmFSzB2vODMK+iYCz0jkTMRtnGXpprVtenvTSS6yPBHqDszUNwb6JgPOc3IUhla77pnTUnzbtROf/RI4ckR580J6yZH0kkB7CmSHYNxFwnpO7MKRTOXdyjVZv30jy+wpIHdOahoi9Y08FfYGA7HBqF4ZUu+6n01E/EpEeekiaM8f+mKku/E5P/QJ+QOXMEOybCDgvtgtDqmdrZmpqLpOV80RnfC9YkJkzvp2c+gX8gsqZIdg3ETDDypX2WYSJqkPZ2IUhU5Xz2Lq1LVvsz6NRqa3txO+W2Lq1vpxYwBtJIPsIZ4Zg30TADE7swpCpwJOLM755IwlkH33O6HMGIIFcnWWYbn+1eB31c9mjjUa9QO+kmj1Yc2aQ2Dv26dPtRb8n7xBQUcEOAUAu5eosw0ysdcvlGd8rV6b2RjKTU7+AnxDODFNcTF8gwI/6GnhyuVCfN5JAdhHODEVfIMBf+hp4cr1QnzeSQPaw5sygNWcAIPUu8GRi3RqA7GLNGQC4VG8q5071aAOQebTSAACPcKJHG4DMI5wBgEc40aMNQOYxrQkAHsJCfcD9CGcA4EGc8Q24lzHTmkuWLFFRUZEGDBig8vJyrV+/Punxq1at0sUXX6xBgwZp2LBhuuWWW3T48OEcjRYAACA7jAhnq1ev1p133qn58+dr+/btuvLKKzV58mTt3bs37vEbNmxQTU2NvvGNb+jPf/6zfvGLX+gPf/iDbr311hyPHAAAILOM6HM2ceJEXXrppVq6dGnHdePGjdOUKVO0cOHCbsf/8Ic/1NKlS7Vnz56O6xYtWqTvf//72rdvX0r/J33OAABwQOMb0p5lklKJHwFp9K3SkDSa+BnMNX3Ojh07pm3btunee+/tcv2kSZO0adOmuF9TWVmp+fPna926dZo8ebIOHjyoF198UZ///OcT/j9Hjx7V0aNHOz5vaWnJzB0AAACpe3+9tHuRFAgq+QReu2RFpYKxnglnqXJ8WvPQoUOKRqMaOnRol+uHDh2qAwcOxP2ayspKrVq1SlOnTlW/fv109tlnq7CwUIsWLUr4/yxcuFDhcLjjMmLEiIzeDwAAkIKiGilUaAcvqy3JJWofV1Tj9IhzzvFwFhMIBLp8bllWt+ti6uvrNXv2bD3wwAPatm2bXn75Zb311luaOXNmwu8/b948NTc3d1xSnf4EAAAZFDpVuvAeSfH/xp8QkC66VwoNzsWojOL4tOYZZ5yhYDDYrUp28ODBbtW0mIULF6qqqkp33323JKmsrEynnHKKrrzySj366KMaNmxYt6/p37+/+vfvn/k7AAAA0jNmllT/uNTWlPiYUFi6YFbOhmQSxytn/fr1U3l5uerq6rpcX1dXp8rKyrhf89FHHykvr+vQgx+3wDbg/AYAAJBMj9Uz/1bNJAPCmSTNnTtXy5Yt0/Lly7Vr1y7NmTNHe/fu7ZimnDdvnmpqTsw5X3/99aqtrdXSpUv15ptvauPGjZo9e7Yuu+wyDR8+3Km7AQAAUjVmll0di8fHVTPJgGlNSZo6daoOHz6shx9+WPv379f48eO1bt06jRo1SpK0f//+Lj3Ppk+frtbWVv3Hf/yHvv3tb6uwsFCf/vSn9fjjjzt1FwAAQDpi1bOd96lrWw1/V80kQ/qcOYE+ZwAAOKytVVo7suvas1ChNGWfJ8NZqtnDiGlNAADgQ93WnlE1kwyZ1gQAAD4T2ymg/aiUF5Laj9kfj7wpbb39pIO9tVNATwhnAAAg9zrvFBBbYdV+XNrz3EkHZmGnAMO3kCKcAQCA3CuqkXbef1Kvs3bJau9+bKZ3CjB8CynWnAEAgNxzcqcAw7eQonIGAJ1EIlJtrdTUJBUWStXVUmmp06MCPMqpnQIStvE4mTMnKFA5AwBJDQ1SVZVUViY98oi0eLH9saxMuuIK+3YAGebkTgHJmuDGONQMl3AGwPcaGqSJE6UtW+zPo1Gprc3+KEmbN9u3E9CALHBqpwCDt5AinAHwvWnTpObmE2HsZNGoffv06TkdFuAPCUNSDsKRoVtIEc4A+FokIm3alDiYxUSj0saN9vEAMixeSMpFOHIyGCZBOAPga7W1UjCY2rHBoLRmTXbHA/iSkzsFOBUMkyCcAfC1piYpL8XfhHl5UmNjVocD+FfnkJTLcGTgFlKEMwC+VlgotcfpeRlPe7s0ZEhWhwP4V0dIUu7DkVPBMAH6nAHo4MceX9XV0oIFqR0bjdrHA8iSktlS/mBp9Izc/r8dfc/mOV41k6SAZVmpbCzlOS0tLQqHw2publZBQYHTwwEc1dBgn7G4aZO9riovz64SRaN2768VK6TiYqdHmT1VVXYbjWQnBQSDUkWFtGFD7sYFIIeOfyTtWW4Hw/xBWfkvUs0ehDPCGXwu1uMrUSuJYFAKh+3w4tWAxmMAIBdSzR6sOQN8jh5fduDassWujEl2GAuFTpzFWVFBMAOQO6w5A3ws1uOrJ517fHl1DVpxsT1lGYnY7TIaG+3F/9XV0vjxTo8OgJ8QzgAfi/X46qkBq3Six5dXw1lMaan37yMAszGtCfgYPb4AwDyEM8DH6PEFAOYhnAE+Vl2d2pSmRI8vAMgVwhngY6WlUmVlz3tLBoN2LzAWxgNA9hHOAJ9budLu4ZUooMV6fK1YkdNhAYBvEc4An6PHFwCYhVYaAOjxBcBdGt+Q9iyTlMomRwFp9K3SkLJsjypjCGcAOtDjC4ArvL9e2r1ICgSVfBKwXbKiUsFYV4UzpjUBAIC7FNVIoUI7eFltSS5R+7iiGqdHnBbCGQAAcJfQqdKF90gK9HBgQLroXik0OBejyhjCGQAAcJ8xs6RQOPkxobB0wazcjCeDCGcAAMB9eqyeubNqJhHOAACAWyWrnrm0aiYRzgAAgFslrJ65t2om0UoDSFskItXWSk1N9sbh1dW0nwAAx4yZJdU/LrU1nbjOxVUziXAGpKyhQZo2Tdq0ye6en5cntbdLCxbY+06uWEEXfQDIuVj1bOd9spvSurtqJjGtCaSkoUGaONHexkiSolGprc3+KEmbN9u3NzQ4N0YA8K3Oa89cXjWTCGdASqZNk5qbT4Sxk0Wj9u3Tp+d0WAAAqdPaM7m+aiYxrQn0KBKxpzJ7Eo1KGzfax7MGDQByrGS2lD9YGj3D6ZH0GeEMfeb1BfK1tfYas0RVs86CQXvjcC/dfwBwhfxBUsltTo8iIwhn6DW/LJBvarLvWyrhLC9PamzM+pAAAB7GmjP0ip8WyBcW2qEzFe3t0pAhWR0OAMDjCGfoFT8tkK+uTq1qJtnHVVdndzwAAG8jnCFtsQXyPQWWzgvk3ay0VKqstKdukwkG7enc8eNzMy4AgDcRzpC22AL5VMQWyLvdypVSOJz4fgeD9u0rVuR0WAAADyKcIW2xBfKp8MoC+eJie31dRYX9eTAohUInwlpFhX27F06AAAA4i7M1kTa/LpAvLpY2bLCnadessUPnkCH2GjOmMgEAmRKwLMtyehBOaGlpUTgcVnNzswoKCpwejqtEIlJZWXrHE14AAH6XavZgWhNpY4E8AADZQzhDr7BAHgCA7CCcoVdYIA8AQHZwQgB6jQXyAABkHuEMfVZaykbfAABkCtOaAAAABiGcAQAAGIRwBgAAYBDCGQAAgEEIZwAAAAYhnAEAABiEcAYAAGAQwhkAAIBBCGcAAAAGIZwBAAAYhHAGAABgEMIZAACAQQhnAAAABiGcAQAAGIRwBgAAYBDCGQAAgEEIZwAAAAbJd3oAAAAggcY3pD3LJFkpHByQRt8qDSnL9qiQZYQzAABS4URQen+9tHuRFAgq+WRXu2RFpYKxhDMPIJwBAJAKJ4JSUY20836prUlSNPmxoUL7eLgea84AAEhFUY0dgKyoZLUluUQzF5RCp0oX3iMp0MOBAemie6XQ4L7/n3Ac4QwAgFQ4FZTGzJJC4R7GFpYumJWZ/w+OY1oTMFwkItXWSk1NUmGhVF0tlZY6PSrAp8bMkuof/3iaMYFMB6VYKNx5n+Kvd6Nq5jVUzgBDNTRIVVVSWZn0yCPS4sX2x7Iy6Yor7NsB5FiP1bMsBaVk1TOqZp5DOAMM1NAgTZwobdlifx6NSm1t9kdJ2rzZvp2ABjjAiaCUMBRSNfMiwhlgoGnTpObmE2HsZNGoffv06TkdFgDJuaAULxRSNfMkwhlgmEhE2rQpcTCLiUaljRvt4wHkmBNBqVsopGrmVYQzwDC1tVIwmNqxwaC0Zk12xwMgDqeCUudQSNXMswhngGGamqS8FH8y8/KkxsasDgdAIk4EpY5QKKpmHkY4AwxTWCi1t6d2bHu7NGRIVocDIBGnglLJbKl8kTTm9tz8f8g5whlgmOrqntebxUSj9vEAHOJEUMofJJXcZn+EJxHOAMOUlkqVlT2vOwsG7T5o48fnZlwA4iAoIQsIZ4CBVq6UwuHEAS0YtG9fsSKnwwIA5ADhDDBQcbHdgLaiwv48GJRCoRNhraLCvr242LkxAgCyg701AUMVF0sbNth9zNassc/KHDLEXmPGVCZ8r/ENac8yxd9r8mQBafSt0pCybI8KyAjCGWC40lI2Oge6eX+9tHuRFAgq+SRQu2RFpYKxhDO4BtOaAAD3KaqRQoV28LLaklyi9nFFNU6PGEgZ4QwA4D4J97c8GVscwX2MCWdLlixRUVGRBgwYoPLycq1fvz7p8UePHtX8+fM1atQo9e/fX6NHj9by5ctzNFoAgOPi7W95MrY4ggsZseZs9erVuvPOO7VkyRJVVVXpmWee0eTJk1VfX6+RI0fG/Zobb7xRf//73/Xss8+quLhYBw8e1PHjx3M8cgCAY2LVs533Kf6JAVTN4E4By7JSOdUlqyZOnKhLL71US5cu7bhu3LhxmjJlihYuXNjt+Jdfflk33XST3nzzTZ122mkp/R9Hjx7V0aNHOz5vaWnRiBEj1NzcrIKCgr7fCQBA7rW1SmtHSm1N3W8LFUpT9hHOYIyWlhaFw+Ees4fj05rHjh3Ttm3bNGnSpC7XT5o0SZs2bYr7Nb/85S81YcIEff/739c555yjMWPG6K677tI//vGPhP/PwoULFQ6HOy4jRozI6P0AADgg4dozqmZwL8fD2aFDhxSNRjV06NAu1w8dOlQHDhyI+zVvvvmmNmzYoD/96U9as2aNnnjiCb344ouaNSvxuoJ58+apubm547Jv376M3g8AgEPirT1jrRlczIg1Z5IUCHR912NZVrfrYtrb2xUIBLRq1SqFw/YP5I9//GN95Stf0eLFizVw4MBuX9O/f3/1798/8wMHADir29ozqmZwN8crZ2eccYaCwWC3KtnBgwe7VdNihg0bpnPOOacjmEn2GjXLsvTuu+9mdbwAAAN1rp5RNYPLOR7O+vXrp/LyctXV1XW5vq6uTpWVlXG/pqqqSu+9956OHDnScd3u3buVl5enc889N6vjBQAYqGPtmaiawfUcD2eSNHfuXC1btkzLly/Xrl27NGfOHO3du1czZ86UZK8Xq6k50d355ptv1umnn65bbrlF9fX1eu2113T33XdrxowZcac0AQA+UDJbKl8kjbnd6ZEAfWLEmrOpU6fq8OHDevjhh7V//36NHz9e69at06hRoyRJ+/fv1969ezuOHzx4sOrq6nT77bdrwoQJOv3003XjjTfq0UcfdeouAACclj9IKrnN6VEAfWZEnzMnpNprBAAAIBNc0+cMAAAAJxDOAAAADEI4AwAAMIgRJwQAMF8kItXWSk1NUmGhVF0tlZY6PSoA8B7CGYCkGhqkadOkTZukYFDKy5Pa26UFC6SqKmnFCqm42OlRAoB3EM4AB5lejWpokCZOlJqb7c+jUfsSs3mzffuWLQQ0AMgUWmnQSgMOSFSNikbNqkZVVdnBq3MgO1kwKFVUSBs25G5cAOBGqWYPKmc+ZnrVxqvcUo2KROzw2JNoVNq40T6e1w8A9B3hzIdYQ+SsadPsYJaoGhWN2rdPn+5sNaq21n59JKuaxQSD0po1hDMAyATCmc+4pWrjVW6qRjU12cE9lXCWlyc1NmZ9SEDvNL4h7VkmKZVVPAFp9K3SkLJsjwpIiHDmM26p2niVm6pRhYV2RTUV7e3SkCFZHQ7Qe++vl3YvkgJBJW/v2S5ZUalgLOEMjiKc+YibqjZe5aZqVHW1PdWdimjUPh7ImExWu4pqpJ33S21Nknr44QsV2scDDiKc+YibqjZe5aZqVGmpVFmZ+tma48fnbmzwgUxWu0KnShfeI+28T8nDXkC66F4pNLj34wYygO2bfCRWtUmF01Ubr6quTi0cS2ZUo1aulMJhO4DFEwzat69YkdNhwQ+KauwqlhWVrLYkl2hq1a4xs6RQOPkxobB0waxM3QOg1whnPuKmqo1XxapRicJOTDBonznrdDWquNiunFVUnBhXKHRi/BUVnDyCLIlVuxTo4cAUq109fj+qZjAHTWh91IQ2EpHK0ljjGok4Hw68qPMZs/GqaLFqlGmhJxKxp7obG+3gXl3N6wNZ1tYqrR358VqxBEKF0pR9qYWqZN8vne8D9BJNaNENa4jMEKtGTZ9un3hx8g4BFRVm9porLWUNInKsx7ViaVa7En4/qmYwC5UzH1XOJPdWbbyKahTQg0xXu+J9P6pmyBEqZ4jLrVUbr6IaBfQg09Wubt+PqhnMQ+XMZ5WzzqjaAHCFTFe7On8/qmbIISpn6BFVGwCukOlqV8f3m0fVDEailQYAwHyd+5Rloh9ZyWypfJE05va+jw3IMMIZAMB8HX3KlJlqV/4gqeQ2+yNgGKY1AQDuUDJbyh8sjZ7h9EiArCKcAQDcIVbtAjyOaU0AAACDUDmDkSIRqbbW3qy9sNBu88GZpQAAPyCcwSgNDdK0adKmTV0b5C5YYG8EToNcAIDXMa0JY8S2ltqyxf48GpXa2k5sM7V5s317Q4NzYwQAINsIZzDGtGmJ9/yU7Oubm+2tpwAA8CqmNWGESMSeyuxJNGrvCRqJsAYNgAEa35D2LFPXfT8TCUijb5WGlGV7VHA5whmMUFtrrzFLVDXrLBi09wQlnAFw3Pvrpd2LpEBQySej2iUrKhWMJZyhR4QzGKGpyV78n0o4y8uzN2t3AmeRAuiiqEbaef/Hm7L38AssVGgf7yZUBh1BOIMRCgvtszJT0d4uDRmS1eF0w1mkAOLqtil7In3crN0pVAYd0esTAv75z38qEonoo48+6nbbxo0b+zQo+E91dWpVM8k+rro6u+PpjLNIASTVeVP2RDKxWbsTimrsip8Vlay2JJeoOyuDhupVOHv99dc1YsQIfepTn9KZZ56pxx57rMvtkydPzsjg4B+lpVJlpV2VSiYYtCtV48fnZlwSZ5EC6EHHpuyBBAe4tGompXDfYlx8Hw3Uq3D27W9/Wz/60Y90+PBhbdu2TbW1tZoxY4baP56XsqxU5qaBrlaulMLhxAEtGLRvX7Eid2OKnUXaU1Wv81mkAHwoWfXMrVWzGC9XBg3Vq3BWX1+vmhq7dDl27Fi9+uqrOnjwoL7yla/o2LFjGR0g/KO42J46rKiwPw8GpVDoRFirqLBvz+XarthZpKmInUUKwIcSVpg8UFHycmXQUL0KZwUFBfrb3/7W8fnAgQO1du1aDRgwQJ/73Oc6KmhAuoqLpQ0bpDfekB54QJo1y/4YidjX53rRfews0lQ4eRYpAAPEqzB5paLk5cqggXoVzq655ho999xzXa7Lz8/XqlWrNHr0aP3jH//IyODgX6Wldij7f//P/pjLNWadmX4WKQCDdKsweaii5OXKoIECVi8WiB07dkzHjx/XoEGD4t6+d+9ejRw5ss+Dy6aWlhaFw2E1NzeroKDA6eHAUJGIVJbGWeGRiHNBEoAB2lqltSPtvmehQmnKPu8El873LcZr9zHLUs0eaVXO/v3f/12S1K9fv4TBTJLxwQxIlclnkQIwUEeFSd6rKHm5MmiYtMLZ448/rjlz5iS8fe/evX0eEGAaE88iBWCwktlS+SJpzO1OjyTzOq89Y61Z1qS1Q0Btba1uvPFGHTlyRD/5yU8UCNjpubW1Vd/97nf11FNPxW1KC7hZ7CzS6dPtdhmddwiIRu2zSNkhAECH/EFSyW2pHeu27ZE6dkSYR9Usi9IKZ1/4whe0bt06ffGLX9SHH36o5557TsuXL9eCBQv0wQcfaMaMGdkaJ+Co2FmkkYjdLqOx0V78X13NVCaAPnDj9kgls6X8wdJo/uZnS69OCNi6dauuvvpq5eXl6ciRI7rhhhv02GOPqaSkJBtjzApOCAAAOC7eIvtEWHzvelk5IUCStm/frvvuu08ffvihWltbVVVVpRdffNFVwQwAACOwPRLiSCuc3XzzzfrkJz+p+vp6LV++XOvXr9ef/vQnTZkyRUePHs3WGAEA8C62R8JJ0gpn//M//6MHH3xQf/3rXzV9+nRVVVXpN7/5jf7whz/oc5/7nI4cOZKtcQIA4E1sj4STpLXm7MCBAzr77LO7Xf+Xv/xF11xzjYYNG6bf//73GR1gtrDmDABgjGRrz1hr5hlZWXMWL5hJUklJidavX69GNhYEACB9bI+ETnq1t2Y85513ntavX5+pbwcAgL94eeN0pCVj4UxKXFkDAAA9YHskfCyj4QwAAPQB2yNBhDMAAMzh5Y3TkbK0tm9CbkUiUm2t1NQkFRbaWwWVljo9KgBAVrE9ku/1avsmLzC5lUZDgzRtmrRpU/dNtquq2GQbAAA3ytr2TciuhgZp4kRpyxb782hUamuzP0rS5s327Q0Nzo0RAABkD9Oahpk2TWpuPhHGThaN2rdPny5t2JDToQEATNP4hrRnmaRUJsEC0uhbpSFl2R4V+ohwZpBIxJ7K7Ek0Km3caB/PGjQA8LH310u7F0mBoJJPhrVLVlQqGEs4cwGmNQ1SW2uvMUtFMCitWZPd8QAADFdUY2/vZEUlqy3JJWofV1Tj9IiRAipnBmlqshf/J5rS7CwvT2K3LADIIROnEGOtN3be18O4aGjrJoQzgxQW2mdlpqK9XRoyJKvDMR6tRgDklKlTiGNmSfWPx980PSYvJB15U9p6e5JvxJo0U9BKw6BWGpGIVJbGz0QkIo0fn73xmIpWIwAc0dYqrR2ZPATFhAqlKftyV6n682M9VM8CUiBZPebjQDlhsTTmW1kYICRaabhSaalUWdnzurNg0A4hfg1mtBoB4Ihue18m4sAUYrxN07uwWJPmIoQzw6xcKYXDiQNaMGjfvmJFTodljHRajQBAxvUYguTMnpgJg2NAGva5ONefjDVpJiGcGaa42K4KVVTYnweDUih0IqxVVNi3+3HaLtZqpKcTJmKtRr7+demhh+yvA4CM6LF65mDIiRccQ2GpYrmZgRIJEc4MVFxsN5h94w3pgQekWbPsj5GIfb0fg5mUXqsRSXr+eemRR+x1fFdcwVQngAxJVj1zMuR0C44fB8WBw8wNlIiLszUNVlrK2YedpdNqROp65mtsLZpfq44AMihh+woDQk7nMzc7B8VkZ3RSNTMOlTO4RjqtRk7GWjQAGZVoCtHpkNNRPVPXoJhsTZrTgRLdEM7gGtXVqVfN4um87RUA9EmiKUQTQk7JbKl8kTTmpJ5mpgZKdEM4g2uk2mokGba9ApAxncOOSSEnf5BUcpv9sTOTAyW6IJzBVXpqNdITtr0CkDGJphBNZmqgRBeEM7hKvFYjeWm8itn2CkBGJZpCNJUbA6UPcbYmXCfWaiQSsaco//pX6Wc/S+1ro1F77RoAZERsCtFNSmZL+YOl0TOcHgkSIJzBtTq3GnnzTbuiluyEgWDQrrj5cdsrAOjgxkDpM0xrwhPY9goA4BWEM3gC214BALyCaU14xslr0Rob7cX/1dVMZQIA3INwBs9h2ysAgJsxrQkAAGAQKmcAAKC7xjekPcvUdXP3RALS6FulIWXZHpUvEM4AAEB376+Xdi+SAkEln2hrl6yoVDCWcJYhhDMAAHLNDVWpohpp5/1SW5OkJE0kJSlUaB+PjCCcAQCQa26oSsW2etp5n5KHSDZQzzROCAAAINeKauxqkxWVrLYkl6izVanOG6UnwgbqGUc4AwAg1zo2IA/0cKDDVakex0nVLBsClmWlMuHtOS0tLQqHw2publZBQYHTw3GOG9Y9AIAXtbVKa0d+vKYrgVChNGWfs+En2ThNGJ+LpJo9WHPmd25Y9wAAXtTjmq5eVqUy/aY74TipmmWLMZWzJUuW6Ac/+IH279+viy66SE888YSuvPLKHr9u48aNuuqqqzR+/Hjt2LEj5f+PytnHUnnnFsM7JADIrLZWac050vHW7rfl9ZPOn25/7JBCmNq9WNp6W+pvuicslsZ8q+dxnvy3gr8JaXNV5Wz16tW68847tWTJElVVVemZZ57R5MmTVV9fr5EjRyb8uubmZtXU1Ogzn/mM/v73v+dwxB7C2TgA/MqEZR2hU6WhV0t/+2X329qPS3ue63xFajMY2WiB0e1vBX8TssmIytnEiRN16aWXaunSpR3XjRs3TlOmTNHChQsTft1NN92kCy64QMFgUGvXrqVy1ltuWfcAAJmUjQpTb3z0nrT2nNSOTfV38Z8fS+1N9yULP17wn4LOfyv4m9ArqWYPx8/WPHbsmLZt26ZJkyZ1uX7SpEnatGlTwq977rnntGfPHj344IMp/T9Hjx5VS0tLlws+xtk4APzIlHYWg4ZLZ1+bwoFp/C7ORguMjr8V4m9Cljkezg4dOqRoNKqhQ4d2uX7o0KE6cOBA3K/561//qnvvvVerVq1Sfn5qM7MLFy5UOBzuuIwYMaLPY/eUZD/I9LAB4EUmtbOoWN7zMen8Ls7Wm+6S2VL5ImnM7el9HdLieDiLCQS6voAsy+p2nSRFo1HdfPPNeuihhzRmzJiUv/+8efPU3Nzccdm3b1+fx+wpCX+QqZoB8DBTmqz2WD3rxe/ibLzpzh8kldxmf0TWOB7OzjjjDAWDwW5VsoMHD3arpklSa2urtm7dqttuu035+fnKz8/Xww8/rJ07dyo/P1+/+c1v4v4//fv3V0FBQZcLThLvB5mqGQAvM2lZx+UrpbyB8W/rze9i3nS7luPhrF+/fiovL1ddXV2X6+vq6lRZWdnt+IKCAkUiEe3YsaPjMnPmTJWUlGjHjh2aOHFirobuPd1+kPkBBuADpizrGDhUKn1AGQ1TvOl2JcfDmSTNnTtXy5Yt0/Lly7Vr1y7NmTNHe/fu1cyZMyXZU5I1NfZCzLy8PI0fP77L5ayzztKAAQM0fvx4nXLKKU7eFffr/IPMDzAAPzCpwpTpMMWbblcyIpxNnTpVTzzxhB5++GFdcskleu2117Ru3TqNGjVKkrR//37t3bvX4VH6BGfjAPAjUypM2QhTvOl2HSP6nDmBPmdJHP9I2rNcGj2DRZ8A/KNLb7A0e4BlUjb6if35MWnnPOmSx5y5T5Dkoj5nMBBn4wDwI1MqTNmYwaAFhqsYsX0TAACO69iiaJ7zyzpKZkv5g+0ZjEyIvemGKxDOAACIyXQo6i3ClK8RzgAAiCEUwQCsOQMAADAI4QwAAMAghDMAAACDEM4AAAAMQjgDAAAwCOEMAADAIIQzAAAAgxDOAAAADEI4AwAAMAjhDAAAwCBs3wQAQKoa35D2LJNkpXBwQBp9qzSkLNujgscQzgAASNX766Xdi6RAUMknn9olKyoVjCWcIW1MawIAkKqiGilUaAcvqy3JJWofV1Tj9IjhQoQzAABSFTpVuvAeSYEeDgxIF90rhQbnYlTwGMIZAADpGDNLCoWTHxMKSxfMys144DmsOUPaIhGptlZqapIKC6Xqaqm01OlRAUCOxKpnO+9T/BMDqJqhbwKWZaVyyonntLS0KBwOq7m5WQUFBU4PxxUaGqRp06RNm6RgUMrLk9rbpWhUqqqSVqyQioudHiUA5EBbq7R2pNTW1P22UKE0ZR/hDN2kmj2Y1kRKGhqkiROlLVvsz6NRqa3N/ihJmzfbtzc0ODdGAMiZhGvPqJqh7whnSMm0aVJz84kwdrJo1L59+vScDgsAnBNv7RlrzZABhDP0KBKxpzITBbOYaFTauNE+HgA8r1v1jKoZMoNwhh7V1tprzFIRDEpr1mR3PABgjM7VM6pmyBDCGXrU1GQv/k9FXp7U2JjV4QCAOTqqZ6JqhoyhlQZ6VFhon5WZivZ2aciQrA4HAMxSMlvKHyyNnuH0SOARVM7Qo+rqntebxUSj9vEA4Bv5g6SS2+yPQAZQOUOPSkulykq7jUaykBYMShUV0vjxff8/aXQLAPArmtDShDYlsT5nidppBINSOGwHuL40oqXRLQDAq2hCi4wqLraDV0WF/XkwKIVCJ87irKjITDCj0S0AwO+Y1kTKioulDRvsKcc1a+yzMocMsaccMzGVmU6j2w0b+v7/AZDU+Ia0Z5ni7xF5soA0+lZpSFm2RwX4GuEMaSstzfz6r1ij2550bnTLGjQgA95fL+1eJAWCSj6Z0i5ZUalgLOEMyDKmNWEEGt0CDimqsTfqtqKS1ZbkErWPK6pxesSA5xHOYAQa3QIOSbiB98nYmgjIFcIZjECjW8BB8TbwPhlbEwE5QziDEWh0Cziox+oZVTMglwhnMEKs0W1P686CQbvfWSbODgXQSbLqGVUzIKcIZzDGypV2I9tEAS3W6HbFipwOC/CHhNUzqmZArhHOYIxcNLoFkES86hlVMyDn6HMGo2S70S2AJGLVs533yW5KS9UMcAJ7a/p0b002FgcQV1urtHak1NZk9zWbso9wBmRIqtmDypnPJNpYfMECNhYHoE7Vs3lUzQCHEM58JLaxeHOz/Xk02rV9RWxjcdZ1AT5XMlvKHyyNnuH0SABfIpz5CBuLw3PYtDs78gdJJbc5PQrAtwhnPsHG4vAkNu3OLcIwkBOEM5+IbSyeShf+2MbihDMYr6hG2nm/vXhdPby42bS77wjDQE7Q58wn2FgcnsSm3blVVGOHXCsqWW1JLlHCMNAHhDOfYGNxeBabducOYRjICcKZT7CxODyLTbtzizAMZB3hzCfYWByexqbduUMYBrKOcOYjbCwOz2LT7twiDANZRTjzETYWh6exaXfuEIaBrKKVhs+wsTj6zNReV2zanVtjZkn1j3/cxuRjhGEgIwhnPlVaSh8z9JLJva46BwaCQnYRhoGsYVoTQHpM7nXVMd0mgkIudJ5KJgwDGUM4A5Ae03tdlcyWyhdJY27P7f/rR4RhICsClmWlsnDEc1paWhQOh9Xc3KyCggKnhwO4S1urtHZk1/VGJwsVSlP28Qfb645/JO1ZLo2eYW+YDiChVLMHlTMA6aPXFWLyB0kltxHMgAwinAHoHXpdAUBWcLYmgN7pdrZeDFUzx5na7gRASghnAHqPXldmMrndCYAeMa0JoPe6rT2jamYEk9udAOgR4QxA39DryjymtzsBkBThDEDf0OvKTMlO2IghTANGIpwB6Dsav5qHdieAaxHOAPQdva7MRLsTwJUIZwDgVQmrZ1TNAJMRzgDAy+JVz6iaAUYjnAGAl9HuBHAdwhkAeB3tTgBXIZwBgNfR7gRwFbZvAgA/KJkt5Q+WRs9weiTZY8qeoqaMA65FOAMAP4i1O/EyU/YUNWUccC2mNQEA3mDKnqKmjAOuRTgDAHiDKXuKmjIOuBbhDADgHabsKWrKOOBKhDMAgHeYsqeoKeOAKxHOAADeYsqeoqaMA65DOAMAeIspe4qaMg64DuEMAOA9puwpaso44CqEMwCA95iyp6gp44CrEM4AAN5kyp6ipowDrsEOAQAAb4pVrXbOc7ZaZco4kmHLKaMQzgAA3mXKnqKmjCMRtpwySsCyrFRisue0tLQoHA6rublZBQUFTg8HAADntLVKa0dKbU09HxsqlKbsM7MCaLhUswdrzgAA8Du2nDIK4QwAALDllEEIZwAAgC2nDGLMCQFLlizRD37wA+3fv18XXXSRnnjiCV155ZVxj62trdXSpUu1Y8cOHT16VBdddJEWLFiga6+9NsejBuAanI0G9GzMLKn+8fhrz6ia5YwR4Wz16tW68847tWTJElVVVemZZ57R5MmTVV9fr5EjR3Y7/rXXXtNnP/tZfe9731NhYaGee+45XX/99dqyZYs+8YlPOHAPABiPs9GAnnW0/bhPXd/IUDXLJSPO1pw4caIuvfRSLV26tOO6cePGacqUKVq4cGFK3+Oiiy7S1KlT9cADD6R0PGdrAj7D2WhAauL9rPAzkRGuOVvz2LFj2rZtmyZNmtTl+kmTJmnTpk0pfY/29na1trbqtNNOS3jM0aNH1dLS0uUCwEc4Gw1IDVtOOc7xcHbo0CFFo1ENHTq0y/VDhw7VgQMHUvoeP/rRj/Thhx/qxhtvTHjMwoULFQ6HOy4jRozo07gBuBBnowGpYcspRzkezmICga7vZi3L6nZdPC+88IIWLFig1atX66yzzkp43Lx589Tc3Nxx2bdvX5/HDMBlOBsNSE3Hz4r4mXCA4ycEnHHGGQoGg92qZAcPHuxWTTvZ6tWr9Y1vfEO/+MUvdM011yQ9tn///urfv3+fxwvA5TgbDUiN6VtOeZjj4axfv34qLy9XXV2dvvSlL3VcX1dXpy9+8YsJv+6FF17QjBkz9MILL+jzn/98LoYKwFTptskoqrHP3ORsNCCx/EFSyW1Oj8KXHA9nkjR37lx9/etf14QJE3T55ZfrJz/5ifbu3auZM2dKsqck//a3v+k///M/JdnBrKamRk8++aQqKio6qm4DBw5UONzDehIA3pNum4xP/MiuknU5G42qGQAzGLHmbOrUqXriiSf08MMP65JLLtFrr72mdevWadSoUZKk/fv3a+/evR3HP/PMMzp+/LhmzZqlYcOGdVzuuOMOp+4CACcV1din+ltRyWpLconaxxX/f5yNBsBYRvQ5cwJ9zgCP+fNjcRpnniwgXbLQDmadeznRwwlADrimzxkAZES6bTI4Gw2AoQhnALyhN20ySmZL5YukMbfnYoQAkBLCGQDvSFY9i7fgP3Y2Wv6g7I8NAFJEOAPgHQmrZyz4B+AehDMA3hKvekabDAAuQjgD4C1s2gzA5QhnALyHTZsBuJgROwQA+Fi62xCNvlUaUpbtUblPrHq2cx5VMwCuQzgDTJLuNkQFYwlniTixaTPhund43IAuCGeASYpqpJ33f7znYzT5saFC+3jE58SmzYTr3uFxA7pgzRlgkh4bqcawyN1I6e7xSbi28bgBXRDOANOkuw0RzEG47h0eN6ALwhlgmt5sQwRzEK57h8cN6EA4A0yU7jZEMAfhund43IAOhDPARGxD5G6E697hcQMkEc4Ac7ENkXsRrnuHxw2QRDgDzMU2RO5GuO4dHjeAcAYYjW2I3Itw3Ts8bgDhDDBaxx8q8QfKjQjXvcPjBp8jnAGmK5ktlS+Sxtzu9EiQLsJ17/C4wefYvgkwnRPbECFznNjj0wt43OBjhDMAyCbCde/wuMHHmNYEAAAwCOEMAADAIIQzAAAAgxDOAAAADEI4AwAAMAjhDAAAwCCEMwAAAIPQ5wwA4B2Nb0h7lkmyUjg4II2+VRpSlu1RAWkhnAEAvOP99dLuRVIgqOSTQ+2SFZUKxhLOYBzCGYDMoWoBpxXVSDvvl9qaJEWTHxsqtI8HDEM4A5A5VC36joDbN7FN03fep+SPYYBN1WEswhmAzKFq0XcE3L4bM0uqf/zj12ECobB0waycDQlIB2drAsicWNVCgR4OpGqRUFGNHVytqGS1JblECbiJ9Pg65PUHsxHOAGTWmFl2VSIZqhaJEXAzI9nrkNcfDEc4A5BZVC36joDbdwlfh7z+YD7CGYDMo2rRNwTczIj3OuT1BxcgnAHIPKoWfUfA7btur0Nef3AHwhmA7KBq0TcE3Mzo/Drk9QeXIJwByA6qFn1HwO27jteheP3BNehzBiB7OvebIlSkr1tDVQJur5TMlvIHS6NnOD0Sf6CRcp8RzgBkT0e4mEeo6C0Cbt/lD5JKbnN6FP5BI+U+Y1oTQHaVzJbKF0ljbnd6JO7EtBzchkbKfUblDEB2UbXoO6bl/MfNU4Psb9pnhDMAMB0B13/cPjXI/qZ9wrQmAACmcfvUII2U+4RwBgCAabywxyqNlHuNcAYAgIncvscqjZR7jXAGAICJvDA1SCPlXiGcAQBgKrdPDbJTSK8QzgAAMJUXpgbZ3zRthDMAAEzm9qlBGimnjXAGAIDJvDA1yE4haSGcAQBgOrdPDcYaKecPcnokrkA4AwDAdEwN+grbNwEA4AbsseobhDMAANyAPVZ9g2lNAAAAgxDOAAAADEI4AwAAMAjhDAAAwCCEMwAAAIMQzgAAAAxCOAMAADAI4QwAAMAgNKEFAKAvGt+Q9iyTZKVwcEAafas0pCzbo4KLEc4AAOiL99dLuxdJgaCST0i1S1ZUKhhLOENSTGsCANAXRTVSqNAOXlZbkkvUPq6oxukRw3CEMwAA+iJ0qnThPZICPRwYkC66VwoNzsWo4GJMawIA0FdjZkn1j0ttTYmPCYWlC2blbEi9xho6xxHOAADoq1j1bOd9ih9qXFQ1Yw2d45jWBAAgE8bMsqtj8bilaiaxhs4AhDMAADIh4dozF1XNJNbQGYBwBgBApsSrnrmpahaTrAoY48b75RKEMwAAMqVb1cml1aUeq2cuvV8uQTgDACCTOled3Fxd8soaOhcinAEAkEkdVSe5u7rklTV0LkQ4AwAg00pmS+WLpDG3Oz2SvvHKGjqXoc8ZAACZlj9IKrnN6VH0Xbf+bQ5XzXzSIJdwBsB7fPILHMiJzrsfOF0180mDXMIZAO/xyS9wICc6qmfznF9rVlQj7bz/422yosmPdXGDXNacAfAeOpwDmWXKGjqfNMglnAHwHp/8AgdyJraGLn+Q0yPxRYNcwhkAb/LBL3DAl3zQIJdwBsCbfPALHPAtjzfIJZwB8C6P/wIHfMvjDXIJZwC8y+O/wAFf83CDXMIZAG/z8C9wwNe8ssl8HIQzAN7m4V/ggO95ZZP5kxgTzpYsWaKioiINGDBA5eXlWr9+fdLjX331VZWXl2vAgAE6//zz9fTTT+dopABcx6O/wAHf88om8ycxIpytXr1ad955p+bPn6/t27fryiuv1OTJk7V37964x7/11lu67rrrdOWVV2r79u267777NHv2bL300ks5HjkAV/DoL3AAMqdBbgYFLMtKZfO5rJo4caIuvfRSLV26tOO6cePGacqUKVq4cGG34++55x798pe/1K5duzqumzlzpnbu3KnXX3897v9x9OhRHT16tOPzlpYWjRgxQs3NzSooKMjgvQFgpOMfSXuWS6NnmNFIE4DvtLS0KBwO95g9HK+cHTt2TNu2bdOkSZO6XD9p0iRt2rQp7te8/vrr3Y6/9tprtXXrVrW1tcX9moULFyocDndcRowYkZk7AMAdTOpwDgBJOB7ODh06pGg0qqFDh3a5fujQoTpw4EDcrzlw4EDc448fP65Dhw7F/Zp58+apubm547Jv377M3AEAAIAMynd6ADGBQNc+RJZldbuup+PjXR/Tv39/9e/fv4+jBAAAyC7HK2dnnHGGgsFgtyrZwYMHu1XHYs4+++y4x+fn5+v000/P2lgBAACyzfFw1q9fP5WXl6uurq7L9XV1daqsrIz7NZdffnm341955RVNmDBBoVAoa2MFAADINsfDmSTNnTtXy5Yt0/Lly7Vr1y7NmTNHe/fu1cyZMyXZ68Vqamo6jp85c6beeecdzZ07V7t27dLy5cv17LPP6q677nLqLgAAAGSEEWvOpk6dqsOHD+vhhx/W/v37NX78eK1bt06jRo2SJO3fv79Lz7OioiKtW7dOc+bM0eLFizV8+HA99dRT+vKXv+zUXQAAAMgII/qcOSHVXiMAAACZ4Jo+ZwAAADiBcAYAAGAQwhkAAIBBCGcAAAAGIZwBAAAYhHAGAABgEMIZAACAQQhnAAAABjFihwAnxHrvtrS0ODwSAADgB7HM0VP/f9+Gs9bWVknSiBEjHB4JAADwk9bWVoXD4YS3+3b7pvb2dr333ns69dRTFQgEMvI9W1paNGLECO3bt48toQzA82EWng/z8JyYhefDLNl4PizLUmtrq4YPH668vMQry3xbOcvLy9O5556ble9dUFDAD5ZBeD7MwvNhHp4Ts/B8mCXTz0eyilkMJwQAAAAYhHAGAABgEMJZBvXv318PPvig+vfv7/RQIJ4P0/B8mIfnxCw8H2Zx8vnw7QkBAAAAJqJyBgAAYBDCGQAAgEEIZwAAAAYhnAEAABiEcAYAAGAQwlmalixZoqKiIg0YMEDl5eVav3590uNfffVVlZeXa8CAATr//PP19NNP52ik/pDO81FbW6vPfvazOvPMM1VQUKDLL79c//u//5vD0Xpfuj8fMRs3blR+fr4uueSS7A7QZ9J9Po4ePar58+dr1KhR6t+/v0aPHq3ly5fnaLT+kO5zsmrVKl188cUaNGiQhg0bpltuuUWHDx/O0Wi967XXXtP111+v4cOHKxAIaO3atT1+TU7/nltI2c9//nMrFApZP/3pT636+nrrjjvusE455RTrnXfeiXv8m2++aQ0aNMi64447rPr6euunP/2pFQqFrBdffDHHI/emdJ+PO+64w3r88cet3//+99bu3butefPmWaFQyPrjH/+Y45F7U7rPR0xTU5N1/vnnW5MmTbIuvvji3AzWB3rzfNxwww3WxIkTrbq6Ouutt96ytmzZYm3cuDGHo/a2dJ+T9evXW3l5edaTTz5pvfnmm9b69eutiy66yJoyZUqOR+4969ats+bPn2+99NJLliRrzZo1SY/P9d9zwlkaLrvsMmvmzJldrhs7dqx17733xj3+O9/5jjV27Ngu1/3bv/2bVVFRkbUx+km6z0c8F154ofXQQw9lemi+1NvnY+rUqdb9999vPfjgg4SzDEr3+fj1r39thcNh6/Dhw7kYni+l+5z84Ac/sM4///wu1z311FPWueeem7Ux+lEq4SzXf8+Z1kzRsWPHtG3bNk2aNKnL9ZMmTdKmTZvifs3rr7/e7fhrr71WW7duVVtbW9bG6ge9eT5O1t7ertbWVp122mnZGKKv9Pb5eO6557Rnzx49+OCD2R6ir/Tm+fjlL3+pCRMm6Pvf/77OOeccjRkzRnfddZf+8Y9/5GLInteb56SyslLvvvuu1q1bJ8uy9Pe//10vvviiPv/5z+diyOgk13/P8zP+HT3q0KFDikajGjp0aJfrhw4dqgMHDsT9mgMHDsQ9/vjx4zp06JCGDRuWtfF6XW+ej5P96Ec/0ocffqgbb7wxG0P0ld48H3/961917733av369crP51dRJvXm+XjzzTe1YcMGDRgwQGvWrNGhQ4f0rW99Sx988AHrzjKgN89JZWWlVq1apalTp+qf//ynjh8/rhtuuEGLFi3KxZDRSa7/nlM5S1MgEOjyuWVZ3a7r6fh416N30n0+Yl544QUtWLBAq1ev1llnnZWt4flOqs9HNBrVzTffrIceekhjxozJ1fB8J52fj/b2dgUCAa1atUqXXXaZrrvuOv34xz/WihUrqJ5lUDrPSX19vWbPnq0HHnhA27Zt08svv6y33npLM2fOzMVQcZJc/j3n7WqKzjjjDAWDwW7vcA4ePNgtTcecffbZcY/Pz8/X6aefnrWx+kFvno+Y1atX6xvf+IZ+8Ytf6JprrsnmMH0j3eejtbVVW7du1fbt23XbbbdJssOBZVnKz8/XK6+8ok9/+tM5GbsX9ebnY9iwYTrnnHMUDoc7rhs3bpwsy9K7776rCy64IKtj9rrePCcLFy5UVVWV7r77bklSWVmZTjnlFF155ZV69NFHmX3JoVz/PadylqJ+/fqpvLxcdXV1Xa6vq6tTZWVl3K+5/PLLux3/yiuvaMKECQqFQlkbqx/05vmQ7IrZ9OnT9fzzz7NuI4PSfT4KCgoUiUS0Y8eOjsvMmTNVUlKiHTt2aOLEibkauif15uejqqpK7733no4cOdJx3e7du5WXl6dzzz03q+P1g948Jx999JHy8rr+mQ4Gg5JOVG2QGzn/e56V0ww8KnYa9LPPPmvV19dbd955p3XKKadYb7/9tmVZlnXvvfdaX//61zuOj516O2fOHKu+vt569tlnaaWRQek+H88//7yVn59vLV682Nq/f3/Hpampyam74CnpPh8n42zNzEr3+WhtbbXOPfdc6ytf+Yr15z//2Xr11VetCy64wLr11luduguek+5z8txzz1n5+fnWkiVLrD179lgbNmywJkyYYF122WVO3QXPaG1ttbZv325t377dkmT9+Mc/trZv397R1sTpv+eEszQtXrzYGjVqlNWvXz/r0ksvtV599dWO26ZNm2ZdddVVXY7/3e9+Z33iE5+w+vXrZ5133nnW0qVLczxib0vn+bjqqqssSd0u06ZNy/3APSrdn4/OCGeZl+7zsWvXLuuaa66xBg4caJ177rnW3LlzrY8++ijHo/a2dJ+Tp556yrrwwgutgQMHWsOGDbP+5V/+xXr33XdzPGrv+e1vf5v074HTf88DlkVtFAAAwBSsOQMAADAI4QwAAMAghDMAAACDEM4AAAAMQjgDAAAwCOEMAADAIIQzAAAAgxDOAAAADEI4AwAAMAjhDADi2L9/vwYPHqybbrqpy/W/+tWvFAqFNH/+fIdGBsDrCGcAEMewYcP0ne98R//1X/+lbdu2SZJ+97vf6atf/aq++c1v6rvf/a7DIwTgVYQzAEjgrrvu0rBhw3TPPffoD3/4g2644QZ97Wtf05NPPtlxzNKlS3XppZcqFAppwYIFzg0WgGcQzgAggUGDBunRRx/V//3f/+nqq6/W5MmT9dOf/lSBQKDjmGHDhumhhx7SlClTnBsoAE/Jd3oAAGCyMWPGSJICgYBWrFihYDDY5fZYKPvv//7vXA8NgEdROQOABHbs2KEvfOELqqqq0pEjR7R8+XKnhwTABwhnABDHX/7yF1177bW6/PLL9dvf/lZf/OIXtWDBAjU3Nzs9NAAeRzgDgJO8/fbbuuaaa1RSUqKXXnpJoVBIjz32mBobG/W9733P6eEB8DjCGQB0sn//fl1zzTU666yz9Ktf/UoDBw6UJI0dO1YzZszQk08+qbffftvZQQLwtIBlWZbTgwAAtzp+/LiOHz+ub37zmzrnnHN0//33KxQKdTtxAABSRTgDgD5YsGCBHnrooS7XPffcc5o+fbozAwLgeoQzAAAAg7DmDAAAwCCEMwAAAIMQzgAAAAxCOAMAADAI4QwAAMAghDMAAACDEM4AAAAMQjgDAAAwCOEMAADAIIQzAAAAgxDOAAAADPL/A/VmqD/F92+/AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 700x700 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize = (7,7))\n",
    "\n",
    "plt.scatter(X[y == 0,0],\n",
    "            X[y == 0,1],\n",
    "            c = \"blue\",\n",
    "            s = 60,\n",
    "            label=\"0\")\n",
    "plt.scatter(X[y == 1,0],\n",
    "            X[y == 1,1],\n",
    "            c = \"orange\",\n",
    "            s = 60,\n",
    "            marker = \"v\",\n",
    "            label=\"1\")\n",
    "\n",
    "plt.legend(fontsize = 12)\n",
    "plt.xlabel(\"$x_1$\",fontsize = 12)\n",
    "plt.ylabel(\"$x_2$\",fontsize = 12)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y,\n",
    "                                    shuffle=True,\n",
    "                                    random_state=435,\n",
    "                                    test_size=.3,\n",
    "                                    stratify=y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The weights are:\n",
      "[ 0.16985431 -0.13407521]\n",
      "The errors are:\n",
      "[7.5, 5.0, 4.5, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 3.5, 3.5, 5.0, 5.0, 5.0, 6.0]\n",
      "The accuracy is:\n",
      "0.8392857142857143\n"
     ]
    }
   ],
   "source": [
    "w, J, pred_train = perceptron(X_train, y_train, z = 0.0, eta = 0.1, num_it = 50)\n",
    "\n",
    "print(\"The weights are:\")\n",
    "print(w)\n",
    "\n",
    "print(\"The errors are:\")\n",
    "print(J)\n",
    "\n",
    "print(\"The accuracy is:\")\n",
    "print(accuracy(y_train, pred_train))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note here that our training accuracy is much lower in this case. Let's do more iterations and see if\n",
    "things improve."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The weights are:\n",
      "[ 0.15745427 -0.08794812]\n",
      "The errors are:\n",
      "[7.5, 5.0, 4.5, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 3.5, 3.5, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 3.5, 6.0, 3.5, 5.0, 5.0, 6.0, 3.5, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 6.0, 4.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 6.0, 4.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 2.5, 6.0, 4.5, 6.0, 3.5, 3.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 6.0, 4.5, 6.0, 3.5, 3.5, 5.0, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 3.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 6.0, 4.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 2.5, 6.0, 4.5, 6.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 6.0, 2.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 3.5, 3.5, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 3.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 6.0, 4.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 4.5, 5.0, 3.5, 6.0, 5.5, 3.5, 5.0, 3.5, 4.5, 6.0, 4.5, 6.0, 3.5, 6.0, 5.0, 3.5, 3.5, 6.0, 3.5, 5.0, 6.0, 4.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 2.5, 6.0, 4.5, 6.0, 3.5, 3.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 3.5, 3.5, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 6.0, 4.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 6.0, 4.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 2.5, 6.0, 4.5, 6.0, 3.5, 3.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 3.5, 3.5, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 3.5, 6.0, 3.5, 5.0, 5.0, 6.0, 3.5, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 3.5, 6.0, 4.5, 6.0, 2.5, 6.0, 4.5, 6.0, 3.5, 3.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 3.5, 3.5, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 3.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 6.0, 4.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 4.5, 5.0, 3.5, 6.0, 5.5, 3.5, 5.0, 3.5, 4.5, 6.0, 4.5, 6.0, 3.5, 6.0, 5.0, 3.5, 3.5, 6.0, 3.5, 5.0, 6.0, 4.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 2.5, 6.0, 4.5, 6.0, 3.5, 3.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 3.5, 3.5, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 3.5, 6.0, 3.5, 5.0, 5.0, 6.0, 3.5, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 6.0, 4.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 6.0, 4.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 5.0, 6.0, 3.5, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 6.0, 4.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 6.0, 4.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 5.0, 6.0, 3.5, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 6.0, 4.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 6.0, 4.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 4.5, 5.0, 3.5, 6.0, 5.5, 3.5, 5.0, 3.5, 4.5, 6.0, 4.5, 6.0, 3.5, 6.0, 4.5, 5.0, 3.5, 5.0, 3.5, 4.5, 6.0, 4.5, 6.0, 3.5, 6.0, 4.5, 5.0, 3.5, 6.0, 5.5, 3.5, 5.0, 3.5, 4.5, 6.0, 4.5, 5.0, 3.5, 6.0, 5.5, 3.5, 6.0, 5.5, 3.5, 5.0, 3.5, 4.5, 6.0, 2.5, 6.0, 4.5, 6.0, 3.5, 3.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 6.0, 2.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 7.0, 3.5, 3.5, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 3.5, 3.5, 5.0, 5.0, 5.0, 6.0, 4.5, 6.0, 3.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 3.5, 5.0, 6.0, 4.5, 6.0, 3.5, 5.0, 5.0, 5.0, 5.0]\n",
      "The accuracy is:\n",
      "0.875\n"
     ]
    }
   ],
   "source": [
    "w, J, pred_train = perceptron(X_train, y_train, z = 0.0, eta = 0.1, num_it = 1000)\n",
    "\n",
    "print(\"The weights are:\")\n",
    "print(w)\n",
    "\n",
    "print(\"The errors are:\")\n",
    "print(J)\n",
    "\n",
    "print(\"The accuracy is:\")\n",
    "print(accuracy(y_train, pred_train))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Test the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The accuracy is:\n",
      "0.7916666666666666\n"
     ]
    }
   ],
   "source": [
    "y_pred = perceptron_test(X_test, w, z = 0.0)\n",
    "\n",
    "print(\"The accuracy is:\")\n",
    "print(accuracy(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is actually pretty good, given how small this data set is. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Iris Dataset\n",
    "\n",
    "Let's try this out on the benchmark Iris data set, which is described in detail at:\n",
    "https://archive.ics.uci.edu/dataset/53/iris\n",
    "\n",
    "Each observation represents an iris (a type of flower) and gives it's measurements including:\n",
    "- `sepal_length`: the length of the iris's sepal in cm.\n",
    "- `sepal_width`: the width of the iris's sepal in cm.\n",
    "- `petal_length`: the length of the iris's petal in cm.\n",
    "- `petal_width`: the width of the iris's petal in cm.\n",
    "- `iris_class`: the class of the iris, can be:\n",
    "    - `0` meaning it is a setosa iris\n",
    "    - `1` meaning it is a versicolor iris\n",
    "    - `2` meaning it is a virginica iris"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "## to get the iris data\n",
    "from sklearn.datasets import load_iris\n",
    "\n",
    "## Load the data\n",
    "iris = load_iris()\n",
    "iris_df = pd.DataFrame(iris['data'], columns = ['sepal_length','sepal_width','petal_length','petal_width'])\n",
    "iris_df['iris_class'] = iris['target']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal_length</th>\n",
       "      <th>sepal_width</th>\n",
       "      <th>petal_length</th>\n",
       "      <th>petal_width</th>\n",
       "      <th>iris_class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   sepal_length  sepal_width  petal_length  petal_width  iris_class\n",
       "0           5.1          3.5           1.4          0.2           0\n",
       "1           4.9          3.0           1.4          0.2           0\n",
       "2           4.7          3.2           1.3          0.2           0\n",
       "3           4.6          3.1           1.5          0.2           0\n",
       "4           5.0          3.6           1.4          0.2           0"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "iris_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Making a statified train test split\n",
    "iris_train, iris_test = train_test_split(iris_df, \n",
    "                                            random_state=431,\n",
    "                                            shuffle=True,\n",
    "                                            test_size=.2,\n",
    "                                            stratify=iris_df['iris_class'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAroAAAIOCAYAAABXpq56AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAABx30lEQVR4nO3dfXxT5d0/8E+ahPLUpkh5KIZACxoUiyIgLQVqb6wo4kQY92S6tjh3i/Kg8mNI1SmoE+bUMZTp8IG2Op2OQkXUaba1hZbCQFGqYqAtkAZBLLZpEYQ2ze+PY0LT5rk5yTnJ5/165eV6cnL67ckp/ezKda6vwmaz2UBEREREFGFiwl0AEREREZEYGHSJiIiIKCIx6BIRERFRRGLQJSIiIqKIxKBLRERERBGJQZeIiIiIIhKDLhERERFFJAZdIiIiIopIqnAXEA7t7e345ptvEBcXB4VCEe5yiIiIiKgTm82GlpYWDBkyBDExgY3NRmXQ/eabbzB06NBwl0FEREREXtTX10Or1Qb02qgMunFxcQCEExcfHx/maoiIiIios+bmZgwdOtSR2wIRlUHXPl0hPj6eQZeIiIhIwrozzZQ3oxERERFRRGLQJSIiIqKIFPag29bWhkceeQTJycno1asXUlJS8Pjjj6O9vd3j68rLyzFu3Dj07NkTKSkpeOmll0JUMRERERHJQdjn6P7hD3/ASy+9hMLCQowePRp79+7F/PnzodFocN9997l8zeHDhzFjxgz85je/wRtvvIHKykrce++9GDBgAObMmRPin4CIiIiIpCjsQbeqqgq33HILbrrpJgDA8OHD8dZbb2Hv3r1uX/PSSy9Bp9Nh7dq1AIDLLrsMe/fuxTPPPMOgS0RERJJks9lgtVrR1tYW7lLCTq1WQ6lUiv59wh50J0+ejJdeegkHDx7EpZdeis8//xwVFRWOEOtKVVUVrr/+eqdt06dPx6uvvorW1lao1Wqn586dO4dz5845vm5ubg7qz0BERETkjs1mQ1NTE7777jtYrdZwlyMZCQkJGDx4sKjNu8IedB988EFYLBaMGjUKSqUSVqsVv//97zFv3jy3rzlx4gQGDRrktG3QoEFoa2tDQ0MDkpKSnJ5bvXo1Vq1aJUr9RERERJ6cOHECTU1NjmVNVSpVVHdmtdlsOHPmDE6ePAkAXXJbMIU96L799tt444038Oabb2L06NH47LPPcP/992PIkCHIzc11+7rOF4jNZnO5HQDy8/OxdOlSx9f2BYiJiIiIxGS1WmGxWDBgwAAkJiaGuxzJ6NWrFwDg5MmTGDhwoGjTGMIedH/7299ixYoVuO222wAAqampOHr0KFavXu026A4ePBgnTpxw2nby5EmoVCr079+/y/6xsbGIjY0NfvFEREREHrS2tsJms6FPnz7hLkVyevfuDUA4R2IF3bAvL3bmzBnExDiXoVQqPS4vlp6eDoPB4LTt448/xvjx47vMzyUiIiIKt2iequBOKM5J2IPuzTffjN///vd4//33ceTIEWzZsgXPPfccbr31Vsc++fn5yMnJcXy9YMECHD16FEuXLsWBAwfw2muv4dVXX8WyZcvC8SMQERERkQSFferC888/j9/97ne49957cfLkSQwZMgR33303Hn30Ucc+x48fh8lkcnydnJyMDz74AA888ADWr1+PIUOGYN26dVxajIiIiIgcFDb7XVxRpLm5GRqNBhaLBfHx8eEuh4iISHSGWgP0iXroNDq3+5gsJhgbjMgekR3CyiLbjz/+iMOHDyM5ORk9e/YMdzmiOH36NB555BG88847+P777zFq1Cin+6/c8XZugpHXwj6iS0REROIy1Bow862Z0MZrUZpb6jLsmiwmZBVmwdxsxrZ52xh2ZaC6Gti8GWhqAhISgNmzgdTU0Ncxe/Zs7NmzB2vWrMGll16KN998E/PmzUN7ezt++ctfhr6gDhh0iYiIIpw+UQ9tvBZ1jXXIKszqEnbtIbeusQ4p/VKgT9SHsVrypqYGyM0Fdu4ElEogJgZobwdWrgQyMoCCAmDkyNDU8sEHH8BgMDjCLQBkZWXh6NGj+O1vf4tf/OIXIemA5k7Yb0YjIiIicek0OpTmliKlX4oj7Joswr0vnUOuuxFfkoaaGmDiRGD3buFrqxVobRX+CwC7dgnP19SEpp4tW7agb9++mDt3rtP2+fPn45tvvsFue6FhwqBLREQUBVyF3UpTJUOuzOTmAhbLhWDbmdUqPJ+XF5p6vvjiC1x22WVQqZwnCYwZM8bxfDgx6BIREUWJzmF38sbJDLkyUl0tTFdwF3LtrFagslLYX2ynTp3CRRdd1GW7fdupU6fEL8IDBl0iIqIootPoUDSryGlb0awihlwZ2LxZmJPrC6US2LJF3HrsPDV+CHejDAZdIiKiKGKymJBTkuO0LackxzFnl6SrqUm48cwXMTFAY6Oo5QAA+vfv73LU9vvvvwcAl6O9ocSgS0REFCU633hWMb/C5Q1qJE0JCcLqCr5obwf69RO1HABAamoqDhw4gLa2Nqft1T/Nm7jiiivEL8IDBl0iIqIo4Gp1hQxdhtvVGEh6Zs/2Pj/XzmoV9hfbrbfeitOnT6O4uNhpe2FhIYYMGYKJEyeKX4QHDLpEREQRztMSYp6WHiNpSU0FJk3yPk9XqRTW0w3FYOqNN96I7Oxs3HPPPXj55ZdRWlqK//u//8M///lPPP3002FdQxdg0CUiIop4xgYjzM1mt6srdAy75mYzjA3GMFVK3hQWAhqN+7CrVArPFxSErqbNmzfjV7/6FR599FHccMMN2L17N9566y3cfvvtoSvCDYXNZrOFu4hQC0bvZCIiIjkx1BqgT9R7XF3BZDHB2GBk+98g+vHHH3H48GEkJyejZ8+eQTlmTY2wTm5lpXNnNKs19J3RusPbuQlGXmMLYCIioijgS3jVaXRcZkwGRo4EKiqEdXK3bBFWV+jXT5iTG+Z7vySHQZeIiIhIhlJThQe5xzm6RERERBSRGHSJiIiIKCIx6BIRERFRRGLQJSIiIqKIxKBLRERERBGJQZeIiIiIIhKDLhERERFFJAZdIiIiIopIDLpEEmCoNcBkMXncx2QxwVBrCFFFvmPtRPLA653E0tLSguXLl+P666/HgAEDoFAosHLlynCXBYBBlyjsDLUGzHxrJrIKs9z+ETJZTMgqzMLMt2ZK6o8QayeSB17vJKZTp05hw4YNOHfuHGbNmhXucpywBTBRmOkT9dDGa1HXWIeswiyU5pY69Zq3//Gpa6xDSr8U6BP1YazWGWsnkgde7xGicT9Q+woAmw87K4ARdwH9xohdFYYNG4bGxkYoFAo0NDTglVdeEf17+oojukRhptPoUJpbipR+KY4/QvYRl85/fDr/cQo31k4kD7zeI8R3O4CDzwOHXgQO/dXD40Vhv+8qQlKWQqGAQqEIyffyF4MukQS4+iNUaaqUxR8f1k4kD7zeI0ByDqBOAGxWwNbq4WEV9kvOCXfFYcegSyQRnf8ITd44WTZ/fFg7kTzwepc5dRxw+YMAvI2eKoDRKwB131BUJWkMukQSotPoUDSryGlb0awiWfzxYe1E8sDrXeYuXQioNZ73UWuASxaGph6JY9AlkhCTxYScEuePmnJKcrwuCSQFrJ1IHni9y5zXUV2O5nbEoEskEZ1vCKmYX+HyxhEpYu1E8sDrPUJ4GtXlaK4TBl0iCXB113OGLsPtXdJSwtqJ5IHXewRxO6rL0dzOGHSJwszT0j6elgSSAtZOJA+83iOQq1HdMI7mfvjhh9i0aRPee+89AMBXX32FTZs2YdOmTThz5kxYagIYdInCzthghLnZ7Pau545/hMzNZhgbjGGqtCvWTiQPvN4jUJdR3fCO5t5zzz2YO3cu7rzzTgDAP/7xD8ydOxdz587FyZMnw1ITAChsNpsv7TUiSnNzMzQaDSwWC+Lj48NdDhEMtQboE/Ue73o2WUwwNhiRPSI7hJV5J+fa11SsQeawTKQPTXe7T1V9FcqPlmPF5BUhrIwo+OT8uypnP/74Iw4fPozk5GT07NkzuAdvbQFKdEBrk7Bu7qx6WU1b8HZugpHX2AKYSAJ8+aOi0+gkufyPXGs31BrwWNlj0MZr3a4farKYcMeWO2BuNmNc0jj+8SdZk+vvKnlgH9X9PJ9zc93g1AUiikr6RD208Vq3cxI7zmnUxmuhT9SHqVIiIg/0S4BxzwOXLg53JZLEoEtEUcnTDTiebtwhIpIUVW9Av0j4L3XBoEtEUctV2K00VTLkEhFFCAZdIopqncPu5I2TGXKJiCIEgy4RRT2dRoeiWUVO24pmFTHkEhHJHIMuEUU9k8WEnJIcp205JTlcNJ+ISOYYdIkoqnW+8axifgU7RBERRQgGXSKKWq5WV8jQZbAdKhFRhGDQJaKo5GkJMU9LjxERkXww6BJRVDI2GGFuNrtdXaFj2DU3m2FsMIapUiIiChSDLhFFpewR2dg2b5vHJcTsYXfbvG1s/0tE5MJ//vMf3HnnnRg1ahT69OmDiy++GLfccgs++eSTcJcGQAJBd/jw4VAoFF0eCxcudLl/WVmZy/2//vrrEFdORHKXPSLb6xJiOo2OIZeIJMNQa/A6lcpkMcFQawhJPS+++CKOHDmC++67Dx988AH+/Oc/4+TJk0hLS8N//vOfkNTgiSrcBezZswdWq9Xx9RdffIHs7GzMnTvX4+uMRiPi4+MdXw8YMEC0GomIiIjCzVBrwMy3ZkIbr3X7aZT9/gNzszkkn0atX78eAwcOdNp2ww03YOTIkXjqqafwP//zP6J+f2/CHnQ7B9Q1a9ZgxIgRyMzM9Pi6gQMHIiEhQcTKiIiIiKRDn6iHNl7ruEm2c9jtfJOtPlEvek2dQy4A9O3bF5dffjnq6+tF//7ehH3qQkfnz5/HG2+8gTvvvBMKhcLjvmPHjkVSUhKmTZuG0tJSj/ueO3cOzc3NTg8iIiIiOfG0IoynlWRCzWKx4NNPP8Xo0aPD8v07klTQLSkpQVNTE/Ly8tzuk5SUhA0bNqC4uBibN2+GXq/HtGnTsH37drevWb16NTQajeMxdOhQEaonIiIiEpersFtpqpRMyAWAhQsX4ocffsDDDz8cthrsFDabzRbuIuymT5+OHj164L333vPrdTfffDMUCgW2bt3q8vlz587h3Llzjq+bm5sxdOhQWCwWp3m+RERERMH0448/4vDhw0hOTkbPnj2DdtyOI7h2Ugi5v/vd7/Dkk0/i+eefx6JFizzu6+3cNDc3Q6PRdCuvSWZE9+jRo/jXv/6Fu+66y+/XpqWl4dChQ26fj42NRXx8vNODiIiISK50Gh2KZhU5bSuaVRTWkLtq1So8+eST+P3vf+815IaKZILuxo0bMXDgQNx0001+v3bfvn1ISkoSoSoiIiIi6TFZTMgpyXHallOSE7YujqtWrcLKlSuxcuVKPPTQQ2GpwRVJBN329nZs3LgRubm5UKmcF4LIz89HTs6FN3Lt2rUoKSnBoUOH8OWXXyI/Px/FxcWS+X8ORERERGLqfONZxfyKsLYsf+KJJ7By5Uo88sgjeOyxx0L6vb0J+/JiAPCvf/0LJpMJd955Z5fnjh8/DpPpwht2/vx5LFu2DMeOHUOvXr0wevRovP/++5gxY0YoSyYiIiIKOXerK5Tmljq2u1p6TCzPPvssHn30Udxwww246aabsGvXLqfn09LSRK/BE0ndjBYqwZjcTEQCQ60B+kS9x39QTRYTjA1GdhgjoqgTzJvRvC0hFo4lxq699lqUl5e7fd5TzIyqm9GISH7sXXo8fVRm/4d35lszQ9aSkogoEhkbjDA3m92G2I5Lj5mbzTA2GEWvqaysDDabze0j3CQxdYGI5EmKXXqIiCJV9ohsbJu3zeOnaPawy0/RBBzRJaKAyaVLDxFRpMgeke3131KdRseQ+xMGXSLqFjl06SEioujEoEtE3dY57E7eOJkhl4iIwo5Bl4iCQopdeoiIKLox6BJRUEitSw8RkZRIYQUCqQnFOWHQJaJuk1qXHiIiqVCr1VAoFPjhhx/CXYrknDlzBoBwjsTC5cWIqFuk1qWHiEhKlEolNBoNvvvuO5w7dw7x8fFQqVRQKBThLi1sbDYbzpw5g5MnTyIhIQFKpVK078XOaOyMRhQwKXbpISKSGpvNBovFgpMnT8JqtYa7HMlISEjA4MGD3Yb+YOQ1jugSUcB87dKTVZjl6NLDoEtE0UahUCAhIQEajQZWqxVtbW3hLins1Gq1qCO5dhzR5YguUbcYag0eu/QAwshuoF16xD4+ERFJE0d0iSjsfAmXOo0uoJFcQ60BM9+aCW281u20B/v0CHOzGdvmbWPYJSIiB666QESSpU/UQxuvdbt6Q8c5wNp4LfSJ+jBVSkREUsSgS0SS5aq9sD3s8kY3IiLyhkGXiCTNVditNFUy5BIRkVe8GY03oxHJQscRXDuGXCKiyBWMvMYRXSKSBZ1Gh6JZRU7bimYVMeQSEZFbDLpEJAsmiwk5JTlO23JKcthemIiI3GLQJSLJ63zjWcX8Cpc3qBEREXXEoEtEkuZqdYUMXYbb1RiIiIjsGHSJSLI8LSHmaekxIiIigEGXiCTM2GCEudnsdnWFjmHX3GyGscHo1/ENtQav4dhkMcFQa/C7drHJuXY543knkhcGXSKSrOwR2dg2b5vHJcTsYdff9r/29sKeRoLtI8oz35opqeAi59rljOedSH4YdIlI0rJHZHtdQkyn0fkVcgF5txeWc+1yxvNOJD8MukQUleTcXljOtcsZzzuR/DDoElHUknN7YTnXLmc870TywhbAbAFMFPXk3F5YzrXLGc87kfjYApiIKAjk3F5YzrXLGc87kTww6BJR1JNze2E51y5nPO9E8sCgS0RRTc7theVcu5zxvBPJB4MuEUUtObcXlnPtcsbzTiQvDLpEFJXk3F5YzrXLGc87kfww6BJRVBK7vbCY5Fy7nPG8E8kPlxfj8mJEUWtNxRpkDstE+tB0t/tU1Veh/Gg5VkxeEcLKvDPUGqBP1Hu8y99kMcHYYPS7axy5x/NOFDrByGsMugy6RFHJUGvAzLdmQhuvdbv2qf2janOzGdvmbWNwISIKIa6jS0QUIH2iHtp4rdv5lB3nY2rjtdAn6sNUKRERBYpBl4iikqebhzzddERERPLBoEtEUctV2K00VTLkEhFFCM7R5RxdoqjXcQTXjiGXiCi8OEeXiCgIdBodimYVOW0rmlXEkEtEJHMMukQU9UwWE3JKcpy25ZTkcMF/IiKZY9AloqjW+cazivkV7G5FRBQhGHSJKGq5Wl0hQ5fBVq5ERBGCQZeIopKnJcQ8LT1GRETywaBLRFHJ2GCEudnsdnWFjmHX3GyGscEYpkqJiChQXF6My4sRRS1DrQH6RL3H1RVMFhOMDUZptf9t3A/UvgLAl3++FcCIu4B+Y8SuiogoqIKR11RBromISDZ8Ca86jU56y4x9twM4+DygUMLzB3PtgM0KxI9i0CWiqBT2qQvDhw+HQqHo8li4cKHb15SXl2PcuHHo2bMnUlJS8NJLL4WwYiKiMEvOAdQJQoi1tXp4WIX9knO8HZGIKCKFPeju2bMHx48fdzwMBgMAYO7cuS73P3z4MGbMmIEpU6Zg3759eOihh7BkyRIUFxeHsmwiovBRxwGXPwhA4WVHBTB6BaDuG4qqiIgkR3JzdO+//35s27YNhw4dgkLR9R/xBx98EFu3bsWBAwcc2xYsWIDPP/8cVVVVPn0PztElItlrbQFKdEBrk/t91AnArHoGXSKSpYhrAXz+/Hm88cYbuPPOO12GXACoqqrC9ddf77Rt+vTp2Lt3L1pbW12+5ty5c2hubnZ6EBHJmtdRXY7mEhFJKuiWlJSgqakJeXl5bvc5ceIEBg0a5LRt0KBBaGtrQ0NDg8vXrF69GhqNxvEYOnRoMMsmIgqPSxcCao3r59Qa4BL39zoQEUUDSQXdV199FTfeeCOGDBnicb/Oo7322RfuRoHz8/NhsVgcj/r6+uAUTEQUTm5HdTmaS0QESGh5saNHj+Jf//oXNm/e7HG/wYMH48SJE07bTp48CZVKhf79+7t8TWxsLGJjY4NWKxGRZFy6EPjqD85zdTmaS0QEQEIjuhs3bsTAgQNx0003edwvPT3dsTKD3ccff4zx48dDrVaLWSIRkfR0GdXlaC4RkZ0kgm57ezs2btyI3NxcqFTOg8z5+fnIybmwBuSCBQtw9OhRLF26FAcOHMBrr72GV199FcuWLQt12URE0tBxri5Hc4mIHCQxdeFf//oXTCYT7rzzzi7PHT9+HCaTyfF1cnIyPvjgAzzwwANYv349hgwZgnXr1mHOnDmhLJmIyLPG/TDsehT6vgnQ9Ypzu5vpbAuMp5uQnfa4X93LnNoX20d1P893Gs2VZPtiIqIQktw6uqHAdXSJSGyG8kWYWb4eWhVQqlVBp+56s6yp1YYscxvMbcC2zEXIznzet2PXGjDzrZnQxmtRmlsqhN22M0Dta8CIOwFVb5gsJmQVZsHcbMa2edsYdolIdiJuHV0iokihH30vtKoY1LUCWeY2mM47t+c1nW9FlrkNda2AVhUD/eh7fT92oh7aeC3qGuuQVZgFk8UEqHoD+kVOIbeusQ7aeC30iXoRf1IiIuli0CUiEoEu8XKUZv8WKWoIYfcYYPqpp43pp6/rWoEUNVB6/XLoEi/z/dgaHUpzS5HSL8U57AJOITelX8qFEV8ioijEoEtEJBLdVQ+jdHi8U9itPNsp5A6Ph+7Kh/0/touwW2mqZMglIuqAQZeISCzqOOiuzEfpxXCE3cnmDiH3YkB31UMBLwXWOexO3jiZIZeIqAMGXSIiMV26ELreCShy7lyOokGArndCt5cC02l0KJpV5HzsWUUMuUREYNAlIhKXOg6m4Xcj51vnzTnfAqbkBd1u7GCymJBTkuO0LackxzFnl4gomjHoEhGJyGQxIWvn247pChXaC9MYsirf7lYg7XzjWcX8Cpc3qBERRSsGXSIikTiCaNMRpPS5CKUXAxm9hLm5KX0uQl3T4YADqavVFTJ0GW5XYyAiikYMukREIugSRPN2CHNyIczNLc0LfPTV0xJinpYeIyKKNgy6REQiMDYYYW42XwiiiZcLbXoBYPQK6BIvcwRSc7MZxgZj4MfudONZx7Dr77GJiCIJWwCzBTARicRQa4A+UX8hiHZq0wsIo7PGBqPfLXrXVKxB5rBMpA9Nd7tPVX0Vyo+WY8XkFQH/DERE4RKMvMagy6BLRDJjqDVg5lszoY3Xul0v1z69wdxsxrZ52/wO0kRE4RaMvMapC0REMqNP1EMbr3U7B7fjHF5tvBb6RH2YKiUiCi8GXSIimfF0w5mnG9WIiKINgy4RkQy5CruVpkqGXCKiDjhHl3N0iUjGOo7g2jHkElEk4BxdIqIop9PoUDSryGlb0awihlwiIjDoEhHJmsliQk5JjtO2nJIcNokgIgKDLhGRbHW+8axifuDd1oiIIhGDLhGRDLlaXSFDl8H2v0REHTDoEhHJjKclxDwtPUZEFG0YdImIZMbYYIS52ex2dYWOYdfcbIaxwRimSomIwovLi3F5MfKRodYAfaLe493sJosJxgYj263KQeN+GHY9Cn3fBOh6xbndzXS2BcbTTchOexzoN8b34x/dBHz5BABf/olVAKN/Bwz7uc+HF/t6XFOxBpnDMpE+NN3tPlX1VSg/Wo4Vk1f4fXwiIm+CkddUQa6JKCIZag2Y+dZMaOO1btcntX+cbG42Y9u8bQy7EmfYvwEz97wLrQoo1aqgUyu67GNqtSHL3AZzG7Ct91BkZz7v+zeofwdo2u/H/pv8Crq+XF86jS6gZcbWVKxB/r/zoYpRYXvedpdht6q+ClMLpqKtvQ0AGHaJSJI4dYHIB/pEPbTxWrdzHjvOmdTGa6FP1IepUvKVfvS90KpiUNcKZJnbYDrfCtguPEznW5FlbkNdK6BVxUA/+l7/vsHVa/3bf5yf+4soc1gmVDEqtLW3YWrBVFTVVzk93zHkqmJUyByWGaZKiYg8Y9Al8oGnG3w83RhE0qVLvByl2b9FihpC2D0GmFqF50w/fV3XCqSogdLrl0OXeJl/36D3EGDwdN/2TboB6DXYv+OLKH1oOrbnbXcZdjuHXHcjvkREUsCgS+QjV2G30lTJkCtjuqseRunweKewW3m2U8gdHg/dlQ8H9g3SXvNxv42BHV9ErsLui3teZMglIlnhzWi8GY381HEE144hV8a+XAPT3nxHuLVLUQOlFwO6CWuAyx8M/Pj/uQE48ZH755NuALI+DPz4Ius8FxcAQy4RhUQw8hpHdIn8pNPoUDSryGlb0awihly5unQhdL0TUDTIeXPRIEDXOwG4ZGH3ju9tVFeCo7kdpQ9Nx7ob1jltW3fDOoZcIpIFBl0iP5ksJuSU5DhtyynJ4aL8cqWOg2n43cj51nlzzreAKXkBoO7bveN7mqsrsbm5rlTVV2HJP5c4bVvyzyVdblAjIpIiBl0iP3S+8axifgU7UMmcyWJC1s63HXNyK7S4MGe38u3gvKfuRnUlPprb+cazv8z4i8fVGIiIpIZBl8hHrlZXyNBlsN2qjDne06YjSOlzEUovBjJ6CXNzU/pchLqmw8F5T12N6kp8NNfV6gr3TLjH7WoMRERSxKBL5ANPS4h5WnqMpKvLe5q3Q5iTC2FubmlekEfrO4/qSng019MSYp6WHiMikhp2RiPygbHBCHOz2e3qCvawa++MZmww8ua0YGjcD9S+Ap/b6I64y+c2vcYjH8JsOYKU3vEoHTsFuiMvAprLgIYqQHMZdEf+gtKxU5C1qwFmyxEYj3wI3ZV3B/6z2Ed1T3wk+dHc8qPlHpcQs4ddexguP1rOm9OISJK4vBiXFyMfGWoN0CfqPQZYk8UEY4OR7X+D5eB6YO8iQKGE5w+g2gGbFRi/HrjUxw5mB9fDUL4I+tgY6NRKYZvNJhwLMYBCaAlsarXCeK4d2Zl+HNudHxuAPfcAE14EeiZ271giW1OxBpnDMj0G2Kr6KpQfLWf7XyISRTDyGoMugy6RdLW2ACU6oLXJ+77qBGBWve+rJIh5bCIi6jauo0tEkU0d91OzBoWXHRXA6BX+BVExj01ERJLAoEtE0nbpQkCt8byPWhNYYwcxj01ERGHHoEtE0uZ15LUbI65iHpuIiMKOQZeIpM/TyGt3R1zFPDYREYUVgy4RSZ/bkdcgjLiKeWwiIgorBl0ikgdXI6/BGnEV89hERBQ2DLpEJA9dRl6DOOIq5rGJiChsGHSJSD46jrwGe8RVzGMTEVFYMOgSkXw4Rl4R/BFXMY9NRERhwaBLRPKiXwKMex64dHG3D2WoNcBkMXk8tsligqHW0O3vFWxdandBqrUTEYWKKpAXVVRU4M0338TRo0dx9uxZp+cUCgX+/e9/B6U4IqIuVL0B/aJuH8ZQa8DMt2ZCG69FaW4pdBpdl2ObLCZkFWbB3GzGtnnbkD0iu9vfNxhc1t6JVGsnIgolv0d0N27ciKlTp+Kdd95BY2MjbDab06O9vd3vIo4dO4Y77rgD/fv3R+/evXHVVVfhk08+cbt/WVkZFApFl8fXX3/t9/cmouikT9RDG69FXWMdsgqzuoyO2oNiXWMdtPFa6BP1Yaq0KznXTkQUSgqbzWbz5wWXXXYZrrzyShQWFiI2NrbbBTQ2NmLs2LHIysrCPffcg4EDB6K2thbDhw/HiBEjXL6mrKwMWVlZMBqNiI+Pd2wfMGAAlEql1+/Z3NwMjUYDi8Xi9Hoiii4dA2FKvxTH6Ki77VIi59qJiHwRjLzmd9Dt3bs3tm7diuuuuy6gb9jZihUrUFlZiR07dvj8GnvQbWxsREJCgt/fk0GXiOw6B8OiWUXIKcmRRVCUc+1ERN4EI6/5PXXhsssuw7fffhvQN3Nl69atGD9+PObOnYuBAwdi7NixePnll3167dixY5GUlIRp06ahtLTU7X7nzp1Dc3Oz04OICAB0Gh1Kc0uR0i8FdY11mLxxsmyCopxrJyIKBb+D7lNPPYU1a9bg2LFjQSmgrq4OL774Ii655BJ89NFHWLBgAZYsWYKioiK3r0lKSsKGDRtQXFyMzZs3Q6/XY9q0adi+fbvL/VevXg2NRuN4DB06NCi1E1Fk0Gl0KJrl/G9O0awiWQRFOddORCQ2n6Yu/OxnP3P6+rPPPsOpU6dw1VVXoX///s4HVCjw7rvv+lxAjx49MH78eOzcudOxbcmSJdizZw+qqqp8Ps7NN98MhUKBrVu3dnnu3LlzOHfunOPr5uZmDB06lFMXiAiA8xQAO7mMisq5diIiT0I2dWH//v2orq52PJRKJQYOHIhvvvnGaXt1dTX279/vVwFJSUm4/PLLnbZddtllMJk8rw/ZWVpaGg4dOuTyudjYWMTHxzs9iIiArvNcK+ZXOKYCuFrRQErkXDsRUSj4tI7ukSNHRCsgIyMDRqPRadvBgwcxbNgwv46zb98+JCUlBbM0Iopw7lYoKM0tdWzPKsyS5OionGsnIgoVv+fobt++HadPn3b53A8//OB2nqw7DzzwAHbt2oWnnnoKNTU1ePPNN7FhwwYsXHihz3x+fj5ycnIcX69duxYlJSU4dOgQvvzyS+Tn56O4uBiLFnV/EXkiig6eluHqfJOX1EZH5Vw7EVEo+R10s7Ky8NVXX7l87uuvv0ZWVpZfx5swYQK2bNmCt956C1dccQWeeOIJrF27Frfffrtjn+PHjztNZTh//jyWLVuGMWPGYMqUKaioqMD777+P2bNn+/vjEFGUMjYYYW42u53P2jEwmpvNMDYY3Rwp9ORcOxFRKPm9jm5MTAx27dqFa665pstzu3btwpQpU9Da2hq0AsXAdXQp2hhqDdAn6j1+hG2ymGBsMPrdKnZNxRpkDstE+tB0t/tU1Veh/Gg5Vkxe4dexxSbmeRH1+I37Ydj1KPR9E6DrFef+2GdbYDzdhOy0x4F+Y/wpnYgo7IKR13yao9vc3IympibH1ydOnOhys9jZs2dRWFiIwYMHB1QIEYnDUGvAzLdmQhuvdTtf0/5RuLnZjG3ztvkcutZUrEH+v/OhilFhe952l2G3qr4KUwumoq29DQAkFXZ9+Tl1Gl1Ac1zFPO/4bgeyG98FmpTw9MGcDu3Q2azAd9cz6BJRVPJp6sKf/vQnJCcnIzk5GQqFArfeeqvja/vj8ssvx1//+lfk5uaKXTMR+UGfqIc2Xut2vmbH+Z7aeC30iXqfj505LBOqGBXa2tswtWAqquqdlwTsGHJVMSpkDssMys8kB2KedyTnAOoEwGYFbK0eHlZhv+Qcb0ckIopIPk1dqKqqws6dO2Gz2bB8+XIsXrwYOp3z6ERsbCxSU1ORmSn9P2ScukDRxt3NS55uavJV5zBrH9l1tz2aiHne8eUa4POHAHj6J1wBXLUauPzB7vwYRERhEYy85vcc3VWrVuE3v/kNhgwZEtA3lAIGXYpGncNV0awi5JTkBKVlbOdQu+6GdVjyzyVRHXLtRDvvrS1AiQ5obXK/jzoBmFUPqPsGWj4RUdiEJehGAgZdilZidtHqPBcXQNSHXDvRzrvHUV2O5hKRvIUl6D7++ONun4uJiUFCQgLGjx+PtLS0gAoKBQZdimaVpkpM3jjZ8XXF/Apk6DKCcuwX97yIez+41/H1X2b8BfdMuCcox5Y7Uc67p1FdjuYSkcyFJejGxMRAoVDA1cvs2xUKBTIzM7F161b07Su9f2QZdClacUQ3PMQ8765HdTmaS0TyF4y85nfDiNraWowcORKrV6/GkSNHcPbsWRw+fBhPPfUURowYgd27d+P111/HJ598gt/97ncBFUVEwdd5rmjF/Iqgdc/qPEf3LzP+4nE1hmgi5nkHAFy6EFBrnLepNcAlC13vT0QURfwe0b355psxceJEPPLII12ee/LJJ1FZWYkPP/wQTzzxBF599VUcOXIkWLUGDUd0Kdpw1YXwEHXVhY6cRnU5mktEkSEsI7plZWVIT3f9xyo9PR0VFRWO/338+PGAiiKi4PEUqjq2ig1khNFTmE0fmo7tedujdmRXzPPeRcdRXY7mEhE5+B10e/TogX379rl87pNPPkGPHj0AAO3t7ejTp0/3qiOibjM2GGGymBzhqvPIYcft9na0vio/Wu5xxLZz2C0/Wu53/YZag9cQaLKYYKg1+H1sMRkbjDA3m92O2HYMu+Zms1/nvQt13IUR3NErun8DWuN+GD6cBdOOPGDvYrcP0448GD6cBTTu7973IyISiU8tgDuaNWsWHnvsMWg0GsydOxcJCQloamrC22+/jccffxy33XYbAKC6uhojR44MesFE5D8FFEHdz25c0jjEKGIwsM9AXBx/sct9Lo6/GAP7DMSJ0ycwLmmcX8cXtY2uyLJHZGPbvG3QJ+rdTkuwh11jg7H7deuXAKq+wIg7u3ccAIb9GzBzz7vQqoBSrQo6ddfrwtRqQ5a5DeY2YFvvocjOfL7b35eIKNj8HtF97rnncPXVV+Puu+9G//79ERsbi/79++Oee+7B2LFj8eyzzwIALr74Yjz22GNBL5iI/KNP1GOoZqgjELprRWuymDBUM9SvVrT6RD2GJwzHNy3feDz2Ny3fYHjCcP/a3ELkNrohkD0i2+vcW51GF5xwruoN6BcJ/+0m/eh7oVXFoK4VyDK3wXTeubWw6XwrssxtqGsFtKoY6Eff6/2gRERhEFDDCJvNhg8//BDbt2/HqVOn0L9/f2RmZuKGG26AQuHfiFA48GY0ijZi3hQl9g1XIbuhi5yY9qxAluEPqGsFUtRA6cWATg2YWoGsY7iw/foV0I1fHe5yiSgCsTNagBh0KRqJ2QJYzGOH4vjkQmsLTP/QIutIsyPUFg0Ccr7tEHKHx0M39xibUhCRKMIedL/77jucPXu2y3adTtp/cBh0KVqJ2bhA1KYIITg+ufDlGpj25jtGcO0cI7wT1nAZMyISTViWF2tpacFdd92FPn36YPDgwUhOTu7yICJp0ml0KJpV5LStaFZRUIKimMcOxfHJhUsXQtc7AUWDnDcXDQJ0vRO4jBkRSZ7fqy7cf//9ePPNN/HrX/8aY8aMQWxsrBh1EZEITBYTckpynLbllOQEbURXrGOH4vjkgjoOpuF3I8fwB6fNOd8CpVcugI5TFohI4vyeujB48GDk5+fjvvvuE6sm0XHqAkUjztElf5ksJmQVZKKu6UjXOboJySjNK+N5JyLRhGXqwo8//ojU1NSAvhkRhYerFQoydBlB6c4l5rFDcXxyzXHem44gpc9FKL0YyOglzM1N6XMR6poO87wTkeT5HXRnzJiBHTt2iFELEYlAzFa0Yre5DWkbXXLoct7zdghzciHMzS3Nq+B5JyJZ8DvoPvLII3j77bfx3HPP4dChQ/j++++7PIhIOsRsRSt2m9uQttElhy7nPfFypxbDusTLeN6JSBb8nqMbEyNkY0+NIaxWa/eqEhnn6HrQuB+ofQWAL5eFAhhxF9BvjNhVUTcZag0eW9ECwiheIK1o11SsQeawTKQPTXe7T1V9FcqPlmPF5BV+HRuN+2HY9Sj0fROg6xXndjfT2RYYTzchO+1xXo9B0uWaaTsD1L4mtBj+qftaoNeMmNcjEUWOYOQ1v1ddePTRR2XR/YwC9N0O4ODzgEIJzwP+7YDNCsSPYrCQAV/Cgk6j8/vGIkOtAY+VPQZtvNbtDWEmiwl3bLkD5mYzxiWN8y+4fLcD2Y3vAk2er0cd2qGzWYHvruf1GCRd3id7i+EOAr1mZr410+s1k1WYBXOzGdvmbWPYJaKA+R10V65cKUIZJBnJOcDnjwCtTQC8jMyrE4T9KWrpE/XQxmsdczU7B5fOcz31iXr/vgGvx4gj+jVDRNSB33N0Ozp79iyOHTuGtra2YNVD4aaO+2kunrdRewUwegVbf0Y5TzeEebqRzGe8HiOO6NcMEVEHAQXd0tJSpKenIy4uDsOGDcP+/fsBAAsXLsTmzZuDWiCFwaULAbXG8z5qDbsiEQDXwaXSVBm8wMLrMeKIfs0QEf3E76D7n//8B9dffz1+/PFHLFu2DO3t7Y7nEhMTUVBQEMz6KBy8jqJx9IycdQ4ukzdODl5g4fUYkUS9ZoiIfuJ30H300UcxY8YM7Nu3D08++aTTc1deeSU+++yzYNVG4eRpFI2jZ+SCTqND0awip21Fs4qCE1h4PUYkUa8ZIiIEEHT37duHu+++G0DXJcYGDBiAkydPBqcyCi+3o2gcPSPXTBYTckqcbwbLKckJTjMBXo8RSdRrhogIAQRdlUqF1tZWl8+dPHkScXHu17okmXE1isbRM3Kh801EFfNF6JzF6zGihOSaIaKo53fQnTBhAl5//XWXz23atAnp6e4XjSeZ6TKKxtEz6srVnfIZuozgt+fl9RgxQnbNEFHU8zvorlixAlu2bMGtt96KrVu3QqFQYPfu3Vi0aBE2bdqE5cuXi1EnhUvHUTSOnlEnnpaD8rSMVMB4PcpeyK8ZIopqfgfd6667DoWFhdixYwfmzJkDm82GhQsX4s0330RBQQEmT54sRp0ULo5RNHD0jLowNhhhbja7vVO+Y3AxN5thbDB27xvyepS9kF8zRBTVFDabzRbIC8+ePYudO3fi22+/RWJiIjIyMtCnT59g1yeKYPROjiouetxTcBlqDdAn6j3ebW6ymGBsMEqrHWrjfhh2PQp93wToermfn2862wLj6SZkpz3uX4vexv1A7SsAOvwz1d4KWA4AmsuAGHWHnRXAiLt8P77YtZNbsr3eiSikgpHX/G4BbNerVy9MmzbNads///lP3Hvvvairqwv0sCRFLnrcU/AYag2Y+dZMaOO1btcPtX/ca242Y9u8bdL54//dDmQ3vgs0KeHpAyId2qGzWYHvrvcvLH63Azj4PKBwcfyGqg5ftAM2KxA/yufjG/ZvwMw970KrAkq1KujUXdfpNbXakGVug7kN2NZ7KLIzn/e9dnLLl+tXp9FxmTEi6rZutQDu7IcffsDRo0eDeUiiiKdP1EMbr3U7J7HjnEZtvBb6RH2YKnUhOQdQJwgh09bq4WEV9kvO8XbEkB1fP/peaFUxqGsFssxtMJ13PqbpfCuyzG2oawW0qhjoR9/rX+1ERBR2QQ26ROQ/TzfgeLpxRxK8di2zC3CFBBGPr0u8HKXZv0WKGkLYPQaYflo50fTT13WtQIoaKL1+OXSJl/lXOxERhR2DLpEEuAq7laZKaYdcO09dy+y6s0KCiMfXXfUwSofHO4XdyrOdQu7weOiufDiw2omIKKwYdIkkonPYnbxxsvRDLuDDqGs317sV8/jqOOiuzEfpxXCE3cnmDiH3YkB31UNc3YGISKYYdIkkRKfRoWhWkdO2ollF0g25dp5GXYOx3q2Yx790IXS9E1A0yHlz0SBA1zuBa/USEcmYT0E3Li4O8fHxXh85OX7eaEJETkwWE3JKnH+PckpypL9ovttR1yB1LxPz+Oo4mIbfjZxvnTfnfAuYkhdwNJeISMZ8Wl5szpw5UCi83QxCRN3R+cazollFyCnJcczZlfT0BUAYdf3qD0Br04VtwexeJtLxTRYTsna+7ZiuUDRICLl1rUBW5dsoveQeaZ93IiJyK+CGEXLGhhEkNe5WV5D8qgudfbkG+PwhCA0eFMBVqy90MpPg8Z3Ob5+LUDrge+jUP6268N1FqPvhe3mcdyKiCBSMvMY5ukRh5inMelp6TJI6zqUN5miuCMfvct7zdghzciHMzS3Nq5DPeSciIpcC7oxGEYptUUPO2GCEudnsduTQHnbtndGMDUbpji7a59J+nh+cubkiHt/YYES9pd5xfnUandOxdYmXoTS3FFM2TkG9pV7a511OXLV1dsvPts5ERJ0w6JITtkUNvewR2dg2bxv0iXq3QcoexowNRum0/3VHvwRQ9QVG3Cn549s6hy03x+6yHwXOU1tnJ/63dSYi6kwSUxeOHTuGO+64A/3790fv3r1x1VVX4ZNPPvH4mvLycowbNw49e/ZESkoKXnrppRBVG9nYFjU8skdkex0t1Gl00g+5AKDqDegXCf+V8PHt/8fCPoXBZDE5Hbvjdp1GJ63Wy3ImdttoIqIOwh50GxsbkZGRAbVajQ8//BBfffUVnn32WSQkJLh9zeHDhzFjxgxMmTIF+/btw0MPPYQlS5aguLg4dIVHKLZFpWgh69bLciZ222giog7CvurCihUrUFlZiR07dvj8mgcffBBbt27FgQMHHNsWLFiAzz//HFVVVV5fz1UXvGhtgekfWmQdaXa55JKjLercY/wjRLLnaVk3hlyRtLYAJTrnpeI6UycAs+r5bwxRFIuIVRe2bt2K8ePHY+7cuRg4cCDGjh2Ll19+2eNrqqqqcP311zttmz59Ovbu3YvW1tYu+587dw7Nzc1OD/KAbVEpisi29bKcid02mojoJz4F3eTkZKSkpPj0GDFihF8F1NXV4cUXX8Qll1yCjz76CAsWLMCSJUtQVFTk9jUnTpzAoEHO/ToHDRqEtrY2NDQ0dNl/9erV0Gg0jsfQoUP9qjEqsS0qRRHZtl6WM7HbRhMRwcdVFzIzM0XrjNbe3o7x48fjqaeeAgCMHTsWX375JV588UWPLYU712OfgeGqzvz8fCxdutTxdXNzM8OuN/a2qIY/OG3O+RYovXIBdBxpoQjirvUyR3RF5Fgqzt4AxI6juUQUPD4F3YKCAtEKSEpKwuWXX+607bLLLvN4Y9ngwYNx4sQJp20nT56ESqVC//79u+wfGxuL2NjY4BQcJdgWlaKF7Fsvy5nYbaOJKOqFfY5uRkYGjEaj07aDBw9i2LBhbl+Tnp4Og8HgtO3jjz/G+PHjoVarRakzmjj+8DcdEdqiXgxk9BLm5qb0uQh1TYfZKYoigqvVFTJ0GfLqRidnXebqcjSXiIIr4KBrsViwZ88ebN++vcvDHw888AB27dqFp556CjU1NXjzzTexYcMGLFx44f/R5+fnO01jWLBgAY4ePYqlS5fiwIEDeO211/Dqq69i2bJlgf449BO2RaVoEVGtl+VM7LbRRBTV/O6M1tbWhgULFqCoqAhWq9XlPu62uzJhwgRs2bIF+fn5ePzxx5GcnIy1a9fi9ttvd+xz/PhxmEwX/sgkJyfjgw8+wAMPPID169djyJAhWLduHebMmePvj0OduGxH66Itqiza0RJ5EFGtl+VM7LbRRBTV/F5H949//COeeeYZPPvss8jJycH69euhVqvx8ssvw2KxYN26dV2W/pIarqPrmaHW4NyOtu0MUPua0Bb1p25UJotJHu1oidxp3A/Drkeh75sAXa84t7uZzrbAeLoJ2WmPS6cVbeN+oPYVwKfWxApgxF3Sqd0VF//GEBEFI6/5HXTHjBmDu+66CwsXLoRarcbevXtx9dVXAxDWsr366quxevXqgIoJFQZdIsLB9cDeRYBCCc+zuNqFdrTj1wOXSqTltZxrJyLyUVgaRtTV1eHKK69ETIzw0h9//NHx3IIFC/C3v/0toEKIiEIqOUfovmWzArZWDw+rsF+y++UOQ07OtRMRhZDfQbdPnz44f/48FAoFLrroIhw9etTxXK9evXDq1KmgFkhEJAqv3bnsJLgSgJxrJyIKIb+D7qhRo3D48GEAwKRJk/Dcc8/BbDbj5MmTePrpp6HX64NeJBGRKDx157KT6koAcq6diChE/A66v/jFL3Dw4EEAwKpVq/D1119j2LBhSEpKws6dO/Hkk08GvUgiIlF4HRmV8IionGsnIgoRv29G66y+vh5btmxBTEwMsrOzZTGiy5vRiMihtQUo0Tl357JTJwCz6qUbFuVcOxGRF8HIa36vo9vZ0KFDsWTJku4ehogoPBzruD4E5+W6ZDAiKufaiYhCIOAR3aqqKpSWluLUqVPo378/srKykJ6eHuz6RMERXSJy4mpkVC4jonKunYjIg7CM6J49exa33XYbtm3bho4ZWaFQYMaMGXjnnXfQq1evgIohIgqLLiOjMhoRlXPtREQi8/tmtOXLl+PDDz/Ek08+icOHD+Ps2bM4fPgwnnjiCXz00UdYvny5GHUSEYmr4yoGclutQM61ExGJyO+g+/bbb+N3v/sd8vPzMWzYMMTGxmLYsGF46KGH8PDDD+Pvf/+7GHUSEYnLsYoB5DciKufaiYhE5PfUhTNnzmDSpEkun8vIyMAf//jHbhdFFFUa9wO1r8D5ZiJ3FMCIu4B+Y3w//tFNWGNYjMy43kiP6+N2t6qWH1DecgYrsp8Hhv3c58Mbag3QJ+qh0+jc7mOymGBsMCJ7RLbvdYeDfgmg6guMuDPclfhPzrUT+aG6Gti8GWhqAhISgNmzgdTUcFdFUuV30E1LS8OePXswbdq0Ls/t2bMH11xzTVAKI4oa3+0ADj4PKJTw/CFLu9DSNX6UX0F3zY7HkX/sBFQAtmuBdBdT6KvOAlPNQBsA7HgcK3wMuoZaA2a+NRPaeC1Kc0tdhl2TxYSswiyYm83YNm+btMOuqjegXxTuKgIj59qJfFBTA+TmAjt3AkolEBMDtLcDK1cCGRlAQQEwcmS4qySp8Xvqwrp16/DXv/4V69evR2NjIwCgsbERL7zwAjZs2IDnn38+6EUSRbTkHOEueZsVsLV6eFiF/ZJz/Dp85sTfQwUhxE41C6G2o44hVwUgM+0pn4+tT9RDG69FXWMdsgqzYLKYnJ63h9y6xjpo47XQJ0p/nW0ikp6aGmDiRGD3buFrqxVobRX+CwC7dgnP19SEr0aSJr+XF4uLi0NraytaW1sBACqVCm1tbQAAtVqNHj16XDi4QgGLxRLEcoODy4uR5Hy5xsVaqJ0pgKtWX5iL6YeqLRMxdf9/HWHWPrLbOeRuv3Ii0mft8uvYHcNsSr8Ux8iuu+1ERP7KyBBCrj3YuqJUAmlpQEVF6OoicQUjr/kddPPy8qBQuGs52dXGjRv9LkpsDLokOZ46XNl1Z23UM9+g6q2LnULtugHAku/gHH5/eRzoNdjvw3cOtUWzipBTksOQS0TdVl0NjPHjtoT9+zlnN1KEZR3dgoKCgL4REXngtsOVXTfXRu09BOnJ07EdHznC7r3fCU85Qm7KDQGFXADQaXQozS11hN3JGycDAEMuEXXb5s3CaK2n0Vw7pRLYsoVBly7we44uEYmk41qonQVjbdS015DeSxjJ7WjdgJ9uUEvr3qcvOo0ORbOKnLYVzSpiyCWibmlqEm4880VMDPDT7UNEAAIMul9//TXmzZuHpKQk9OjRA59++ikAYNWqVSgtLQ1qgURRw7EWauepQUHqdNV7CKp6XYMl3zlvXvIdUNV7YsCjuXYmiwk5Jc43yuWU5HS5QY2IyB8JCcLqCr5obwf69RO1HJIZv4PuZ599hgkTJqC8vBzXXnstrB0+Szh9+jReeumloBZIFFVcjeoGqdNVVX0Vpn7xqWNO7l8G4MJqDNWfoKq+KuBjd56jWzG/Ain9UtyuxkBE5KvZs32btgAI+82eLW49JC9+B90VK1ZgzJgxqKmpweuvv46O97Jdc8012LNnT1ALJIoqXUZ1gzOaW1VfhakFU9HW3gaVQoHtWuCeBGFurkqhQFt7G6YWTA0o7LpaXSFDl4HS3FKGXSLqttRUYNIkYf6tJ0qlsDrDFVeEpi6SB7+DbmVlJZYvX47evXt3WX1h0KBBOHHiRNCKI4pKHUd1gzCa6xRyY1TYPu9dR9OI9F7A9l9uhSpGFVDY9bSEmP0GNYZdIuquwkJAo3EfdpVK4XneL0+d+b3qgs1mc1ort6PGxkbExsZ2uyiiqOKqBbDmMqChSvjv5/kddva/BXD50fILITdvO9KHpgP104ETHwFJNyB95Exsz9vuCMPlR8uFfXxg/OplmJsOIyW2B0qHqqGrnOn0vA5A6VA1ss70gLnpMIxfvQxd+hM+1y5qe2SxWy8TSZjc2uiOHCmso5uXB1RWOndGs1qF9XPZGU1ccrtm7PwOumPGjMGWLVtw4403dnnun//8J8aNGxeUwoiihqsWwDab8L8b9gCn9v60Y2AtgFdMXgEAyByWeSHATnoD2HMPMOFFAED60HRsz9uO8qPljv19kW01YluSDfoe56E7awTOdt1HB6A0CTCeB7Kth3w+NgBx2yOL3HqZSIrk3EZ35EihGUR1tbCEWGOjcOPZ7NmcriAmOV8zQAANI95++2388pe/xJIlS/CrX/0K48ePx+bNm1FTU4OHH34YmzZtws033yxWvUHBhhEkKb40i7DrTtMIMZz5Bii52Pf9b/WzIYWY50bO550oAPY2uhaL65u77B//794t7eBCoRPuayYYec3vObq/+MUv8MQTT2D9+vWYMGECAGDOnDl4+OGHsWrVKsmHXCLJcbusWGdBWmYsmHoPAQZP923fpAAaUoh5buR83okCkJvrPrAAwnaLRZgeQARExjXj94iundlsxkcffYRvv/0WiYmJmD59OoYNGxbs+kTBEV2SHLFbAIvJ11Fdf0dz7cQ8N3I+70R+YBtd8pcUrpmwjOjaabVa/PrXv8ZDDz2E//u//5NNyCWSJK+jixIeVfRlVDeQ0Vw7Mc+NnM87kR/sbXR9YW+jS9EtUq6ZbrUA/v7777FixQrMnDkTd999N7788stg1UUUfcRuASymtNe8PN+99sKinhs5n3ciH7GNLvkrUq4Zn36EZcuWQadz7lf/ww8/YPz48fjjH/+IDz74AC+//DImTZoEo9EoSqFEEU/sFsBi8jSq253RXDsxz42czzuRj9hGl/wVKdeMT0F3586duO2225y2vfDCCzhy5Ajuv/9+NDU1YefOnejbty/WrFkjSqFEUUHEFsCiczeq293RXDsxz42czzuRD9hGl/wVKdeMT0G3rq4O48ePd9r23nvvYcCAAXj66acRHx+PtLQ0LF26FGVlZWLUSRQdRGoBHBKuRnWDMZprJ+a5kfN5J/IB2+iSvyLlmvEp6DY1NSEpKcnxdVtbG/bs2YNrr70Wyg5nYOzYsTh+/HjwqySKJkFuARxSnUd1gzWaayfmuZHzeSfyAdvokr8i4ZrxKegOGjTIKcB++umnaG1t7TLKGxMTwxbARN3lGF2E/EYVO47qBnM0107McyPn807kA3sb3bQ04WulElCrL4SYtDQ2iyBnkXDN+NQCeNy4cXj55Zcxd+5cKBQK/O1vf4NCocC0adOc9vv666+dRn6JIkbjfqD2FQC+LDutAEbc1b12sfolgKovMOLOwI9hJ3LthloD9Il66DQ/3bDaqb0wAJgsJhgbjMgekd392tvbgAFTgdNHgL2LA69dzGMTSRTb6JK/5H7N+BR0H3zwQWRkZECv1yMxMRG7du3ClClTcPXVVzvt99577zm6pRFFlO92AAefBxRKeP4gpB2wWYH4Ud0LRaregH5R4K/vSMTaDbUGzHxrJrTxWpTmlgpht2ciMOUfjn1MFhOyCrNgbjZj27xt/oVdT7U3VHWrdlGPTSRxqalsCEH+kes149PUhYkTJ+Ldd9/FkCFD0NLSgrvuugtbOq0MfOLECZjNZtxyyy2iFEoUVsk5QocsmxWwtXp4WIX9knPCXfEFItauT9RDG69FXWMdsgqzYLKYnJ63h9y6xjpo47XQJ+olU7us31MiIvKJzw0jbrrpJpSVlaG6uhobNmzARRdd5PT84MGD8fnnn+PnP/950IskCjuvHbTsJHi3voi16zQ6lOaWIqVfSpew2zHkpvRLuTDiK5HaZf2eEhGRT7rVGY0oqnjqoGUn1bv1RazdVditNFV2P+SGoHZZv6dEROQVgy6Rr7yOAEp45E/k2juH3ckbJwcn5ALi1i7n95SIiLxi0CXyh6cRQKmP/Ilcu06jQ9GsIqdtRbOKuhdy7cSsXc7vKRERecSgS+QPtyOAMhj5E7l2k8WEnBLnG7ZySnK63KAWEDFrl/N7SkREHjHoEvnL1QigXEb+RKq9841nFfMrXN6g1i1innc5v6dEROQWgy6Rv7qMAMpo5E+E2l2trpChy3C7GoOUag/JsYmIKGwYdIkC0XEEUG4jf0Gs3dMSYp6WHpNC7SE9NhERhQWDLlEgHCOAkN/IXxBrNzYYYW42u11doWPYNTebYWwwdqdycc+7nN9TIiJySWGz2Wzed4sszc3N0Gg0sFgsiI+PD3c50aNxP1D7CgBfLjkFMOIu/1quHt0EfPmE78cf/TtgmI8NTlzV3t4KWA4AmsuAGHXgtYt9XlxpOwPUvgaMuFNoN9wNhloD9Il6j6srmCwmGBuM/rX/dSeItYf02BQW1dXA5s1AUxOQkADMni3PNqYUGXg9+icYeY1Bl0E3dA6uB/YuAhRKeP4woV1ouzp+PXDpvb4fv+J/AdM/fN9f9wtg8t9921fM2sU+L0RRqKYGyM0Fdu4ElEogJgZobwesViAjAygoAEaODHeVFC14PQYmGHkt7FMXVq5cCYVC4fQYPHiw2/3Lysq67K9QKPD111+HsGoKSHIOoE4Qwpqt1cPDKuyXnOPtiM6uXuvf/uP82F/M2sU+L0RRpqYGmDgR2L1b+NpqBVpbhf8CwK5dwvM1NeGrkaIHr8fwCnvQBYDRo0fj+PHjjkd1dbXX1xiNRqfXXHLJJSGolLrFaxcquwDveO89BBg83bd9k24Aern/P1RdiFm72OeFKMrk5gIWy4Ug0ZnVKjyflxfSsihK8XoML0kEXZVKhcGDBzseAwYM8PqagQMHOr1GqVSGoFLqNk9dqOy6c8d72ms+7rfR/2OLWbvY54UoSlRXCx8PuwsVdlYrUFkp7E8kFl6P4SeJoHvo0CEMGTIEycnJuO2221BXV+f1NWPHjkVSUhKmTZuG0tJSj/ueO3cOzc3NTg8KE6+jl90ctfRlVNff0Vw7MWsX+7wQRYnNm4U5kL5QKoEtW8Sth6Ibr8fwC3vQnThxIoqKivDRRx/h5ZdfxokTJzBp0iScOnXK5f5JSUnYsGEDiouLsXnzZuj1ekybNg3bt293+z1Wr14NjUbjeAwdOlSsH4d84Wn0Mhijlt5GdQMZzbUTs3axzwtRFGhqEm708UVMDNDYKGo5FOV4PYZf2IPujTfeiDlz5iA1NRXXXXcd3n//fQBAYWGhy/31ej1+85vf4Oqrr0Z6ejr+8pe/4KabbsIzzzzj9nvk5+fDYrE4HvX19aL8LOQjt6OXQRq19DSqG+horp2YtYt9XoiiQEKCcDe7L9rbgX79RC2Hohyvx/ALe9DtrE+fPkhNTcWhQ4d8fk1aWprH/WNjYxEfH+/0oDBzNXoZzFFLd6O63RnNtROzdrHPC1GEmz3b+3xIO6tV2J9ILLwew09yQffcuXM4cOAAkpKSfH7Nvn37/NqfJKDL6GWQRy1djep2dzTXTszaxT4vRBEuNRWYNMn7vEilUli/9IorQlMXRSdej+EX9qC7bNkylJeX4/Dhw9i9ezd+/vOfo7m5Gbm5uQCEaQc5ORfWDV27di1KSkpw6NAhfPnll8jPz0dxcTEWLVoUrh+BAtVx9FKMUcvOo7rBGM21E7N2sc8LUYQrLAQ0GvfhQqkUni8oCGlZFKV4PYaXKtwFmM1mzJs3Dw0NDRgwYADS0tKwa9cuDBs2DABw/PhxmEwmx/7nz5/HsmXLcOzYMfTq1QujR4/G+++/jxkzZoTrR6BA2UcvP88XZ9TSPqp74qPgjebaiVm72OeFBOFovUxdiNESdeRIYXH+vDxhyabOnajS0uTRiUrO7WKLi4EXXhDWh9VogEWLgDlzwl1VeETK9ShXbAHM+brh1XYGqH0NGHEnoOod/OP/2ADsuQeY8CLQMzG4xxazdrHPC7H1cpiFqiVqdbWwZFNjo3Cjz+zZ0v94WM7tYv/9b+DWW4GWlq7PxcUJ78W0aaGvSyrkeD2GUzDyGoMugy5RdGptAUp0QGuT933VCcCseo6uB4m9Jaq7blH2j3J375ZuoBOLnM/Nv/8NZGcDnlKFQgEYDNEddsl3wchrYZ+jS0QUFmy9HDZsieqenM/Nrbd6DrmA8Pytt4amHiKAQZeIohlbL4ccW6K6J+dzU1zserqCKy0twv5EocCgS0TRi62XQ44tUd2T87l54QX/9l+/Xpw6iDpj0CWi6MbWyyHFlqjuyfncWCz+7d/UJEoZRF0w6BJRdGPr5ZBiS1T35HxuNF5mAHWWkCBKGURdMOgSEbH1csiwJap7cj43/vZsWrxYnDqIOmPQJSJi6+WQYUtU9+R8bubMEdbJ9UVcHFdeoNBh0CUiAth6OYTYEtU9OZ+bLVuEdXI9USikdRMdRT4GXSIioMOoLjiaKzJ7S9S0NOFrpRJQqy+Eu7Q0aTZECAU5n5tp04RmEO5GduPi2CyCQo+d0dgZjYjs2Ho55NgS1b3iYmHZLotFGMVdvFha83I9KS4WlhBrahJuPFu8mNMVyH9sARwgBl0iIpKqmhqhQ9rOncJIbkyMsMqC1SrMzS0okOaILlGwsQUwERFRBKmpASZOFKYnAEK4bW29sBrDrl3C8zU14auRSE4YdImIiCQiN1eYquBumTGrVXg+Ly+kZRHJFoMuERGRBFRXC9MVvK2la7UClZXC/kTkGYMuERGRBGze7H0NXTulkst0EfmCQZeIiEgCmpqEG898ERMjrFRBRJ4x6BIREUlAQoKwuoIv2tuF5diIyDMGXSIiIgmYPdv7/Fw7q1U+a+oShRODLhERkQSkpgKTJnmfp6tUCuvpsrEGkXcMukRERBJRWCh0QXMXdpVK4fmCgpCWRSRbDLpEREQSMXKk0CwiLU34WqkE1OoLwTctTXiendGIfKMKdwFEJLLG/UDtKwB86fatAEbcBfQbI3ZVRACEtWA3bxZWHEhIEOadpqbK5/hiGDkSqKgQat+yRVhdoV8/oXZOVxCfHK8Zck9hs9l8+esXUYLRO5lINg6uB/YuAhRKeP4Qpx2wWYHx64FL7w1VdRSlamqELmA7dwqjlTExwkoCVqsw/7SgoHujlmIfnyIPrxnpCUZeY9Bl0KVI19oClOiA1ibv+6oTgFn1gLqv2FVRFKupASZOdN/q1j4PNdCP6MU+PkUeXjPSFIy8xjm6RJFOHQdc/iAAhZcdFcDoFQy5JLrcXPeBAhC2WyxAXp40j0+Rh9dM5GLQJYoGly4E1BrP+6g1wCULQ1MPRa3qauGjYW/rxVqtQGWlsL+Ujk+Rh9dMZGPQJYoGXkd1OZpLobF5s/d1Yu2USuFmLCkdnyIPr5nIxqBLFC08jepyNJdCpKlJuMnHFzExwooDUjo+RR5eM5GNQZcoWrgd1eVoLoVOQoJwJ7sv2tuFZbWkdHyKPLxmIhuDLlE0cTWqy9FcCqHZs73PhbSzWoX9pXR8ijy8ZiIbgy5RNOkyqsvRXAqt1FRg0iTvcyKVSmHtUn8bJIh9fIo8vGYiG4MuUbTpOKrL0VwKg8JCYU1Sd8HCvmZpQYE0j0+Rh9dM5GILYCJfRFIbXfuo7uf5HM2lsBg5Ulh4Py9PWK5JoRAeNpvwSEvrXhcqsY8fKmK2on3uOeD554EffgD69AEWLwaWLg3OseWo8zXTuTOaXK4Z6opBl8gX3+0ADj7vexvd+FHSDboAoF8CqPoCI+4MdyUUxcTuyynXvp/uWtGuXNn9VrRFRcD8+c43X333HfD//h/w298CGzcCOTnB+CnkZ+RIoKJC+D8YW7YIqyv06yf8HwxOV5AvtgBmC2DyBdvoEgUNWwC7J2btRUVCgPamsDB6wy5JC1sAE4UK2+gSBQ1bALsnZu3z5wd3PyI5YNAl8hXb6BJ1G1sAuydm7c89599asc895/uxiaSMQZfIV2yjS9RtbAHsnpi1P/+8f7X4uz+RVDHoEvmDbXSJuoUtgN0Ts/YffvCvFn/3J5IqBl0if7CNLlG3sAWwe2LW3qePf7X4uz+RVDHoEvmLbXSJAsYWwO6JWfvixf7Vct99/u1PJFUMukT+YhtdooCxBbB7Yta+dKl/0yLuv9/3YxNJGYMuUSDYRpcoYGwB7J6YtW/cGNz9iOSAQZcoEI5RXXA0l8hP9naraWnC10oloFZfCHdpad1r5iD28cUkZu05OUKQdjeyGxPDZhEUedgZTYzOaI37gdpXAPhyahXAiLuk3S6WXGs7A9S+JrTRVfUOdzXUTdXVwvJOTU3CTUGzZwsfJcuB2LWLefziYuCFF4QmCBqNMJc0mPNmxTx+KM67WK1on3tOWELshx+EG8/uu4/TFUh6gpHXGHTFCLoH1wN7FwEKJTwPmrcDNiswfj1w6b3Br4OIvKqpEbpR7dwpjJrFxAh3tFutwjzIggJpjvwB4tcu5vFZOxF5ExEtgFeuXAmFQuH0GDx4sMfXlJeXY9y4cejZsydSUlLw0ksvhahaHyXnAOoEIcTaWj08rMJ+yfyciCgcamqAiROFj4IBIai0tl64833XLuH5mprw1eiO2LWLeXzWTkShEvagCwCjR4/G8ePHHY9qD30NDx8+jBkzZmDKlCnYt28fHnroISxZsgTFxcUhrNgLrx207Hi3PlE45eYKH2m7W9LJahWez8sLaVk+Ebt2MY/P2okoVMI+dWHlypUoKSnBZ5995tP+Dz74ILZu3YoDBw44ti1YsACff/45qqqqfDqG6FMXAKC1BSjRAa1N7vdRJwCz6hl0icKguhoY48fU+P37pTNnV+zaxTw+ayciX0XE1AUAOHToEIYMGYLk5GTcdtttqKurc7tvVVUVrr/+eqdt06dPx969e9Ha2uryNefOnUNzc7PTQ3ReR3U5mksUTps3e1+v1E6pFG4Kkgqxaxfz+KydiEIp7EF34sSJKCoqwkcffYSXX34ZJ06cwKRJk3Dq1CmX+584cQKDBg1y2jZo0CC0tbWhoaHB5WtWr14NjUbjeAwdOjToP4dLrjpo2XHtVaKwamrybwH9xkZRy/GL2LWLeXzWTkShFPage+ONN2LOnDlITU3Fddddh/fffx8AUFhY6PY1CoXzKKl99kXn7Xb5+fmwWCyOR319fZCq98LtqC5Hc4nCLSFBuFPeF+3twvJOUiF27WIen7UTUSiFPeh21qdPH6SmpuLQoUMunx88eDBOnDjhtO3kyZNQqVTo37+/y9fExsYiPj7e6REyrkZ1OZpLFHazZ7u/oagzqzW4a7t2l9i1i3l81k5EoSS5oHvu3DkcOHAASUlJLp9PT0+HwWBw2vbxxx9j/PjxUKvVoSjRP11GdTmaSyQFqanApEne51wqlcLaqMFaqD8YxK5dzOOzdiIKpbAH3WXLlqG8vByHDx/G7t278fOf/xzNzc3Izc0FIEw7yOnQj3DBggU4evQoli5digMHDuC1117Dq6++imXLloXrR/Cu46guR3OJJKOwUOiW5S64KJXC8wUFIS3LJ2LXLubxWTsRhUrYg67ZbMa8efOg1+sxe/Zs9OjRA7t27cKwYcMAAMePH4fJZHLsn5ycjA8++ABlZWW46qqr8MQTT2DdunWYM2dOuH4E7xyjuuBoLpGEjBwpLPyfliZ8rVQCavWFEJOWJjwvxS5XYtcu5vFZOxGFStjX0Q2HkKyj21nbGaD2NWDEnYCqd2i+Z7Rp3A/UvgLAl0taAYy4C+jn+6KYhloD9Il66DQ6t/uYLCYYG4zIHpHt83FJGoqLgRdeEBb712iAxYuDO8ey8/EXLQKC9f/Pq6uFpawaG4UboGbPDu7H5mIen7UTkTvByGsMuqG8MY3EdXA9sHcRoFDC84cV7UL75fHrgUvv9enQhloDZr41E9p4LUpzS12GXZPFhKzCLJibzdg2bxvDrkzU1AjdrnbuFEblYmKEO+atVmGeZUFB90bn/v1v4NZbgZaWrs/FxQlBadq0wI9PRBSpIqZhBFFQJOcI3eZsVsDW6uFhFfZLzvF2RAd9oh7aeC3qGuuQVZgFk8Xk9Lw95NY11kEbr4U+UR/cn41EUVMDTJwofNQMCOG2tfXCnfW7dgnP19QEdvx//xvIznYdcgFhe3a2sB8REQUfgy5FDq/d6Oz8X/lCp9GhNLcUKf1SuoTdjiE3pV+K2xFfkp7cXGEqgbslo6xW4fm8vMCOf+utgLfPzGw2YT8iIgo+Bl2KLJ660dkFuPKFq7BbaapkyJWp6mphuoK3dVGtVqCyUtjfH8XF7kdyO2tpEfYnIqLgYtClyOJ1VLd76xh3DruTN05myJWpzZu9r4dqp1QKc2n98cIL/u2/fr1/+xMRkXcMuhR5PI3qBmEdY51Gh6JZRU7bimYVMeTKTFOTcOOZL2JihDvr/WGx+F8PEREFF4MuRR63o7rB6UpnspiQU+J8I1tOSU6XG9RI2hIShNUVfNHeLiwf5Q+Nlxk0ruohIqLgYtClyORqVDcIo7mdbzyrmF/h8gY1kr7Zs73Pz7WzWv1fU3fRIv/2X7zYv/2JiMg7Bl2KTF1Gdbs/mutqdYUMXYbb1RhI2lJTgUmTvM/TVSqF9XT9bQQwZ46wTq4v4uK48gIRkRgYdClydRzV7eZorqclxDwtPUbSVlgoTDFwF3aVSuH5goLAjr9lC6DwstqdQuH/jW5EROQbBl2KXI5RXXR7NNfYYIS52ex2dYWOYdfcbIaxwdidyilERo4UmkWkpQlfKxTCjWf2cJqWJjwfaGe0adMAg8H9yG5cnPB8NHdGKy4GsrKAq68W/stl1ogomFThLoBIVPolgKovMOLObh0me0Q2ts3bBn2i3u3qCvawa2wwsv2vzIjZCH3YMGGaxM6dXZ8bM0Z4Phq5a41cVsbWyEQUPAqbTcx/4qUpGL2TiUj+7C2A3XVHs09dCHRUV+zjy5W9NbKnvz4KBUe7iaJdMPIapy4QUdQSuwWw2MeXK7ZGJqJQYdAloqgkdgtgsY8vV2yNTEShxKBLRFFJ7BbAYh9frtgamYhCiUGXiKKS2C2AxT6+XLE1MhGFEoMuEUUlsVsAi318uWJrZCIKJQZdIopKYrcAFvv4csXWyEQUSgy6RBSVxG4BLPbx5YqtkYkolBh0iShqid0CWOzjyxVbIxNRqDDoElHU6twCWKkE1OoLwbS7LYDFPr5csTUyEYUKO6OxMxoRQVjHdssWYfWDfv2EObPBnE4g5vGrq4XlzJqahJu3Zs8Wpk7IQXGxsISYvfbFizldgYgEwchrDLoMukQkUzU1Qve1nTuFUeKYGGEFB6tVmPdbUBB9o8VEFDnYApiIKErV1AATJwpTHwAh3La2XljpYdcu4fmamvDVSEQUbgy6REQylJsrNF9wt4SZ1So8n5cX0rKIiCSFQZeISGaqq4XpCt7W6bVagcpKYX8iomjEoEtEJDObN3tfn9dOqeQyXUQUvRh0iYhkpqlJuPHMFzExwkoPRETRiEGXiEhmEhKE1RV80d4uLGdGRBSNGHSJiGRm9mzv83PtrFZhfyKiaMSgS0QkM6mpwKRJ3ufpKpXCerrBbHxBRCQnDLpERDJUWAhoNO7DrlIpPF9QENKyiIgkhUGXiEiGRo4UmkWkpQlfK5WAWn0h+KalCc+zMxoRRTNVuAsgIqLAjBwJVFQAxcXACy8IDSI0GmDxYvnMy62uFpZLa2oSbrKbPVuYmkFEFAwKm81mC3cRoRaM3slEROFWUyN0SNu5UxjJjYkRVlmwWoW5uQUF0h3RlXPtRBQawchrnLpARCRDNTXAxInC9ARACIitrRdWY9i1S3i+piZ8Nboj59qJSF4YdImIZCg3V5iq4G6ZMatVeD4vL6Rl+UTOtRORvDDoEhHJTHW18JG/t7V0rVagslLYXyrkXDsRyQ+DLhGRzGze7H0NXTulEtiyRdx6/CHn2olIfhh0iYhkpqlJuHnLFzExQGOjqOX4Rc61E5H8MOgSEclMQoKwQoEv2tuBfv1ELccvcq6diOSHQZeISGZmz/Y+x9XOapXWmrpyrp2I5IdBl4hIZlJTgUmTvM91VSqFNWmvuCI0dflCzrUTkfww6BIRyVBhodAFzV1gVCqF5wsKQlqWT+RcOxHJC4MuhU11NbBqFfDAA8J/uYwQke9GjhQaLqSlCV8rlYBafSE8pqUJz0uxu5icaycieWELYLYADjm2/iQKrupqYRmuxkbh5q3Zs+Xzkb+caycicQUjr0kq6K5evRoPPfQQ7rvvPqxdu9blPmVlZcjKyuqy/cCBAxg1apRP34dBN3zsrT/ddUWyf2TJ0RwiIqLoFoy8pgpyTQHbs2cPNmzYgDFjxvi0v9FodPqhBwwYIFZpFET+tP6sqAhpaURERBRhJDFH9/Tp07j99tvx8ssvo5+PiyYOHDgQgwcPdjyUvrbaobBh608iIiIKJUkE3YULF+Kmm27Cdddd5/Nrxo4di6SkJEybNg2lpaUe9z137hyam5udHhR6bP1JREREoRT2qQt///vf8emnn2LPnj0+7Z+UlIQNGzZg3LhxOHfuHF5//XVMmzYNZWVlmDp1qsvXrF69GqtWrQpm2RQAe+tPXxaLZ+tPIiIi6q6wBt36+nrcd999+Pjjj9GzZ0+fXqPX66HX6x1fp6eno76+Hs8884zboJufn4+lS5c6vm5ubsbQoUO7Vzz5ja0/iYiIKJTCOnXhk08+wcmTJzFu3DioVCqoVCqUl5dj3bp1UKlUsPrYJzItLQ2HDh1y+3xsbCzi4+OdHhR6bP1JREREoRTWEd1p06ahutMdR/Pnz8eoUaPw4IMP+nyD2b59+5CUlCRGiRRE9tafu3d7DrxKpbBgPNfSJCIiou4Ia9CNi4vDFZ3STJ8+fdC/f3/H9vz8fBw7dgxFRUUAgLVr12L48OEYPXo0zp8/jzfeeAPFxcUoLi4Oef3kv8JC39bRZetPIiIi6q6w34zmzfHjx2EymRxfnz9/HsuWLcOxY8fQq1cvjB49Gu+//z5mzJgRxirJV/bWn3l5whJinTujpaWxMxoREREFh6Q6o4UKO6NJA1t/kr+qq4Vl6pqahJsbZ88WpsRI/dhik3PtRETuRFwL4FBh0CWSl5oaoavezp1dPwXIyOjepwBiHltscq6diMgbBt0AMegSyUdNjW/zunfv9j/UiXlsscm5diIiXwQjr0miMxoRkTu5ue7DHCBst1iEed9SOrbY5Fw7EVGoMOgSkWRVVwsfy3tbf9lqFW5u7LRaYdiOLTY5105EFEoMukQkWZs3Cx/B+0KpFG5ulMKxxSbn2omIQolBl4gkq6lJuMHKFzExwgoeUji22ORcOxFRKDHoEpFkJSQIqwj4or1dWKZOCscWm5xrJyIKJQZdIpKs2bO9z0O1s1qF/aVwbLHJuXYiolBi0CUiyUpNBSZN8j4fVakU1o31p+GImMcWm5xrJyIKJQZdIpK0wkJhPVh3oc6+XmxBgbSOLTY5105EFCoMukQkaSNHCk0P0tKEr5VKQK2+EPDS0gJviiDmscUm59qJiEKFndHYGY1INqqrhaWyGhuFG6xmzw7ex/LFxcALLwhNFjQaYPFi+cxtFfO8EBGFC1sAB4hBl4jsamqELmM7dwqjoTExwkoFVqswv7WggKOiREThwBbARETdUFMDTJwofMQPCOG2tfXCiga7dgnP19SEr0YiIgocgy4RRa3cXGGqgruluqxW4fm8vJCWRUREQcKgS0RRqbpamK7gbT1aqxWorBT2JyIieWHQJaKotHmz93Vo7ZRK4WYvIiKSFwZdIopKTU3CjWe+iIkRVjQgIiJ5YdAloqiUkCCsruCL9nZh2S4iIpIXBl0iikqzZ3ufn2tntcpnTV0iIrqAQZeIolJqKjBpkvd5ukqlsJ4uGzAQEckPgy4RRa3CQqELmruwq1QKzxcUhLQsIiIKEgZdIopaI0cKzSLS0oSvlUpArb4QfNPShOfZGY2ISJ5U4S6AiCicRo4EKiqEdXK3bBFWV+jXT5iTy+kKRETyxqBLRARhzm5qarirICKiYOLUBSIiIiKKSAy6RERERBSRGHSJiIiIKCIx6BIRERFRRGLQJSIiIqKIxKBLRERERBGJQZeIiIiIIhKDLhERERFFJAZdIiIiIopIDLpEREREFJEYdImIiIgoIjHoEhEREVFEUoW7gHCw2WwAgObm5jBXQkRERESu2HOaPbcFIiqDbktLCwBg6NChYa6EiIiIiDxpaWmBRqMJ6LUKW3disky1t7fjm2++QVxcHBQKRUi+Z3NzM4YOHYr6+nrEx8eH5HuSuPieRh6+p5GJ72vk4XsamTq/rzabDS0tLRgyZAhiYgKbbRuVI7oxMTHQarVh+d7x8fH8pYwwfE8jD9/TyMT3NfLwPY1MHd/XQEdy7XgzGhERERFFJAZdIiIiIopIDLohEhsbi8ceewyxsbHhLoWChO9p5OF7Gpn4vkYevqeRSYz3NSpvRiMiIiKiyMcRXSIiIiKKSAy6RERERBSRGHSJiIiIKCIx6BIRERFRRGLQDYLVq1djwoQJiIuLw8CBAzFr1iwYjUavrysvL8e4cePQs2dPpKSk4KWXXgpBteSLQN7TsrIyKBSKLo+vv/46RFWTJy+++CLGjBnjWIg8PT0dH374ocfX8HdU+vx9X/l7Kj+rV6+GQqHA/fff73E//r7Khy/vabB+Vxl0g6C8vBwLFy7Erl27YDAY0NbWhuuvvx4//PCD29ccPnwYM2bMwJQpU7Bv3z489NBDWLJkCYqLi0NYObkTyHtqZzQacfz4ccfjkksuCUHF5I1Wq8WaNWuwd+9e7N27F//zP/+DW265BV9++aXL/fk7Kg/+vq92/D2Vhz179mDDhg0YM2aMx/34+yofvr6ndt3+XbVR0J08edIGwFZeXu52n+XLl9tGjRrltO3uu++2paWliV0eBcCX97S0tNQGwNbY2Bi6wqhb+vXrZ3vllVdcPsffUfny9L7y91Q+WlpabJdcconNYDDYMjMzbffdd5/bffn7Kg/+vKfB+l3liK4ILBYLAOCiiy5yu09VVRWuv/56p23Tp0/H3r170draKmp95D9f3lO7sWPHIikpCdOmTUNpaanYpVEArFYr/v73v+OHH35Aenq6y334Oyo/vryvdvw9lb6FCxfipptuwnXXXed1X/6+yoM/76ldd39XVX6/gjyy2WxYunQpJk+ejCuuuMLtfidOnMCgQYOctg0aNAhtbW1oaGhAUlKS2KWSj3x9T5OSkrBhwwaMGzcO586dw+uvv45p06ahrKwMU6dODWHF5E51dTXS09Px448/om/fvtiyZQsuv/xyl/vyd1Q+/Hlf+XsqD3//+9/x6aefYs+ePT7tz99X6fP3PQ3W7yqDbpAtWrQI+/fvR0VFhdd9FQqF09e2n5rUdd5O4eXre6rX66HX6x1fp6eno76+Hs888wz/gEqEXq/HZ599hqamJhQXFyM3Nxfl5eVuQxF/R+XBn/eVv6fSV19fj/vuuw8ff/wxevbs6fPr+PsqXYG8p8H6XeXUhSBavHgxtm7ditLSUmi1Wo/7Dh48GCdOnHDadvLkSahUKvTv31/MMskP/rynrqSlpeHQoUMiVEaB6NGjB0aOHInx48dj9erVuPLKK/HnP//Z5b78HZUPf95XV/h7Ki2ffPIJTp48iXHjxkGlUkGlUqG8vBzr1q2DSqWC1Wrt8hr+vkpbIO+pK4H8rnJENwhsNhsWL16MLVu2oKysDMnJyV5fk56ejvfee89p28cff4zx48dDrVaLVSr5KJD31JV9+/bxIzMJs9lsOHfunMvn+DsqX57eV1f4eyot06ZNQ3V1tdO2+fPnY9SoUXjwwQehVCq7vIa/r9IWyHvqSkC/q926lY1sNpvNds8999g0Go2trKzMdvz4ccfjzJkzjn1WrFhh+9WvfuX4uq6uzta7d2/bAw88YPvqq69sr776qk2tVts2bdoUjh+BOgnkPf3Tn/5k27Jli+3gwYO2L774wrZixQobAFtxcXE4fgTqJD8/37Z9+3bb4cOHbfv377c99NBDtpiYGNvHH39ss9n4OypX/r6v/D2Vp8536PP3Vf68vafB+l3liG4QvPjiiwCAa6+91mn7xo0bkZeXBwA4fvw4TCaT47nk5GR88MEHeOCBB7B+/XoMGTIE69atw5w5c0JVNnkQyHt6/vx5LFu2DMeOHUOvXr0wevRovP/++5gxY0aoyiYPvv32W/zqV7/C8ePHodFoMGbMGPzzn/9EdnY2AP6OypW/7yt/TyMDf18jj1i/qwqb7afZ2kREREREEYQ3oxERERFRRGLQJSIiIqKIxKBLRERERBGJQZeIiIiIIhKDLhERERFFJAZdIiIiIopIDLpEREREFJEYdImIfrJ7927ceuut0Ol0iI2NxaBBg5Ceno7/9//+X7hLc8jLy8Pw4cM97rNo0SIoFAqcOHHCafv333+PmJgYqNVqnD592uk5s9kMhUKBpUuXAgBWrlwJhULhU03Dhw93NFIBgK+++gorV67EkSNHuux77bXX4oorrvDpuERE3cWgS0QE4P3338ekSZPQ3NyMp59+Gh9//DH+/Oc/IyMjA2+//Xa4y/NLVlYWAKCsrMxpe3l5OVQqFRQKBSoqKpyeKy0tdXrtXXfdhaqqqoC+/1dffYVVq1a5DLpERKHEFsBERACefvppJCcn46OPPoJKdeGfxttuuw1PP/10GCvz37XXXguFQoGysjLcdtttju1lZWWYMGECbDYbSktLccMNNzg9FxMTg6lTpwIAtFottFptyGsnIgomjugSEQE4deoUEhMTnUKuXUxM138q3377baSnp6NPnz7o27cvpk+fjn379jntk5eXh759++LLL7/EtGnT0KdPHwwYMACLFi3CmTNnnPZdv349pk6dioEDB6JPnz5ITU3F008/jdbWVr9/lv79+yM1NbXLiG5ZWRmuvfZaZGZmOkZwOz539dVXQ6PRAHA9daG1tRXLly/H4MGD0bt3b0yePBn//e9/nfYpKCjA3LlzAQijwwqFAgqFAgUFBU777dmzB1OmTEHv3r2RkpKCNWvWoL293e+flYjIEwZdIiIA6enp2L17N5YsWYLdu3d7DJhPPfUU5s2bh8svvxzvvPMOXn/9dbS0tGDKlCn46quvnPZtbW3FjBkzMG3aNJSUlGDRokX461//il/84hdO+9XW1uKXv/wlXn/9dWzbtg2//vWv8cc//hF33313QD9PVlYWjEYjjh8/DkAI8tXV1cjMzERmZiY+/fRTNDc3AwDq6+tRV1fnmLbgzm9+8xs888wzyMnJwbvvvos5c+Zg9uzZaGxsdOxz00034amnngIghPeqqipUVVXhpptucuxz4sQJ3H777bjjjjuwdetW3HjjjcjPz8cbb7wR0M9KROSWjYiIbA0NDbbJkyfbANgA2NRqtW3SpEm21atX21paWhz7mUwmm0qlsi1evNjp9S0tLbbBgwfb/vd//9exLTc31wbA9uc//9lp39///vc2ALaKigqXtVitVltra6utqKjIplQqbd9//73TMYcNG+b15ykpKbEBsL355ps2m81mKy4utqlUKltLS4utubnZplQqbdu2bbPZbDZbYWGhDYDtgw8+cLz+scces3X8E3HgwAEbANsDDzzg9H3+9re/2QDYcnNzHdv+8Y9/2ADYSktLu9SVmZlpA2DbvXu30/bLL7/cNn36dK8/FxGRPziiS0QE4eP+HTt2YM+ePVizZg1uueUWHDx4EPn5+UhNTUVDQwMA4KOPPkJbWxtycnLQ1tbmePTs2ROZmZldpgsAwO233+709S9/+UsAcJo+sG/fPvzsZz9D//79oVQqoVarkZOTA6vVioMHD/r982RmZiImJsZRT1lZGcaPH4++ffsiLi4OV199teP7l5WVQaVSYfLkyW6PZ9+388/yv//7vy6ne3gyePBgXHPNNU7bxowZg6NHj/p1HCIib3gzGhFRB+PHj8f48eMBCNMOHnzwQfzpT3/C008/jaeffhrffvstAGDChAkuX995Pq9KpUL//v2dtg0ePBiAMJ0AAEwmE6ZMmQK9Xo8///nPGD58OHr27In//ve/WLhwIc6ePev3z5GQkICrrrrKEVBLS0udpg90nKdbWlqK8ePHIy4uzu3x7LXaa/f083njav/Y2NiAfk4iIk8YdImI3FCr1Xjsscfwpz/9CV988QUAIDExEQCwadMmDBs2zOsx2tracOrUKadwZ1/f1r6tpKQEP/zwAzZv3ux0zM8++6xb9WdlZeHZZ5/F/v378eWXXzqtHpGZmYnnnnsO+/fvx5EjRzBv3jyPx7LXeuLECVx88cVdfj4iIini1AUiIsBx01ZnBw4cAAAMGTIEADB9+nSoVCrU1tY6Rn87Pzr729/+5vT1m2++CUBYBgyAY3WD2NhYxz42mw0vv/xyt34m+81lq1atQkxMjNPUBPv/XrVqldO+7thr7fyzvPPOO2hra3PaZv85OEJLROHGEV0iIggBVqvV4uabb8aoUaPQ3t6Ozz77DM8++yz69u2L++67D4DQBezxxx/Hww8/jLq6Otxwww3o168fvv32W/z3v/9Fnz59HOERAHr06IFnn30Wp0+fxoQJE7Bz5048+eSTuPHGGx1hMzs7Gz169MC8efOwfPly/Pjjj3jxxRedVjMIxNSpU6FUKrFly5YuUxMSEhJw5ZVXYsuWLVCr1cjIyPB4rMsuuwx33HEH1q5dC7Vajeuuuw5ffPEFnnnmGcTHxzvta+98tmHDBsTFxaFnz55ITk72e4oDEVF3cUSXiAjAI488gn79+uFPf/oTfvazn+HGG2/EunXrcN111+G///0vUlNTHfvm5+dj06ZNOHjwIHJzczF9+nQsX74cR48edTRcsFOr1di2bRsMBgNuueUWrFu3Dr/5zW/wj3/8w7HPqFGjUFxcjMbGRsyePRuLFy/GVVddhXXr1nXrZ4qLi8O4ceNgs9mQmZnZ5fnMzEzYbDZcc8016N27t9fjvfrqq1i6dCkKCgrws5/9DO+88w6Ki4vRr18/p/2Sk5Oxdu1afP7557j22msxYcIEvPfee936WYiIAqGw2Wy2cBdBRBSJ8vLysGnTJpw+fTrcpRARRSWO6BIRERFRRGLQJSIiIqKIxKkLRERERBSROKJLRERERBGJQZeIiIiIIhKDLhERERFFJAZdIiIiIopIDLpEREREFJEYdImIiIgoIjHoEhEREVFEYtAlIiIioojEoEtEREREEen/A0LvW702YuWOAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 800x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## Plotting the training data\n",
    "## sepal_width against sepal_length\n",
    "plt.figure(figsize=(8,6))\n",
    "\n",
    "plt.scatter(iris_train.loc[iris_train.iris_class==0].sepal_width,\n",
    "            iris_train.loc[iris_train.iris_class==0].sepal_length,\n",
    "            c='blue',\n",
    "            s=60,\n",
    "            label=\"0\")\n",
    "\n",
    "plt.scatter(iris_train.loc[iris_train.iris_class==1].sepal_width,\n",
    "            iris_train.loc[iris_train.iris_class==1].sepal_length,\n",
    "            c='orange',\n",
    "            s=60,\n",
    "            marker='v',\n",
    "            label=\"1\")\n",
    "\n",
    "plt.scatter(iris_train.loc[iris_train.iris_class==2].sepal_width,\n",
    "            iris_train.loc[iris_train.iris_class==2].sepal_length,\n",
    "            c='green',\n",
    "            s=60,\n",
    "            marker='x',\n",
    "            label=\"2\")\n",
    "\n",
    "plt.xticks(fontsize=10)\n",
    "plt.yticks(fontsize=10)\n",
    "plt.xlabel(\"Sepal Width\", fontsize=12)\n",
    "plt.ylabel(\"Sepal Length\", fontsize=12)\n",
    "plt.legend(fontsize=12)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There seems to be a clear linear bound between the blue (setosa) iris and the other two species.\n",
    "Since our perceptron model only allows for linear decision boundaries and is a binary classifier,\n",
    "let's just see if we can separate the setosa iris observations from the other two.\n",
    "\n",
    "To do so, we need to reassign any observation with iris_class = 2 to iris_class = 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal_length</th>\n",
       "      <th>sepal_width</th>\n",
       "      <th>petal_length</th>\n",
       "      <th>petal_width</th>\n",
       "      <th>iris_class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>101</th>\n",
       "      <td>5.8</td>\n",
       "      <td>2.7</td>\n",
       "      <td>5.1</td>\n",
       "      <td>1.9</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>143</th>\n",
       "      <td>6.8</td>\n",
       "      <td>3.2</td>\n",
       "      <td>5.9</td>\n",
       "      <td>2.3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>107</th>\n",
       "      <td>7.3</td>\n",
       "      <td>2.9</td>\n",
       "      <td>6.3</td>\n",
       "      <td>1.8</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>120</th>\n",
       "      <td>6.9</td>\n",
       "      <td>3.2</td>\n",
       "      <td>5.7</td>\n",
       "      <td>2.3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>133</th>\n",
       "      <td>6.3</td>\n",
       "      <td>2.8</td>\n",
       "      <td>5.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>6.2</td>\n",
       "      <td>3.4</td>\n",
       "      <td>5.4</td>\n",
       "      <td>2.3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125</th>\n",
       "      <td>7.2</td>\n",
       "      <td>3.2</td>\n",
       "      <td>6.0</td>\n",
       "      <td>1.8</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>129</th>\n",
       "      <td>7.2</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.8</td>\n",
       "      <td>1.6</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>117</th>\n",
       "      <td>7.7</td>\n",
       "      <td>3.8</td>\n",
       "      <td>6.7</td>\n",
       "      <td>2.2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>6.3</td>\n",
       "      <td>3.3</td>\n",
       "      <td>6.0</td>\n",
       "      <td>2.5</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>6.7</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.2</td>\n",
       "      <td>2.3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>131</th>\n",
       "      <td>7.9</td>\n",
       "      <td>3.8</td>\n",
       "      <td>6.4</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>128</th>\n",
       "      <td>6.4</td>\n",
       "      <td>2.8</td>\n",
       "      <td>5.6</td>\n",
       "      <td>2.1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>106</th>\n",
       "      <td>4.9</td>\n",
       "      <td>2.5</td>\n",
       "      <td>4.5</td>\n",
       "      <td>1.7</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>111</th>\n",
       "      <td>6.4</td>\n",
       "      <td>2.7</td>\n",
       "      <td>5.3</td>\n",
       "      <td>1.9</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>134</th>\n",
       "      <td>6.1</td>\n",
       "      <td>2.6</td>\n",
       "      <td>5.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>114</th>\n",
       "      <td>5.8</td>\n",
       "      <td>2.8</td>\n",
       "      <td>5.1</td>\n",
       "      <td>2.4</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>126</th>\n",
       "      <td>6.2</td>\n",
       "      <td>2.8</td>\n",
       "      <td>4.8</td>\n",
       "      <td>1.8</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>104</th>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.8</td>\n",
       "      <td>2.2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>108</th>\n",
       "      <td>6.7</td>\n",
       "      <td>2.5</td>\n",
       "      <td>5.8</td>\n",
       "      <td>1.8</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>118</th>\n",
       "      <td>7.7</td>\n",
       "      <td>2.6</td>\n",
       "      <td>6.9</td>\n",
       "      <td>2.3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>144</th>\n",
       "      <td>6.7</td>\n",
       "      <td>3.3</td>\n",
       "      <td>5.7</td>\n",
       "      <td>2.5</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>138</th>\n",
       "      <td>6.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.8</td>\n",
       "      <td>1.8</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>102</th>\n",
       "      <td>7.1</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.9</td>\n",
       "      <td>2.1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>103</th>\n",
       "      <td>6.3</td>\n",
       "      <td>2.9</td>\n",
       "      <td>5.6</td>\n",
       "      <td>1.8</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>109</th>\n",
       "      <td>7.2</td>\n",
       "      <td>3.6</td>\n",
       "      <td>6.1</td>\n",
       "      <td>2.5</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>139</th>\n",
       "      <td>6.9</td>\n",
       "      <td>3.1</td>\n",
       "      <td>5.4</td>\n",
       "      <td>2.1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>119</th>\n",
       "      <td>6.0</td>\n",
       "      <td>2.2</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1.5</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>132</th>\n",
       "      <td>6.4</td>\n",
       "      <td>2.8</td>\n",
       "      <td>5.6</td>\n",
       "      <td>2.2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>127</th>\n",
       "      <td>6.1</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.9</td>\n",
       "      <td>1.8</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>123</th>\n",
       "      <td>6.3</td>\n",
       "      <td>2.7</td>\n",
       "      <td>4.9</td>\n",
       "      <td>1.8</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>124</th>\n",
       "      <td>6.7</td>\n",
       "      <td>3.3</td>\n",
       "      <td>5.7</td>\n",
       "      <td>2.1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>110</th>\n",
       "      <td>6.5</td>\n",
       "      <td>3.2</td>\n",
       "      <td>5.1</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>105</th>\n",
       "      <td>7.6</td>\n",
       "      <td>3.0</td>\n",
       "      <td>6.6</td>\n",
       "      <td>2.1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>121</th>\n",
       "      <td>5.6</td>\n",
       "      <td>2.8</td>\n",
       "      <td>4.9</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>137</th>\n",
       "      <td>6.4</td>\n",
       "      <td>3.1</td>\n",
       "      <td>5.5</td>\n",
       "      <td>1.8</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>142</th>\n",
       "      <td>5.8</td>\n",
       "      <td>2.7</td>\n",
       "      <td>5.1</td>\n",
       "      <td>1.9</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>130</th>\n",
       "      <td>7.4</td>\n",
       "      <td>2.8</td>\n",
       "      <td>6.1</td>\n",
       "      <td>1.9</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>135</th>\n",
       "      <td>7.7</td>\n",
       "      <td>3.0</td>\n",
       "      <td>6.1</td>\n",
       "      <td>2.3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>122</th>\n",
       "      <td>7.7</td>\n",
       "      <td>2.8</td>\n",
       "      <td>6.7</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     sepal_length  sepal_width  petal_length  petal_width  iris_class\n",
       "101           5.8          2.7           5.1          1.9           2\n",
       "143           6.8          3.2           5.9          2.3           2\n",
       "107           7.3          2.9           6.3          1.8           2\n",
       "120           6.9          3.2           5.7          2.3           2\n",
       "133           6.3          2.8           5.1          1.5           2\n",
       "148           6.2          3.4           5.4          2.3           2\n",
       "125           7.2          3.2           6.0          1.8           2\n",
       "129           7.2          3.0           5.8          1.6           2\n",
       "117           7.7          3.8           6.7          2.2           2\n",
       "100           6.3          3.3           6.0          2.5           2\n",
       "145           6.7          3.0           5.2          2.3           2\n",
       "131           7.9          3.8           6.4          2.0           2\n",
       "128           6.4          2.8           5.6          2.1           2\n",
       "106           4.9          2.5           4.5          1.7           2\n",
       "111           6.4          2.7           5.3          1.9           2\n",
       "134           6.1          2.6           5.6          1.4           2\n",
       "114           5.8          2.8           5.1          2.4           2\n",
       "126           6.2          2.8           4.8          1.8           2\n",
       "104           6.5          3.0           5.8          2.2           2\n",
       "108           6.7          2.5           5.8          1.8           2\n",
       "118           7.7          2.6           6.9          2.3           2\n",
       "144           6.7          3.3           5.7          2.5           2\n",
       "138           6.0          3.0           4.8          1.8           2\n",
       "102           7.1          3.0           5.9          2.1           2\n",
       "103           6.3          2.9           5.6          1.8           2\n",
       "109           7.2          3.6           6.1          2.5           2\n",
       "139           6.9          3.1           5.4          2.1           2\n",
       "119           6.0          2.2           5.0          1.5           2\n",
       "132           6.4          2.8           5.6          2.2           2\n",
       "127           6.1          3.0           4.9          1.8           2\n",
       "123           6.3          2.7           4.9          1.8           2\n",
       "124           6.7          3.3           5.7          2.1           2\n",
       "110           6.5          3.2           5.1          2.0           2\n",
       "105           7.6          3.0           6.6          2.1           2\n",
       "121           5.6          2.8           4.9          2.0           2\n",
       "137           6.4          3.1           5.5          1.8           2\n",
       "142           5.8          2.7           5.1          1.9           2\n",
       "130           7.4          2.8           6.1          1.9           2\n",
       "135           7.7          3.0           6.1          2.3           2\n",
       "122           7.7          2.8           6.7          2.0           2"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "iris_train[iris_train.iris_class == 2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal_length</th>\n",
       "      <th>sepal_width</th>\n",
       "      <th>petal_length</th>\n",
       "      <th>petal_width</th>\n",
       "      <th>iris_class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [sepal_length, sepal_width, petal_length, petal_width, iris_class]\n",
       "Index: []"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Changing all the iris_class = 2 to class 1 so that we can use our binary classifier\n",
    "iris_train.loc[iris_train.iris_class == 2, 'iris_class'] = 1\n",
    "\n",
    "# Check that class was changed\n",
    "iris_train[iris_train.iris_class == 2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal_length</th>\n",
       "      <th>sepal_width</th>\n",
       "      <th>petal_length</th>\n",
       "      <th>petal_width</th>\n",
       "      <th>iris_class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>85</th>\n",
       "      <td>6.0</td>\n",
       "      <td>3.4</td>\n",
       "      <td>4.5</td>\n",
       "      <td>1.6</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95</th>\n",
       "      <td>5.7</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.2</td>\n",
       "      <td>1.2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101</th>\n",
       "      <td>5.8</td>\n",
       "      <td>2.7</td>\n",
       "      <td>5.1</td>\n",
       "      <td>1.9</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>143</th>\n",
       "      <td>6.8</td>\n",
       "      <td>3.2</td>\n",
       "      <td>5.9</td>\n",
       "      <td>2.3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>80</th>\n",
       "      <td>5.5</td>\n",
       "      <td>2.4</td>\n",
       "      <td>3.8</td>\n",
       "      <td>1.1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>142</th>\n",
       "      <td>5.8</td>\n",
       "      <td>2.7</td>\n",
       "      <td>5.1</td>\n",
       "      <td>1.9</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>60</th>\n",
       "      <td>5.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>130</th>\n",
       "      <td>7.4</td>\n",
       "      <td>2.8</td>\n",
       "      <td>6.1</td>\n",
       "      <td>1.9</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>135</th>\n",
       "      <td>7.7</td>\n",
       "      <td>3.0</td>\n",
       "      <td>6.1</td>\n",
       "      <td>2.3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>122</th>\n",
       "      <td>7.7</td>\n",
       "      <td>2.8</td>\n",
       "      <td>6.7</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>80 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     sepal_length  sepal_width  petal_length  petal_width  iris_class\n",
       "85            6.0          3.4           4.5          1.6           1\n",
       "95            5.7          3.0           4.2          1.2           1\n",
       "101           5.8          2.7           5.1          1.9           1\n",
       "143           6.8          3.2           5.9          2.3           1\n",
       "80            5.5          2.4           3.8          1.1           1\n",
       "..            ...          ...           ...          ...         ...\n",
       "142           5.8          2.7           5.1          1.9           1\n",
       "60            5.0          2.0           3.5          1.0           1\n",
       "130           7.4          2.8           6.1          1.9           1\n",
       "135           7.7          3.0           6.1          2.3           1\n",
       "122           7.7          2.8           6.7          2.0           1\n",
       "\n",
       "[80 rows x 5 columns]"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "iris_train[iris_train.iris_class == 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal_length</th>\n",
       "      <th>sepal_width</th>\n",
       "      <th>petal_length</th>\n",
       "      <th>petal_width</th>\n",
       "      <th>iris_class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [sepal_length, sepal_width, petal_length, petal_width, iris_class]\n",
       "Index: []"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# also be sure to make the same changes to the test set\n",
    "iris_test.loc[iris_test.iris_class == 2, 'iris_class'] = 1\n",
    "\n",
    "# Check that class was changed\n",
    "iris_test[iris_test.iris_class == 2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define our input and output for the training data\n",
    "X_train = iris_train[['sepal_length', 'sepal_width', 'petal_length', 'petal_width']].to_numpy()\n",
    "X_test = iris_test[['sepal_length', 'sepal_width', 'petal_length', 'petal_width']].to_numpy()\n",
    "\n",
    "y_train = iris_train.iris_class.to_numpy()\n",
    "y_test = iris_test.iris_class.to_numpy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The weights are:\n",
      "[-0.16 -0.46  0.81  0.43]\n",
      "The errors are:\n",
      "[3.5, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]\n",
      "The accuracy is:\n",
      "1.0\n"
     ]
    }
   ],
   "source": [
    "w_train, J_train, pred_train = perceptron(X_train, y_train, z = 0.0, eta = 0.1, num_it = 50)\n",
    "\n",
    "print(\"The weights are:\")\n",
    "print(w_train)\n",
    "\n",
    "print(\"The errors are:\")\n",
    "print(J_train)\n",
    "\n",
    "print(\"The accuracy is:\")\n",
    "print(accuracy(y_train, pred_train))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The accuracy is:\n",
      "1.0\n"
     ]
    }
   ],
   "source": [
    "y_pred = perceptron_test(X_test, w_train, z = 0.0)\n",
    "\n",
    "print(\"The accuracy is:\")\n",
    "print(accuracy(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looks like our classifier did well here, and it only seemed to need a few iterations.\n",
    "\n",
    "Let's try to apply it to just the iris class 1 and 2 observations, which do not seem to have a clear\n",
    "linear decision boundary when sepal length and width are compared.\n",
    "\n",
    "We will just have to reload the original data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Load the data\n",
    "iris12 = load_iris()\n",
    "iris12_df = pd.DataFrame(iris12['data'], columns = ['sepal_length','sepal_width','petal_length','petal_width'])\n",
    "iris12_df['iris_class'] = iris12['target']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal_length</th>\n",
       "      <th>sepal_width</th>\n",
       "      <th>petal_length</th>\n",
       "      <th>petal_width</th>\n",
       "      <th>iris_class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>50</th>\n",
       "      <td>7.0</td>\n",
       "      <td>3.2</td>\n",
       "      <td>4.7</td>\n",
       "      <td>1.4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>6.4</td>\n",
       "      <td>3.2</td>\n",
       "      <td>4.5</td>\n",
       "      <td>1.5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52</th>\n",
       "      <td>6.9</td>\n",
       "      <td>3.1</td>\n",
       "      <td>4.9</td>\n",
       "      <td>1.5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53</th>\n",
       "      <td>5.5</td>\n",
       "      <td>2.3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1.3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>54</th>\n",
       "      <td>6.5</td>\n",
       "      <td>2.8</td>\n",
       "      <td>4.6</td>\n",
       "      <td>1.5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    sepal_length  sepal_width  petal_length  petal_width  iris_class\n",
       "50           7.0          3.2           4.7          1.4           1\n",
       "51           6.4          3.2           4.5          1.5           1\n",
       "52           6.9          3.1           4.9          1.5           1\n",
       "53           5.5          2.3           4.0          1.3           1\n",
       "54           6.5          2.8           4.6          1.5           1"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Remove all the iris class 0 observations\n",
    "iris12_df = iris12_df[iris12_df.iris_class > 0]\n",
    "iris12_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Re-assign iris class 1 to 0\n",
    "iris12_df.loc[iris12_df.iris_class == 1, 'iris_class'] = 0\n",
    "\n",
    "# Re-assign iris class 2 to 1\n",
    "iris12_df.loc[iris12_df.iris_class == 2, 'iris_class'] = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Making a statified train test split\n",
    "iris12_train, iris12_test = train_test_split(iris12_df, \n",
    "                                            random_state=431,\n",
    "                                            shuffle=True,\n",
    "                                            test_size=.2,\n",
    "                                            stratify=iris12_df['iris_class'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(80, 4)\n",
      "(80,)\n"
     ]
    }
   ],
   "source": [
    "# Define our input and output for the training data\n",
    "X_train = iris12_train[['sepal_length', 'sepal_width', 'petal_length', 'petal_width']].to_numpy()\n",
    "print(X_train.shape)\n",
    "X_test = iris12_test[['sepal_length', 'sepal_width', 'petal_length', 'petal_width']].to_numpy()\n",
    "\n",
    "y_train = iris12_train.iris_class.to_numpy()\n",
    "print(y_train.shape)\n",
    "y_test = iris12_test.iris_class.to_numpy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The weights are:\n",
      "[-5.89 -5.99  8.39  5.67]\n",
      "The errors are:\n",
      "[18.0, 12.0, 12.0, 11.0, 13.5, 6.5, 5.5, 2.0, 5.5, 5.5, 6.5, 5.5, 3.5, 2.0, 3.5, 2.0, 4.5, 5.5, 2.0, 7.0, 5.5, 5.5, 4.5, 2.0, 4.5, 2.0, 4.5, 2.0, 5.5, 3.0, 5.5, 2.0, 5.0, 3.5, 5.5, 2.0, 2.0, 5.5, 3.0, 2.5, 5.0, 3.5, 2.5, 4.5, 3.0, 4.5, 5.0, 3.5, 2.5, 4.5]\n",
      "The training accuracy is:\n",
      "0.8875\n"
     ]
    }
   ],
   "source": [
    "w_train, J_train, pred_train = perceptron(X_train, y_train, z = 0.0, eta = 0.1, num_it = 50)\n",
    "\n",
    "print(\"The weights are:\")\n",
    "print(w_train)\n",
    "\n",
    "print(\"The errors are:\")\n",
    "print(J_train)\n",
    "\n",
    "print(\"The training accuracy is:\")\n",
    "print(accuracy(y_train, pred_train))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This actually seemed to do very well. Let's see if more iterations get us better training accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The weights are:\n",
      "[-14.22 -15.19  21.    19.7 ]\n",
      "The errors are:\n",
      "[18.0, 12.0, 12.0, 11.0, 13.5, 6.5, 5.5, 2.0, 5.5, 5.5, 6.5, 5.5, 3.5, 2.0, 3.5, 2.0, 4.5, 5.5, 2.0, 7.0, 5.5, 5.5, 4.5, 2.0, 4.5, 2.0, 4.5, 2.0, 5.5, 3.0, 5.5, 2.0, 5.0, 3.5, 5.5, 2.0, 2.0, 5.5, 3.0, 2.5, 5.0, 3.5, 2.5, 4.5, 3.0, 4.5, 5.0, 3.5, 2.5, 4.5, 3.0, 3.0, 4.5, 3.0, 4.5, 3.0, 4.5, 2.0, 3.5, 4.5, 3.0, 3.5, 3.5, 3.5, 3.5, 3.5, 4.5, 4.5, 4.5, 4.5, 3.0, 2.0, 4.5, 3.0, 5.5, 4.5, 4.5, 4.5, 4.5, 2.0, 4.5, 2.0, 4.5, 4.5, 4.5, 3.0, 2.0, 3.5, 3.0, 2.0, 4.5, 2.0, 4.5, 3.0, 2.0, 5.5, 2.0, 4.5, 3.0, 2.0, 4.0, 2.5, 4.5, 3.0, 2.0, 3.5, 2.0, 4.5, 2.0, 4.5, 3.0, 2.0, 4.0, 2.5, 3.0, 3.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.0, 2.5, 4.5, 2.0, 4.5, 2.0, 4.5, 3.0, 2.0, 3.5, 3.0, 2.0, 3.5, 3.0, 3.0, 2.5, 4.5, 4.0, 2.5, 4.5, 3.0, 3.5, 4.0, 4.5, 2.5, 4.5, 4.0, 2.5, 4.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 4.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 4.0, 4.5, 3.5, 4.0, 3.5, 4.0, 3.5, 4.0, 3.5, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 4.5, 3.5, 4.0, 3.5, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 4.5, 3.5, 4.0, 3.5, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.0, 3.5, 3.0, 2.5, 4.0, 3.5, 3.5, 3.0, 3.0, 3.5, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.0, 3.5, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.0, 3.5, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.0, 3.5, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.0, 3.5, 3.5, 3.0, 3.0, 3.5, 3.5, 3.5, 3.0, 3.5, 3.0, 3.0, 3.5, 3.5, 3.5, 3.0, 3.5, 3.0, 3.0, 3.5, 3.5, 3.5, 3.0, 3.5, 3.0, 3.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.5, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 4.0, 3.5, 3.0, 3.5, 4.0, 3.5, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 2.5, 3.0]\n",
      "The accuracy is:\n",
      "0.925\n"
     ]
    }
   ],
   "source": [
    "w_train, J_train, pred_train = perceptron(X_train, y_train, z = 0.0, eta = 0.1, num_it = 1000)\n",
    "\n",
    "print(\"The weights are:\")\n",
    "print(w_train)\n",
    "\n",
    "print(\"The errors are:\")\n",
    "print(J_train)\n",
    "\n",
    "print(\"The accuracy is:\")\n",
    "print(accuracy(y_train, pred_train))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The weights are:\n",
      "[-1.472 -1.514  2.081  1.966]\n",
      "The errors are:\n",
      "[18.0, 12.0, 12.0, 11.0, 13.5, 6.5, 5.5, 2.0, 5.5, 5.5, 6.5, 5.5, 3.5, 2.0, 3.5, 2.0, 4.5, 5.5, 2.0, 7.0, 5.5, 5.5, 4.5, 2.0, 4.5, 2.0, 4.5, 2.0, 5.5, 3.0, 5.5, 2.0, 5.0, 3.5, 5.5, 2.0, 2.0, 5.5, 3.0, 2.5, 5.0, 3.5, 2.5, 4.5, 3.0, 4.5, 5.0, 3.5, 2.5, 4.5, 3.0, 3.0, 4.5, 3.0, 4.5, 3.0, 4.5, 2.0, 3.5, 4.5, 3.0, 3.5, 3.5, 3.5, 3.5, 3.5, 4.5, 4.5, 4.5, 4.5, 3.0, 2.0, 4.5, 3.0, 5.5, 4.5, 4.5, 4.5, 4.5, 2.0, 4.5, 2.0, 4.5, 4.5, 4.5, 3.0, 2.0, 3.5, 3.0, 2.0, 4.5, 2.0, 4.5, 3.0, 2.0, 5.5, 2.0, 4.5, 3.0, 2.0, 4.0, 2.5, 4.5, 3.0, 2.0, 3.5, 2.0, 4.5, 2.0, 4.5, 3.0, 2.0, 4.0, 2.5, 3.0, 3.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.0, 2.5, 4.5, 2.0, 4.5, 2.0, 4.5, 3.0, 2.0, 3.5, 3.0, 2.0, 3.5, 3.0, 3.0, 2.5, 4.5, 4.0, 2.5, 4.5, 3.0, 3.5, 4.0, 4.5, 2.5, 4.5, 4.0, 2.5, 4.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 4.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 4.0, 4.5, 3.5, 4.0, 3.5, 4.0, 3.5, 4.0, 3.5, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 4.5, 3.5, 4.0, 3.5, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 4.5, 3.5, 4.0, 3.5, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.0, 3.5, 3.0, 2.5, 4.0, 3.5, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.0, 3.5, 3.0, 2.5, 4.0, 3.5, 3.5, 3.0, 3.0, 3.5, 4.0, 3.5, 3.5, 3.0, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.0, 3.5, 4.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.0, 3.5, 3.5, 3.5, 3.0, 3.5, 3.0, 3.0, 3.5, 3.5, 3.5, 3.0, 3.5, 3.0, 3.0, 3.5, 3.5, 3.5, 3.0, 3.5, 3.0, 3.0, 3.5, 3.5, 3.5, 3.0, 3.5, 3.0, 3.0, 3.5, 3.5, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 4.0, 3.5, 3.0, 3.5, 3.5, 3.0, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 2.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 3.5, 3.0, 2.5, 3.0, 3.5, 3.0, 2.5]\n",
      "The accuracy is:\n",
      "0.9375\n"
     ]
    }
   ],
   "source": [
    "w_train, J_train, pred_train = perceptron(X_train, y_train, z = 0.0, eta = 0.01, num_it = 1000)\n",
    "\n",
    "print(\"The weights are:\")\n",
    "print(w_train)\n",
    "\n",
    "print(\"The errors are:\")\n",
    "print(J_train)\n",
    "\n",
    "print(\"The accuracy is:\")\n",
    "print(accuracy(y_train, pred_train))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That is probably about as good as we will get with this simple neural network. Let's see how it does\n",
    "on our test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The accuracy is:\n",
      "0.95\n"
     ]
    }
   ],
   "source": [
    "y_pred = perceptron_test(X_test, w_train, z = 0.0)\n",
    "\n",
    "print(\"The accuracy is:\")\n",
    "print(accuracy(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All in all a pretty good showing for this single layer perceptron on the Iris data set. From here,\n",
    "a custom class could be written for this neural network and used in conjunction with some cross\n",
    "validation and parameter tuning, but we will save that for a more sophisticated algorithm in other\n",
    "notebooks in this repository."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Compare with scikit-learn Perceptron\n",
    "\n",
    "Let's compare our model to the built-in perceptron model from scikit-learn. Hopefully the weights we\n",
    "computed here line up nicely."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import Perceptron"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train the perceptron\n",
    "perc = Perceptron(random_state = None, eta0 = 0.01, shuffle = False, fit_intercept= False, max_iter=50)\n",
    "\n",
    "# fit the model\n",
    "perc.fit(X_train, y_train)\n",
    "\n",
    "y_predict = perc.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.416, -0.393,  0.631,  0.388]])"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "perc.coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.975"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "perc.score(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.95"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "perc.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'alpha': 0.0001,\n",
       " 'class_weight': None,\n",
       " 'early_stopping': False,\n",
       " 'eta0': 0.01,\n",
       " 'fit_intercept': False,\n",
       " 'l1_ratio': 0.15,\n",
       " 'max_iter': 50,\n",
       " 'n_iter_no_change': 5,\n",
       " 'n_jobs': None,\n",
       " 'penalty': None,\n",
       " 'random_state': None,\n",
       " 'shuffle': False,\n",
       " 'tol': 0.001,\n",
       " 'validation_fraction': 0.1,\n",
       " 'verbose': 0,\n",
       " 'warm_start': False}"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "perc.get_params(deep=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}