09_lin_reg_optax.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Optax without Flax ? \n",
    "\n",
    "So in notebook number 07, I showed how you can train a simple linear regression model using jax. The whole optimisation process there was done manually. Although it was just an introduction, such manual optimisation isn't really recommended for larger projects. So continuing from notebook 08, what if you don't want to use flax, write everything from scratch using jax and now want to use optax? Let's use the same code from notebook 07 and replace the manual optimisation process with optax. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import jax\n",
    "import jax.numpy as jnp\n",
    "\n",
    "masterkey = jax.random.key(0)\n",
    "masterkey, *subkeys = jax.random.split(masterkey, 10)\n",
    "\n",
    "\n",
    "\n",
    "f = lambda x: 4 * x + 3\n",
    "f(1)\n",
    "\n",
    "\n",
    "\n",
    "xs = jax.random.normal(subkeys[0], shape=(1000, ))\n",
    "ys = f(xs)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "@jax.jit\n",
    "def lin_reg_model(params, x):\n",
    "    w, b = params\n",
    "    \n",
    "    y_pred = w * x + b\n",
    "    return y_pred\n",
    "\n",
    "\n",
    "def init_params():\n",
    "    return jax.random.normal(subkeys[1], shape=(2, ))\n",
    "\n",
    "params = init_params()\n",
    "\n",
    "\n",
    "@jax.jit\n",
    "def compute_loss(params, x, y):\n",
    "    y_pred = lin_reg_model(params, x)\n",
    "    \n",
    "    # mse error\n",
    "    error = jnp.mean((y - y_pred) ** 2)\n",
    "    \n",
    "    return error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Enter the Optax"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import optax\n",
    "\n",
    "optimiser = optax.sgd(learning_rate=0.01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a look at the previous `update` function from notebook 07. \n",
    "\n",
    "```python\n",
    "@jax.jit\n",
    "def update(params, x, y, lr=0.01): \n",
    "    # get loss value and also gradients\n",
    "    values, grads = jax.value_and_grad(compute_loss)(params, x, y)\n",
    "    \n",
    "    updated_params = params - lr * grads\n",
    "    # alternative\n",
    "    # updated_params = jax.tree_map(lambda p, g: p - lr * g, params, grads)\n",
    "\n",
    "    return values, updated_params\n",
    "\n",
    "```\n",
    "\n",
    "We no longer need the manual update. Optax will do that but we also have to pass in the optimiser and the initial optimiser state. Where's the problem with that? The `jit`. Optimisers aren't jit'able. So we have to rewrite the function a bit. Actually, let's just write one training function that takes in the data, optimiser and everything else and gives us trained params. (Funny that you're always training the params and not the model and yet we call it model training. xD) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tqdm.auto import trange\n",
    "\n",
    "\n",
    "def train_model(params, x, y, optimiser, epochs=100):\n",
    "    # create the initial optimiser state\n",
    "    # well this way, every time train_model is called, the params are trained with a new optimiser state\n",
    "    optimiser_state = optimiser.init(params)\n",
    "    \n",
    "    @jax.jit\n",
    "    def update(params, x, y, optimiser_state): \n",
    "        # get loss value and also gradients\n",
    "        values, grads = jax.value_and_grad(compute_loss)(params, x, y)\n",
    "        # get updates\n",
    "        updates, optimiser_state = optimiser.update(grads, optimiser_state, params)\n",
    "        # now apply updates\n",
    "        params = optax.apply_updates(params, updates)\n",
    "\n",
    "        return values, params   \n",
    "    \n",
    "    \n",
    "    \n",
    "\n",
    "    epochs = 100\n",
    "\n",
    "    losses = list() # keep a list to plot later\n",
    "\n",
    "\n",
    "    for _ in trange(epochs):\n",
    "        values, params = update(params, xs, ys, optimiser_state)\n",
    "    \n",
    "        # append the loss\n",
    "        losses.append(values)\n",
    "        \n",
    "    \n",
    "    return losses, params"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f3df0d0eb7314f7b9f739a8cf97d4d9d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/100 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "losses, trained_params = train_model(params, xs, ys, optimiser)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As usual, let's plot the losses and compare the actual and predicted function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7ffa76bc61d0>]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAGdCAYAAAAxCSikAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA5OUlEQVR4nO3de3xU9Z3/8Ve4BCtM8IIkQhFREFRsFERMK6AiXa0XrFJsrRW17a7K1u3Wrsj2guhv1datuiJetlpqq8W2KopWQePaVi1opYpVUasmXgJEMUCCJITL9/fHGQLBBDIwkzMzeT0fj89jzpw5M/nMaWXej3O+53sKgIAkSVIW6xR3A5IkSTtiYJEkSVnPwCJJkrKegUWSJGU9A4skScp6BhZJkpT1DCySJCnrGVgkSVLW6xJ3A+nSp08f6urq4m5DkiSlIJFIsHTp0h1ulxeBpU+fPlRVVcXdhiRJ2gl9+/bdYWjJi8Cy+chK3759PcoiSVKOSCQSVFVVtem3Oy8Cy2Z1dXUGFkmS8pCDbiVJUtYzsEiSpKxnYJEkSVnPwCJJkrKegUWSJGU9A4skScp6BhZJkpT1DCySJCnrGVgkSVLWM7BIkqSsl1Jgufzyy3n++eepra2lurqaOXPmcNBBBzXbplu3btx8882sWLGCuro67rvvPnr37r3Dz54+fTpLly5l7dq1PPHEEwwcODC1byJJkvJWSoFlzJgxzJw5k6OPPppx48bRtWtXHn/8cXbfffembW644QZOPfVUvvKVrzBmzBj69OnDAw88sN3Pveyyy7jkkku48MILGTlyJJ988gnz58+nW7duO/etJElS3gk7W7169QohhDBq1KgAhKKiorBu3bpw5plnNm0zePDgEEIII0eObPVzli5dGi699NKm50VFRaG+vj6cddZZbeojkUiEEEJIJBI7/V1aqj0gXArhjjR+pmVZlmVZUaXy+71LY1h69uwJQE1NDQDDhw+nsLCQ8vLypm3eeOMN3n33XcrKylr8jAEDBrDvvvs2e09tbS3PPfdcq+8pLCwkkUg0q0zoAvwE+CawX0b+giRJaoudDiwFBQXceOONPPPMM7z66qsAlJSUsG7dOlavXt1s2+rqakpKSlr8nM3rq6ur2/yeqVOnUltb21RVVVU7+zW2awXwTHL59Iz8BUmS1BY7HVhmzpzJ0KFD+epXv5rOftrkmmuuoaioqKn69u2bsb+1efTNGRn7C5IkaUd2KrDMmDGDU045heOOO67Z0Y3ly5fTrVu3plNFmxUXF7N8+fIWP2vz+uLi4ja/p7Gxkbq6umaVKQ8mH48BemXsr0iSpO1JObDMmDGDL3/5yxx//PFUVlY2e23RokU0NjYyduzYpnUHHXQQ/fv3Z8GCBS1+XkVFBcuWLWv2nkQiwciRI1t9T3t6D1gEdAZOi7kXSZI6sjaP5p05c2ZYuXJlGD16dCguLm6q3XbbrWmbW265JVRWVoZjjz02DBs2LDz77LPh2WefbfY5S5YsCaeffnrT88suuyzU1NSEU089NQwdOjTMmTMnvP3226Fbt25pH2W8M/WfEAKER7JgRLVlWZZl5Uul+Pvd9g9uzaRJk5q26datW7j55pvDxx9/HNasWRPuv//+UFxc/KnP2fo9QJg+fXpYtmxZqK+vD0888UQYNGhQpr5wynUwUWBpgJDIgv+BLcuyLCsfKpXf74LkQk5LJBLU1tZSVFSUsfEsrwODgbOA32XkL0iS1LGk8vvtvYTayKuFJEmKj4GljeYkH78EeMMASZLal4GljV4APgASwNgdbCtJktLLwNJGgS1HWTwtJElS+zKwpGBzYDmNaF4WSZLUPgwsKfgz8DGwD/CFmHuRJKkjMbCkYCMwN7l8ZpyNSJLUwRhYUrT58uYzgYI4G5EkqQMxsKTocWA10Bcoi7kXSZI6CgNLihrZclroK3E2IklSB2Jg2Qm/Tz5OwNNCkiS1BwPLTngcqAU+Cxwdcy+SJHUEBpadsA5PC0mS1J4MLDvJ00KSJLUfA8tOmk90WqgfMDLmXiRJyncGlp20Dng4uTwhzkYkSeoADCy7YOvTQpIkKXMMLLtgPlAH9AeOirkXSZLymYFlFzQAjySXvVpIkqTMMbDsIk8LSZKUeQaWXfQYsAbYH68WkiQpUwwsu6gBeCi5/NU4G5EkKY8ZWNLg3uTjRNyhkiRlgr+vaTAfqAH6AKNj7kWSpHxkYEmD9cD9yeWvxdmIJEl5ysCSJptPC50JdI2zEUmS8pCBJU3+CCwH9gbGxduKJEl5x8CSJpuA3yWXPS0kSVJ6GVjSaHbycTzwmTgbkSQpzxhY0mghUAkkgJPjbUWSpLxiYEmzzYNvPS0kSVL6GFjSbPNpoS8BRXE2IklSHkk5sIwaNYq5c+dSVVVFCIHx48c3ez2E0GJ9//vfb/Uzp02b9qntlyxZkvq3yQIvA68BuwGnx9uKJEl5I+XA0r17dxYvXszkyZNbfL2kpKRZnX/++WzatIn777+/xe03e+WVV5q975hjjkm1tayx+bTQ2bF2IUlS/uiS6hvmzZvHvHnzWn29urq62fPx48fz1FNPUVFRsd3P3bBhw6fem6t+A1wJnACUEM3PIkmSdl5Gx7D07t2bk08+mTvvvHOH2w4aNIiqqirefvtt7r77bvr169fqtoWFhSQSiWaVTd4GFgCd8Q7OkiSlQ0YDy6RJk6irq+OBBx7Y7nbPPfcc5513HieeeCIXXXQRAwYM4Omnn6ZHjx4tbj916lRqa2ubqqqqKhPt75JfJx+/EWsXkiTlj7CzFUII48ePb/X1JUuWhJtuuinlz+3Zs2dYtWpVuOCCC1p8vbCwMCQSiabq06dPCCGERCKx098l3bUXhEYIAcIhWdCPZVmWZWVbJRKJNv9+Z+wIyzHHHMOQIUO44447Un7v6tWrefPNNxk4cGCLrzc2NlJXV9essk0N8Ghy+Zw4G5EkKQ9kLLB885vf5IUXXuDll19O+b3du3fnwAMPZNmyZRnorP1sPi30daAgzkYkScpxO3VZc2lpKaWlpQAMGDCA0tLSZoNkE4kEX/nKV1o9ulJeXt7ssujrrruO0aNH079/f8rKypgzZw4bN25k9uzZLb4/VzwCrAL2A0bH24okSTkvpfNNY8aMCS2ZNWtW0zbf/va3wyeffBKKiopa/IyKioowbdq0puezZ88OVVVVoaGhIbz//vth9uzZ4YADDsjIObD2rv8lGsdyRxb0YlmWZVnZVKn8fhckF3JaIpGgtraWoqKirBvPMhr4E7CaaE6WhnjbkSQpa6Ty++29hDLsaeBdoCdwasy9SJKUqwwsGRaAe5LLzskiSdLOMbC0g7uTjycCveJsRJKkHGVgaQdLgBeArsDXYu5FkqRcZGBpJ3clH8+PtQtJknKTgaWd/AZYBxwBlMbciyRJucbA0k5qgLnJ5fNi7EOSpFxkYGlHs5KPXycazyJJktrGwNKOHgeWAvsAJ8fciyRJucTA0o42suWGiA6+lSSp7Qws7eyXyccvAb1j7EOSpFxiYGlnrwMLgS7AOTH3IklSrjCwxGDz4FtPC0mS1DYGlhjcC9QDQ4HhMfciSVIuMLDEoBZ4ILnsURZJknbMwBKTXyYfzwZ2i7EPSZJygYElJk8CFcCewISYe5EkKdsZWGISgDuTy9+OsxFJknKAgSVGs4gmkxsNHBRzL5IkZTMDS4yWAn9ILn8rzkYkScpyBpaY3ZF8nIQ3RJQkqTUGlpg9ClQRTdM/PuZeJEnKVgaWmG1ky8y3Dr6VJKllBpYssPlqoS8C+8fYhyRJ2crAkgUqgceTy9+MsQ9JkrKVgSVL/Dz5eD7QOc5GJEnKQgaWLDEX+AjoC5wccy+SJGUbA0uWaGTL/YUujLEPSZKykYEli9yWfPwnYECcjUiSlGUMLFnkHWAe0f8o/xJzL5IkZRMDS5a5Nfn4TaBbnI1IkpRFUg4so0aNYu7cuVRVVRFCYPz45vOzzpo1ixBCs3rsscd2+LkXX3wxFRUV1NfXs3DhQkaMGJFqa3nhD8B7QC9gQsy9SJKULVIOLN27d2fx4sVMnjy51W0ee+wxSkpKmuprX/vadj9z4sSJXH/99UyfPp1hw4axePFi5s+fzz777JNqezlvI/C/yeWL4mxEkqQsE3a2Qghh/PjxzdbNmjUrzJkzJ6XPWbhwYZgxY0bT84KCgvDBBx+EKVOmtOn9iUQihBBCIpHY6e+STVUMoRFCgPC5LOjHsizLsjJRqfx+Z2QMy7HHHkt1dTWvv/46t9xyC3vttVer23bt2pXhw4dTXl7etC6EQHl5OWVlZS2+p7CwkEQi0azySTXwQHLZoyySJGVg0O28efM499xzGTt2LFOmTGHMmDE89thjdOrU8p/q1asXXbp0obq6utn66upqSkpKWnzP1KlTqa2tbaqqqqp0f43YbR58ew6QX3FMkqTUdUn3B/72t79tWn7llVd4+eWXeeeddzj22GP5v//7v7T8jWuuuYbrr7++6Xkikci70PIn4DXgEKLQcuv2N5ckKa9l/LLmiooKPvroIwYOHNji6ytWrGDDhg0UFxc3W19cXMzy5ctbfE9jYyN1dXXNKh9tDikXx9qFJEnxy3hg6du3L3vvvTfLli1r8fX169ezaNEixo4d27SuoKCAsWPHsmDBgky3l9V+BawBhgLHxtuKJEmx2qnLmktLSyktLQVgwIABlJaW0q9fP7p3785Pf/pTRo4cSf/+/Tn++ON56KGHeOutt5g/f37TZ5SXlze7LPr666/n29/+Nueeey5Dhgzh1ltvpXv37syaNSsNXzF31QJ3JZcvibMRSZKyQEqXII0ZMya0ZNasWWG33XYL8+bNC9XV1WHdunWhoqIi3H777aF3797NPqOioiJMmzat2brJkyeHysrK0NDQEBYuXBiOOuqojFwWlWs1hOjy5g0Q9s+CfizLsiwrXZXK73dBciGnJRIJamtrKSoqysvxLPOIboh4HXBZzL1IkpQuqfx+ey+hHDAj+fgtYPc4G5EkKSYGlhzwKPAWsCfRJc6SJHU0BpYcEICbk8sOvpUkdUQGlhwxi+gS50OB42PuRZKk9mZgyRG1wC+Tyx5lkSR1NAaWHLL5tNCpwIA4G5EkqZ0ZWHLIG0SXOHcCvhNzL5IktScDS465Mfn4LaAoxj4kSWpPBpYcMx94FUgA3465F0mS2ouBJQddn3z8N6BLnI1IktRODCw56B6gGugHTIi5F0mS2oOBJQetY8sVQ5fG2YgkSe3EwJKjbgXqgSOB0TH3IklSphlYctTHwF3JZY+ySJLynYElh92QfDwNGBRnI5IkZZiBJYe9CcxNLv97nI1IkpRhBpYc97Pk43lArxj7kCQpkwwsOe7PwF+Bz+B0/ZKk/GVgyQPXJh//FegRZyOSJGWIgSUPPEh0Y8S9cLp+SVJ+MrDkgU3AT5PLlwKFMfYiSVImGFjyxN1AFdAX+HrMvUiSlG4GljzRyJabIl4GFMTYiyRJ6WZgySP/C6wEhgDjY+5FkqR0MrDkkTXAzOTy5XE2IklSmhlY8sxNRDdFHAkcG28rkiSljYElz3wE3Jlc/kGcjUiSlEYGljx0HbAeOAE4OuZeJElKBwNLHnoP+FVy+UdxNiJJUpoYWPLUNcBG4EvA8Jh7kSRpVxlY8tTbwD3J5R/G2YgkSWlgYMljVxNN2386cFi8rUiStEtSDiyjRo1i7ty5VFVVEUJg/PgtU5R16dKFa6+9lpdffpk1a9ZQVVXFXXfdxb777rvdz5w2bRohhGa1ZMmS1L+NmnkD+F1y2aMskqRclnJg6d69O4sXL2by5Mmfem333Xdn2LBhXHXVVQwbNowzzjiDwYMHM3fu3B1+7iuvvEJJSUlTHXPMMam2phb8V/JxAtEMuJIk5aIuqb5h3rx5zJs3r8XXamtr+eIXv9hs3b/+67/y17/+lX79+vH++++3+rkbNmyguro61Xa0A68ADwBnEM3L8o1425EkaadkfAxLz5492bRpE6tWrdrudoMGDaKqqoq3336bu+++m379+rW6bWFhIYlEolmpdf8v+fg1YFCcjUiStJMyGli6devGT37yE2bPnk1dXV2r2z333HOcd955nHjiiVx00UUMGDCAp59+mh49erS4/dSpU6mtrW2qqqqqTH2FvPAiMBfoDEyLuRdJknZW2NkKIYTx48e3+FqXLl3CQw89FBYtWhQSiURKn9uzZ8+watWqcMEFF7T4emFhYUgkEk3Vp0+fEEJI+e90pDocQoCwEcLBWdCPZVmWZSUSiTb/fmfkCEuXLl343e9+R//+/Rk3btx2j660ZPXq1bz55psMHDiwxdcbGxupq6trVtq+l4jGsnQCroi1E0mSUpf2wLI5rAwaNIgTTjiBmpqalD+je/fuHHjggSxbtizd7XVo04jmZZkIfC7mXiRJSsVOXdZcWlpKaWkpAAMGDKC0tJR+/frRpUsX7rvvPo488ki+/vWv07lzZ4qLiykuLqZr165Nn1FeXt7ssujrrruO0aNH079/f8rKypgzZw4bN25k9uzZafiK2uwVtszLckWMfUiStDNSOt80ZsyY0JJZs2aF/v37t/haCCGMGTOm6TMqKirCtGnTmp7Pnj07VFVVhYaGhvD++++H2bNnhwMOOCAj58A6eg2BsIFoPMuwLOjHsizL6riVyu93QXIhpyUSCWpraykqKnI8Sxv8img+lkeAU2PuRZLUcaXy++29hDqgK4ENwCnAyJh7kSSpLQwsHdBbREdZYMukcpIkZTMDSwd1JdAInACMjbkXSZJ2xMDSQb0L3JpcvibORiRJagMDSwf2X8AaYARwZsy9SJK0PQaWDuwj4GfJ5f8iuteQJEnZyMDSwf2MKLgMBs6LtxVJklplYOng6oiOrkA0++1u8bUiSVKrDCziNqJBuJ8F/jXmXiRJaomBRawjujEiwFRgj/hakSSpRQYWAfBr4FVgL6LQIklSNjGwCIBNwGXJ5X8D+sfYiyRJ2zKwqMmjwJNAN+DqmHuRJGlrBhY1cynR0ZaziSaUkyQpGxhY1MxittwY8Wfb21CSpHZkYNGn/BCoB0YBp8fbiiRJgIFFLahiy9GVnwJdY+xFkiQwsKgVPwGqgUHAhTH3IkmSgUUtWgP8OLl8BdH8LJIkxcXAolbdCbxMFFamx9yLJKljM7CoVRuB7yaXLwKGxteKJKmDM7Bou54C7gc6AzfG24okqQMzsGiHvg80AGPxMmdJUjwMLNqhSuC/k8s/I5q6X5Kk9mRgUZtcA3wAHEA0fb8kSe3JwKI2WcuWuzn/J9A3xl4kSR2PgUVtNht4FujOllNEkiS1BwOLUjKZ6HLnrxINwpUkqT0YWJSSxcDM5PJMoDDGXiRJHYeBRSn7EbAcGIwDcCVJ7cPAopTVEs3NAvBDoH+MvUiSOoaUA8uoUaOYO3cuVVVVhBAYP378p7aZPn06S5cuZe3atTzxxBMMHDhwh5978cUXU1FRQX19PQsXLmTEiBGptqZ2dA/wR2B3nAFXkpR5KQeW7t27s3jxYiZPntzi65dddhmXXHIJF154ISNHjuSTTz5h/vz5dOvW+nRjEydO5Prrr2f69OkMGzaMxYsXM3/+fPbZZ59U21M7mgysJ5r99kvxtiJJ6gDCzlYIIYwfP77ZuqVLl4ZLL7206XlRUVGor68PZ511Vqufs3DhwjBjxoym5wUFBeGDDz4IU6ZMaVMfiUQihBBCIpHY6e9i7Vz9BEKA8DaEz2RBP5ZlWVbuVCq/32kdwzJgwAD23XdfysvLm9bV1tby3HPPUVZW1uJ7unbtyvDhw5u9J4RAeXl5q+8pLCwkkUg0K8XjSuA9ohlwp8XciyQpf6U1sJSUlABQXV3dbH11dXXTa9vq1asXXbp0Sek9U6dOpba2tqmqqqrS0L12xidEp4YgumLoczH2IknKXzl5ldA111xDUVFRU/Xt60TxcXoE+D3QBfg5Ofp/KklSVkvrb8vy5csBKC4ubra+uLi46bVtrVixgg0bNqT0nsbGRurq6pqV4nUJsAo4ii1HXCRJSpe0BpaKigqWLVvG2LFbJm1PJBKMHDmSBQsWtPie9evXs2jRombvKSgoYOzYsa2+R9lnOTAluXw10C/GXiRJ+SmlEb3du3cPpaWlobS0NIQQwne/+91QWloa+vXrF4Bw2WWXhZqamnDqqaeGoUOHhjlz5oS33347dOvWrekzysvLw+TJk5ueT5w4MdTX14dzzz03DBkyJNx2222hpqYm9O7dO+2jjK3MVQGEp4muGnooC/qxLMuysrtS/P1O7cPHjBkTWjJr1qymbaZPnx6WLVsW6uvrwxNPPBEGDRrU7DMqKirCtGnTmq2bPHlyqKysDA0NDWHhwoXhqKOOytQXtjJYB0NYRxRaJmRBP5ZlWVb2Viq/3wXJhZyWSCSora2lqKjI8SxZ4AqiS5w/BA4BPo61G0lStkrl99sLOpR2VwOvAL2Bm2LuRZKUHwwsSrtG4HxgI3A2cFq87UiS8oCBRRnxAnBdcvk2YM8Ye5Ek5T4DizLmCmAJsC9wQ7ytSJJynIFFGbMOuADYBEzCOzpLknaegUUZtZAtR1duB/aIrxVJUg4zsCjjfgS8CXwWmBFzL5Kk3GRgUcbVA98gumroHGBCvO1IknKQgUXt4nmi+VkgumqoJMZeJEm5x8CidnMV8Ddgb+COmHuRJOUWA4vazXqiU0MNwMnAt+NtR5KUQwwsalevAf+ZXL4eOCDGXiRJucPAonZ3I/BHoAdwD9AlzmYkSTnBwKJ2F4gmklsFHA38ONZuJEm5wMCiWLwH/HNy+QfA6Bh7kSRlPwOLYvN74BdE/ye8G2fBlSS1zsCiWF1CNAtuP+B/Y+5FkpS9DCyK1SfA2USXPH+F6GaJkiRty8Ci2C0iGscCcBNwcIy9SJKyk4FFWeG/gSeA7sDvgN3jbUeSlGUMLMoKgejGiMuAoXhXZ0lScwYWZY0PicazbCQay3JuvO1IkrKIgUVZ5Y/AFcnlW3A8iyQpYmBR1rkaeJxoPMvvcTyLJMnAoiy0iWg8y1LgUODWeNuRJGUBA4uy0kfA14ANRGNZLoq3HUlSzAwsylp/Bi5PLt9IdKNESVLHZGBRVvsZ0TiWQuA+oHe87UiSYmJgUda7AHgN6Av8FugcbzuSpBgYWJT11gBnAHXAscC1sXYjSYqDgUU54Q3gvOTy94GvxteKJCkGBhbljAeAa5LLvwCGxdiLJKl9pT2wVFRUEEL4VN18880tbj9p0qRPbVtfX5/utpQnfgg8AnwGeAgojrcdSVI76ZLuDxwxYgSdO28ZFjl06FDKy8v5/e9/3+p7Vq9ezeDBg5uehxDS3ZbyxCbg68BComn7HwCOAxrjbEqSlHFpDywrVqxo9vzyyy/nrbfe4k9/+lOr7wkhUF1dne5WlKdqgdOA54HPA7cRXUkkScpfGR3D0rVrV8455xx+8YtfbHe7Hj16UFlZyXvvvceDDz7IIYccst3tCwsLSSQSzUody1vAWUR3dj4f+G6s3UiSMi2jgeX0009njz324Je//GWr27zxxhtccMEFjB8/nnPOOYdOnTrxl7/8hb59+7b6nqlTp1JbW9tUVVVVGehe2e4J4NLk8n8DJ8fYiyQpswqAjA0YmTdvHo2NjZx22mltfk+XLl1YsmQJs2fP5sc//nGL2xQWFtKtW7em54lEgqqqKoqKiqirq9vlvpVbbgf+mWi+lmOAxfG2I0lqo0QiQW1tbZt+v9M+hmWz/fbbjxNOOIEzzjgjpfdt2LCBF198kYEDB7a6TWNjI42NDrNUZDIwABhHdAXRSKI7PUuS8kfGTgmdf/75fPjhh/zhD39I6X2dOnXisMMOY9myZRnqTPlmA/AVoun7Pws8DHSPtSNJUrplJLAUFBRw/vnnc9ddd7Fx48Zmr911111cffXVTc9/9KMfMW7cOAYMGMARRxzB3XffTf/+/bnjjjsy0Zry1GqiMSwfEk0odw/OiihJ+SQjp4ROOOEE+vfv3+LVQfvttx+bNm1qer7nnnvy85//nJKSElauXMmiRYv4/Oc/z5IlSzLRmvJYJTAeeCr5eCNwSYz9SJLSJ6ODbttLKoN2lP8mEN3VuRMwBfhpvO1IklqRyu+3R82Vd+4D/j25/BPgnBh7kSSlh4FFeekm4Lrk8i+AE2LsRZK06wwsyltTgN8AXYnuOXREvO1IknaBgUV5KxBN2/8kkAAeA1qf3UeSlM0MLMprjcCXgReBYqLp/PvE2pEkaWcYWJT36oATgTeB/YlCy95xNiRJSpmBRR3Ch0RT978PHAI8CvSItSNJUioMLOow3iMKLR8BRwEPAd22+w5JUrYwsKhDeYPo9FAtcDzRnC1dY+1IktQWBhZ1OH8DTgXWAqcA95LB25ZLktLCwKIO6c9E9xtqAM4Afo3/MUhSNvPfaHVY5URhpRH4KtGMuAWxdiRJao2BRR3aY8BEYAMwCbgdQ4skZSMDizq8h4CzgY3At4GfY2iRpGxjYJGA3xPd1Xkj8E2i00P+xyFJ2cN/k6Wke4GvEZ0eOg/4Jf4HIknZwn+Ppa38HjgLWA98g+jqoc6xdiRJAgOL9CkPEA3EXU80tuVeoDDWjiRJBhapBQ8CZwLrgAlEA3M/E2dDktTBGVikVjwMnAx8QjSd/zygKNaOJKnjMrBI2/Ek0Q0TVwGjgf8D9o6zIUnqoAws0g4sAI4jusvzcKJp/fvG2pEkdTwGFqkNXgJGAR8AhwB/AYbE2ZAkdTAGFqmN3gC+ALwO7Ac8Axwda0eS1HEYWKQUvAccAywkGsvyJPClWDuSpI7BwCKl6GNgLPAosDvRJc/nx9qRJOU/A4u0E9YC44G7gC5E9x66MtaOJCm/GViknbT5nkNXJZ//iGgqf2fFlaT0M7BIu+jHwAVEU/mfAzwO7BlrR5KUfwwsUhrMIhp8uxoYQ3TZ84GxdiRJ+cXAIqVJOdEVRO8RzdHyPNGEc5KkXZf2wDJt2jRCCM1qyZIl233PhAkTWLJkCfX19bz88sucdNJJ6W5LahevAEcRXfa8FzAf+JdYO5Kk/JCRIyyvvPIKJSUlTXXMMce0um1ZWRmzZ8/mzjvv5IgjjuDBBx/kwQcf5NBDD81Ea1LGVQPHAncDXYHbgJuAzjH2JEn5IKSzpk2bFl588cU2b3/vvfeGhx9+uNm6BQsWhFtvvbXNn5FIJEIIISQSibR+F8va1bocQkhWOYS9s6Any7KsbKlUfr8zcoRl0KBBVFVV8fbbb3P33XfTr1+/VrctKyujvLy82br58+dTVlbW6nsKCwtJJBLNSspG1wKnA2uIJpt7ATgizoYkKUelPbA899xznHfeeZx44olcdNFFDBgwgKeffpoePXq0uH1JSQnV1dXN1lVXV1NSUtLq35g6dSq1tbVNVVVVldbvIKXTQ0T3HPoHsD/wLNHlz5Kk1GT0cE/Pnj3DqlWrwgUXXNDi6+vWrQtf/epXm6276KKLwvLly1v9zMLCwpBIJJqqT58+nhKysr56QniELaeI/gdClyzoy7IsK66K/ZTQ1lavXs2bb77JwIEDW3x9+fLlFBcXN1tXXFzM8uXLW/3MxsZG6urqmpWU7VYDp7JlCv9LgD8CfeNqSJJySMYDS/fu3TnwwANZtmxZi68vWLCAsWPHNls3btw4FixYkOnWpHYXgGlE9yFaBXwBeBEYF2NPkpQr0np457rrrgujR48O/fv3D2VlZeHxxx8PH374YejVq1cAwl133RWuvvrqpu3LyspCY2Nj+N73vhcGDx4cpk2bFtatWxcOPfTQjBxSsqxsqQMgLCI6PbQRwhUQOmVBX5ZlWe1VKf5+p/ePz549O1RVVYWGhobw/vvvh9mzZ4cDDjig6fWnnnoqzJo1q9l7JkyYEF5//fXQ0NAQ/v73v4eTTjopk1/YsrKmukG4lS3jWp6AUJIFfVmWZbVHpfL7XZBcyGmJRILa2lqKioocz6Kc9HXgdqA78CEwCZgXa0eSlHmp/H57LyEpC9wDDAdeAnoDjwH/TTRTriTJwCJljTeI5mu5Kfn8UqK7Prd8fZ0kdSwGFimLrAP+DTgN+Bg4kuioyz/H2JMkZQMDi5SFHgZKgXKicS23J9f1jrMpSYqRgUXKUlXAF4F/BxqAU4BXiOZwkaSOxsAiZbEA3Eh0amgxsA/wIPArYM/YupKk9mdgkXLAq8BRwE+AjcA3iI62nBxnU5LUjgwsUo5oBC4nms7/daAP8AgwC9gjvrYkqV0YWKQc8xxwBNE8LZuA84DXgDNi7EmSMs3AIuWgBuA/gGOIjrbsC9yfrH1j7EuSMsXAIuWwBcDhwFXAeqKjLEuI5m0piK8tSUo7A4uU49YBPyaa2v95oCfRvC3PAIfF2JckpZOBRcoTfwfKgO8CdcDngb8RjXXpEV9bkpQWBhYpj2wC/gc4GLgP6EJ0T6IlwIQY+5KkXWVgkfJQFfAV4CTgHeCzwO+Jpvo/JMa+JGlnGVikPDYPOBSYTnRl0ViiGXOvB4pi7EuSUmVgkfJcA3AF0WmiOUSnif4deBP4NtA5ts4kqe0MLFIHUUl02fMXica0FAP/C7wInBBfW5LUJgYWqYN5AvgccAlQQ3Tp8xNE0/wfHGNfkrQ9BhapA9oAzAAGAjcQTTp3MtGl0T8H+sbXmiS1yMAidWArge8RDcx9gGg8y7eAfwDXEE1CJ0nZwMAiiX8AZxJNNvcM8BmiO0O/A1wG7B5fa5IEGFgkbWUBMAo4jegO0HsBPwHeBr4DFMbXmqQOzsAi6VMeJhqMOwmoAEqAm4iOxHwL6Bpfa5I6KAOLpBZtAn4FDAYuJJo9dz+iQbn/ILojtMFFUnsxsEjarvVEd38eSHRjxWVA/+S6t4jCjKeKJGWagUVSmzQQ3VjxAODfgKVER1xuJRqc+z2ge2zdScp3BhZJKWkgGs9yINFA3A+I5m35GfAuMI1osK4kpZOBRdJOaQBuJjri8k2iexPtTXTfoveIjsbsH1NvkvKPgUXSLlkP/IJoWv+JRPcm6k409f9bwGxgeGzdScoXBhZJabEJ+D0wDBgHzCeaOferwAvAn4Av4z86knaO/3ZISrty4ESglOjS6PXAaKLp/98mGqDrtP+SUpH2wHL55Zfz/PPPU1tbS3V1NXPmzOGggw7a7nsmTZpECKFZ1dfXp7s1Se3sZaLJ5/YH/gtYkVz+GdFg3VuBoTH1Jim3pD2wjBkzhpkzZ3L00Uczbtw4unbtyuOPP87uu2//biSrV6+mpKSkqfr375/u1iTFZCnwQ6Af0Uy5fwd6EM3h8nei00UTcSI6SdsXMlm9evUKIYQwatSoVreZNGlSWLly5U7/jUQiEUIIIZFIZPS7WJaVvhoN4bcQ1kMIyVoO4WoIB2RBf5ZlZb5S+f3O+BiWnj17AlBTU7Pd7Xr06EFlZSXvvfceDz74IIccckir2xYWFpJIJJqVpNzyZ+AsosnnriA6ClMMTCUa5/I4MAFn0ZW0RcaSU0FBQXj44YfD008/vd3tjj766PCNb3wjlJaWhtGjR4e5c+eGVatWhb59+7a4/bRp00JLPMJiWblbXSCcDuFRCBvZctTlIwjXQxiaBT1alpXeSvEMSeYaueWWW0JFRUWrwaO16tKlS/jHP/4RrrzyyhZfLywsDIlEoqn69OljYLGsPKr9Ifw/CFVsCS4BwvMQLoawVxb0aFnWrldWnBKaMWMGp5xyCscddxxVVVUpvXfDhg28+OKLDBw4sMXXGxsbqaura1aS8kcl0SDd/YAvAfcTXRo9AphJdAPG+4HTcKCu1FFkJLDMmDGDL3/5yxx//PFUVlam/P5OnTpx2GGHsWzZsvQ3JylnbAQeIxrL0gf4d+BvRONazgAeIhr7cgvwBaAgnjYltZO0Ht6ZOXNmWLlyZRg9enQoLi5uqt12261pm7vuuitcffXVTc9/9KMfhXHjxoUBAwaEI444IvzmN78Ja9euDQcffHDaDylZlpX7NRTCT/j0KaNKCNdAKM2CHi3L2nHFOoalNZMmTWra5qmnngqzZs1qen799deHysrK0NDQEJYtWxYeeeSRcPjhh2fqC1uWlSfVCcJYCHdCWEXz8PI6hCtxsK5lZXOl8vtdkFzIaYlEgtraWoqKihzPInVQuxGNdzk7+fiZrV57jWjMy/3A4vZvTVIrUvn99l5CkvJCA9G9iiYAvYmCy0PAOuAQ4EfAS0R3kP4JMBLHvEi5xMAiKe+sAWYDpxOFl3OAOUA9cCBwGbAQqAJuA04CusXRqKQ2M7BIymu1wD1EVxXtA3wFuDe5fl/gX4BHgY+A+4DziEKOpOziGBZJHVJX4FiiozCnAZ/d5vXngD8QXVa9iDz4h1LKQqn8fhtYJAkYDpwMnEI0Qd3WPgLmA/OI7nH0Ufu2JuUtA4sk7YISovByEnAC0HOb118kCi6PA88SDeyVlDoDiySlSRegjCi8nAQcvs3r9cAzwJPJ+huwqR37k3KZgUWSMqQ30VGXccAXiW4ZsLWVwJ+APybrZfLgH1kpQwwsktRODgbGJutYYI9tXq8Bngb+nHx8EdjQfu1JWc3AIkkx6AQMIwouxwKjgKJttvkEWEB0GulZovlg1rRbh1J2MbBIUhboDBzBlvByDLDXNttsJDpt9BeiILMAeKf9WpRiZWCRpCxUABxKFF6+kKz9W9juI6IjLwuB54EXgFXt0qHUvgwskpQj+gCfT9bRRKeUWrpNwJtsCS8vEI2FWdtOPUqZYmCRpBxVSHQaqYxoArujgIEtbLcRWEI0C++LRJdTvwT4L6BySSq/313aqSdJUhs0Et0W4Lmt1u0NHEkUXoYnl/sCQ5M1aatt3yIKMIuJAsxi4INMNy21AwOLJGW5j4luDTB/q3UlROFlWLKOAPoTHY0ZSHSTx81qiAb2vgz8Pfn4Gl6dpNxiYJGkHLSc6OaMf9hq3V5EweVwoDT5eHBy/bHJ2loF8CrwSvLxNeB1HBuj7GRgkaQ8UcOWWwRs1o0otBwGfG6rx32BAck6ZZvPqSQaH7OEKMBsLm/6qDgZWCQpj60jGsvy0jbr9yK6xPpQonEwhxIFm2KiS633J7p30tZqgDeIrljaut7CozLKPAOLJHVAm28Z8PQ26/cmCi6HAIOBIcnanyjklCVrW0uBfyTr7W1qddq7V0dkYJEkNfmY6LYBz2yzfjdgEHBQsgYnaxBRyOmTrDEtfOYKovEy72xVFUSnnt4nujJK2hEDiyRphxqIrjD6ewuv7Ul0ZdKgZB24VRUDvZI1ooX3biI6OvNuK/U+Xs2kiIFFkrRLVgJ/Tda2ehAN7D0gWQOIgsz+ydod+GyyvrCdz3+PKLy8TzSvzAfJ5apkfZKOL6KsZmCRJGXMGlo/MgOwD1vCS/9tqh/R0ZvNVbqdv7OKLeFl6Ta1bKvy9FPuMrBIkmLzUbJaOjoDkCAKLv2A/dhyNOazyXV9gSJgj2QduoO/9zFRcFm+TVVvUyuIbn+g7GFgkSRlrTqiCe1e2842PYiCy2eTj/sSDQDuu9XzEqI5afZO1tAd/N1NROHmw61qc7jaevkjonBTgwEn0wwskqSctoZofpg3drDdXkThZV+iwcAlW1XxVo+9gE5Ep6v2YcdHbTarIQovH2/1uHXVbPW4uRx703YGFklSh7A5JLy6g+06E4WWfYDe29Q+21QvoiM2EAWivVLsqZFoUHFrtWqrx61rdbI60lEdA4skSVvZyJaxLG3RmWhQ8NYBZu9tljeHmc3P9yQ6RVVIdFSneCd7XUMUXGrZEmI2P6/darluq3V1LVQuDEY2sEiStAs2Ep0CWpHi+3YnCi6bA8y2tUeyNj/vmaw9iMbtkHzcPIZnVzSyJbys2WZ565qyi39nVxhYJEmKwdpkVe3Ee7sQXR3Vs4Uq2uYxkVze/Lh5OUEUmiA60rP56E9r6snTwHLxxRfzH//xH5SUlLB48WK+853v8Ne/tnbhGkyYMIGrrrqK/fffn3/84x9MmTKFxx57LFPtSZKUszawZUzOruhMFFx6sCXE9NjqcevqtIt/Kx1CumvixImhoaEhnHfeeeHggw8Ot99+e6ipqQn77LNPi9uXlZWF9evXh+9///thyJAh4corrwzr1q0Lhx56aJv+XiKRCCGEkEgk0v5dLMuyLMvKTKX4+53+BhYuXBhmzJjR9LygoCB88MEHYcqUKS1uf++994aHH3642boFCxaEW2+9NRNf2LIsy7KsLKhUfr/TfoSna9euDB8+nPLy8qZ1IQTKy8spK2vppuRQVlbWbHuA+fPnt7p9YWEhiUSiWUmSpPyV9sDSq1cvunTpQnV18wvCqqurKSkpafE9JSUlKW0/depUamtrm6qqameGLEmSpFyRDWNoUnbNNddQVFTUVH377uoFXZIkKZul/SqhFStWsGHDBoqLm0+DU1xczPLly1t8z/Lly1PavrGxkcbGXJjmRpIkpUPaj7CsX7+eRYsWMXbs2KZ1BQUFjB07lgULFrT4ngULFjTbHmDcuHGtbi9JkjqetI/6nThxYqivrw/nnntuGDJkSLjttttCTU1N6N27dwDCXXfdFa6++uqm7cvKykJjY2P43ve+FwYPHhymTZvmZc2WZVmWlecV+2XNQJg8eXKorKwMDQ0NYeHCheGoo45qeu2pp54Ks2bNarb9hAkTwuuvvx4aGhrC3//+93DSSSdl6gtblmVZlpUFlcrvd0FyIaclEglqa2spKiqirq4u7nYkSVIbpPL7nZNXCUmSpI7FwCJJkrKegUWSJGU9A4skScp6aZ84Lk7eU0iSpNyRyu92XgSWzV/YewpJkpR7EonEDq8SyovLmgH69OmTkUuaE4kEVVVV9O3b10umM8x93X7c1+3Hfd1+3NftJ537OpFIsHTp0h1ulxdHWIA2fdldUVdX538A7cR93X7c1+3Hfd1+3NftJx37uq3vd9CtJEnKegYWSZKU9QwsO7Bu3TquuOIK1q1bF3crec993X7c1+3Hfd1+3NftJ459nTeDbiVJUv7yCIskScp6BhZJkpT1DCySJCnrGVgkSVLWM7DswMUXX0xFRQX19fUsXLiQESNGxN1STrv88st5/vnnqa2tpbq6mjlz5nDQQQc126Zbt27cfPPNrFixgrq6Ou677z569+4dU8f5Y8qUKYQQuOGGG5rWua/Tp0+fPvz6179mxYoVrF27lpdffpnhw4c322b69OksXbqUtWvX8sQTTzBw4MCYus1tnTp14sorr+Sdd95h7dq1vPXWW/zwhz/81Hbu79SNGjWKuXPnUlVVRQiB8ePHf2qbHe3XPffck7vvvpvVq1ezcuVK7rjjDrp3756W/oLVck2cODE0NDSE8847Lxx88MHh9ttvDzU1NWGfffaJvbdcrcceeyxMmjQpHHLIIeFzn/tceOSRR0JlZWXYfffdm7a55ZZbwrvvvhuOO+64MGzYsPCXv/wlPPPMM7H3nst15JFHhnfeeSe89NJL4YYbbnBfp7n22GOPUFFREX7xi1+EESNGhP333z+MGzcuHHDAAU3bXHbZZWHlypXhtNNOC4cddlh48MEHw9tvvx26desWe/+5VlOnTg0fffRR+NKXvhT69+8fzjzzzFBbWxu+853vuL93sU488cRw1VVXhdNPPz2EEML48eObvd6W/froo4+GF198MRx11FHhC1/4QnjzzTfDPffck47+4t9B2VoLFy4MM2bMaHpeUFAQPvjggzBlypTYe8uX6tWrVwghhFGjRgUgFBUVhXXr1oUzzzyzaZvBgweHEEIYOXJk7P3mYnXv3j288cYbYezYseGpp55qCizu6/TVNddcE/785z9vd5ulS5eGSy+9tOl5UVFRqK+vD2eddVbs/edaPfzww+GOO+5otu6+++4Lv/71r93faayWAsuO9uuQIUNCCCEMHz68aZt/+qd/Chs3bgz77rvvLvXjKaFWdO3aleHDh1NeXt60LoRAeXk5ZWVlMXaWX3r27AlATU0NAMOHD6ewsLDZfn/jjTd499133e87aebMmfzhD3/gySefbLbefZ0+p512Gi+88AK/+93vqK6u5m9/+xvf+ta3ml4fMGAA++67b7N9XVtby3PPPee+3gl/+ctfGDt2LIMGDQLgc5/7HMcccwyPPfYY4P7OlLbs17KyMlauXMmiRYuatikvL2fTpk2MHDlyl/5+3tz8MN169epFly5dqK6ubra+urqaIUOGxNRVfikoKODGG2/kmWee4dVXXwWgpKSEdevWsXr16mbbVldXU1JSEkebOe2ss85i2LBhLY69cl+nzwEHHMBFF13E9ddfz9VXX82IESO46aabaGxs5Fe/+lXT/mzp3xP3dequvfZaioqKeP3119m4cSOdO3fmBz/4Ab/5zW8A3N8Z0pb9WlJSwocfftjs9Y0bN1JTU7PL+97AotjMnDmToUOHcswxx8TdSl767Gc/y//8z/8wbtw4pyrPsE6dOvHCCy/wgx/8AICXXnqJoUOHcuGFF/KrX/0q5u7yz8SJE/n617/O2Wefzauvvsrhhx/OjTfeyNKlS93fecxTQq1YsWIFGzZsoLi4uNn64uJili9fHlNX+WPGjBmccsopHHfccVRVVTWtX758Od26dWs6VbSZ+z11w4cPp7i4mL/97W+sX7+e9evXc+yxx3LJJZewfv16qqur3ddpsmzZMl577bVm65YsWcJ+++0H0LQ//fckPa677jquvfZafvvb3/LKK69w9913c8MNNzB16lTA/Z0pbdmvy5cv/9SVhp07d2avvfba5X1vYGnF+vXrWbRoEWPHjm1aV1BQwNixY1mwYEGMneW+GTNm8OUvf5njjz+eysrKZq8tWrSIxsbGZvv9oIMOon///u73FD355JMMHTqUww8/vKn++te/cs8993D44YfzwgsvuK/T5Nlnn2Xw4MHN1h100EG8++67AFRUVLBs2bJm+zqRSDBy5Ej39U7Yfffd2bRpU7N1GzdupFOn6CfN/Z0ZbdmvCxYsYM8992TYsGFN2xx//PF06tSJ5557bpd7iH0kcrbWxIkTQ319fTj33HPDkCFDwm233RZqampC7969Y+8tV2vmzJlh5cqVYfTo0aG4uLipdtttt6ZtbrnlllBZWRmOPfbYMGzYsPDss8+GZ599Nvbe86G2vkrIfZ2+OvLII0NjY2OYOnVqOPDAA8PXvva1sGbNmnD22Wc3bXPZZZeFmpqacOqpp4ahQ4eGOXPmeJntTtasWbPC+++/33RZ8+mnnx4+/PDDcO2117q/d7G6d+8eSktLQ2lpaQghhO9+97uhtLQ09OvXr8379dFHHw2LFi0KI0aMCJ///OfDG2+84WXN7VGTJ08OlZWVoaGhISxcuDAcddRRsfeUy9WaSZMmNW3TrVu3cPPNN4ePP/44rFmzJtx///2huLg49t7zobYNLO7r9NXJJ58cXn755VBfXx9ee+218K1vfetT20yfPj0sW7Ys1NfXhyeeeCIMGjQo9r5zsXr06BFuuOGGUFlZGdauXRveeuutcNVVV4WuXbu6v3exxowZ0+K/0bNmzWrzft1zzz3DPffcE2pra8OqVavCnXfeGbp3777LvRUkFyRJkrKWY1gkSVLWM7BIkqSsZ2CRJElZz8AiSZKynoFFkiRlPQOLJEnKegYWSZKU9QwskiQp6xlYJElS1jOwSJKkrGdgkSRJWc/AIkmSst7/B63jbs8u0oqOAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "plt.plot(jnp.arange(100), losses, c=\"r\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7ff92034f5b0>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAGdCAYAAAA8F1jjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA+SElEQVR4nO3dd3wUdf7H8VcSEkrYANKJgEhV0aBIVQRFEe9OUU9BubsfiB3PXgD1zogFRaWIiAiKoKennIpYAEUBG1ioNqRFSoBAaAkECIH5/TEkmZlNIJvs7szuvp+Px/dxmc+U/WT1cXk75TtxgIGIiIiIB8W73YCIiIhIaRRURERExLMUVERERMSzFFRERETEsxRURERExLMUVERERMSzFFRERETEsxRURERExLMqud1AMDRq1Ijc3Fy32xAREZEA+Hw+Nm/efMxtIj6oNGrUiMzMTLfbEBERkXJITU09ZliJ+KBSeCYlNTVVZ1VEREQihM/nIzMz87h/uyM+qBTKzc1VUBEREYkyuplWREREPEtBRURERDxLQUVEREQ8K2ruUTmWuLg4atasic/nIy4uzu12JECGYZCbm8vu3bsxDMPtdkREJIyiPqjUrVuXG2+8kTZt2rjdilTQypUrmTRpEtu3b3e7FRERCZM4IKL/E9Xn85GTk0NKSorfUz+VKlXixRdfZO/evbzzzjts27aNw4cPu9SplFdCQgL16tWjb9++VK9encGDB1NQUOB2WyIiUgHH+vttFdVnVBo2bEiVKlV49tlnWbVqldvtSAWsW7eOnTt38vDDD9OgQQM2bdrkdksiIhIGUX0zbXy8+esdPHjQ5U4kGAr/OSYkJLjciYiIhEtUBxURERGJbAoqIiIi4lkKKhKQKVOm8P7777vdhoiIxAgFlSj0yCOPsHTpUrfbEBERqTAFFRERESnRn4EHgBou9hBzQaWaSyNQF198MV999RW7du0iOzubDz/8kJNPPrlofWpqKm+++SY7duxg7969/PDDD3Ts2JEBAwaQnp5Ou3btMAwDwzAYMGAATZs2xTAM0tLSio5Ro0YNDMOge/fugPmU1OTJk1m3bh15eXmsXLmSO+64oxzdi4hIpFsIfAQ8DVzmYh9RPY+KUzVgn0ufnQzkBbJ9cjKjRo1ixYoVVK9eneHDh/P+++/Trl07qlWrxoIFC8jMzOSyyy5j69atnHXWWcTHx/P222/Ttm1bevfuzYUXXgjAnj17qF+//nE/Mz4+nk2bNnH11VezY8cOunbtyssvv8yWLVuYPn16+X5xERGJKA2BzY6am3cmxlRQiSTvvfeebXnQoEFkZ2dz6qmn0rVrV+rWrUuHDh3YtWsXAGvXri3adu/evRQUFJCVlRXQZxYUFJCenl60/Mcff9ClSxf69u2roCIiEgMGAK9ZlncA9YAjrnRjiqmgkod5ZsOtzw5EixYtGD58OJ06daJOnTpFk9c1adKEdu3asXTp0qKQEkyDBw9m0KBBNGnShKpVq5KUlMSyZcuC/jkiIuIty4EzLMv3A8+61ItVTAUVCDwwuOXDDz9k/fr13HjjjWzevJn4+Hh++eUXkpKS2L9/f8DHO3LEzMPWt0cnJibatunXrx/PPvss9957LwsXLiQ3N5f777+fTp06VeyXERERz2oMbHDUWgGrXeilJDF3M20kOOGEE2jTpg2PP/44X3zxBStXrqRWrVpF61esWEG7du1sNav8/Hy/aeYL3zjcsGHDolq7du1s25xzzjl8++23TJgwgWXLlrF27VqaN28epN9KRES85ibsIWUTZjDwSkgBBRVPKnzS56abbqJ58+acf/75jBo1qmj9W2+9xdatW5kxYwZdu3alWbNmXHnllXTu3Bkw7y1p1qwZaWlp1K5dm6SkJA4cOMDChQsZOnQobdq04bzzzuPxxx+3fe7q1as5++yz6dWrFy1btmT48OF06NAhrL+7iIiEXhxmGJloqd2BeXbFcKWj0imoeJBhGFxzzTW0b9+en3/+mdGjR3P//fcXrT906BC9evVi27ZtfPLJJ/z0008MHTqUw4cPA/Duu+8ye/Zs5s2bR3Z2Ntdeey1g3pBbqVIlFi9ezJgxY3j44Ydtnztx4kTee+893n77bb777jtq167Niy++GL5fXEREQq4Z5s2xLSy1k4Fx7rRTJkaoRrdu3YyZM2camZmZhmEYRp8+fWzrp0yZYjjNmjUroM/w+XyGYRiGz+fzW9e0aVNj2rRpRtOmTUP2O2qEb+ifp4aGhkbFxu1gGJbxOxhxLvVyrL/f1hHSMyrJycksX76c2267rdRtZs2aRYMGDYpG4X/9i4iISHDEYd5/8ryldhPQGjMNeFlIn/qZPXs2s2fPPuY2Bw8eDHi+DxERESmblsAqR60xZnCJBK7fo9KjRw+ysrJYuXIlL774IieccMIxt09KSsLn89mGiIiI+Lsfe0hZRvHZlUjhalCZPXs2//d//0fPnj0ZMmQI3bt3Z9asWUWTm5Vk2LBh5OTkFI3MzMwwdiwiIuJ98cBOYKSlNgA40512KsTVCd/efvvtop9//vlnVqxYwbp16+jRowdffPFFifuMGDHC9qiuz+dTWBERETnqVOAXR60RsMWFXoLB9Us/VhkZGWzfvp0WLVqUuk1+fj65ubm2ISIiIvAv7CHlW8xLPZEaUsBjU+inpqZSu3ZttmyJ5K9UREQkvCoB+4AkS+0a4O2SN48oIQ0qycnJtrMjhbOl7ty5k507d/LII4/w7rvvsnXrVpo3b87IkSNZs2YNc+bMCWVbIiIiUeMMzBcKWtUDtrvQSyiE9NLP2WefzbJly4revjt69GiWLVvG8OHDOXz4MGeccQYzZ85k1apVvPLKKyxevJhu3bqRn58fyrbEISMjgzvvvLNo2TAM+vTpE/Y+HnnkEZYuXRr2zxURiVRPYA8pn2Ne6glaSOkM9AKSg3XAwIX0jMqCBQtsb+t16t27dyg/XsqpQYMG7Nq1q0zbPvLII1x++eWceWYk3ksuIhKZEgHnf9JfAcwI1gdUAYZalrcCK4J18MB46h4VKb/ExEQOHToUlGNpAj4REe9qD/zoqNXGfBw5KM4EnCfVnY8RhZGnnvqRYvPmzWPcuHGMGzeO3bt3s337doYPH160PiMjg4cffpipU6eyZ88eXn75ZQDOOeccvvzyS/Ly8tiwYQNjx46lWrVqRfvVrVuXmTNnkpeXx7p16+jfv7/fZzsv/aSmpvLmm2+yY8cO9u7dyw8//EDHjh0ZMGAA6enptGvXDsMwMAyDAQMGAFCjRg0mTZrEtm3b2LNnD59//jlnnHGG7XOGDBnC1q1bycnJYfLkyVSpUiWo36GISLQZhT2kfIR5qScoIaUS5mND1pCyGEgHDgfjA8on9oJKokujHAYMGEBBQQEdO3bkzjvv5J577uGGG24oWn/fffexfPlyzjzzTB577DFOPvlkZs+ezbvvvssZZ5xBv379OPfcc3nhhReK9nnttddo3Lgx559/PldddRWDBw+mXr16pfaQnJzMggULSE1N5bLLLiMtLY2RI0cSHx/P22+/zbPPPsvPP/9c9K6mwrlxpk+fTr169bjkkkto3749S5Ys4fPPP6dWrVoAXH311aSnp/Pggw9y9tlns2XLFgYPHly+L0pEJMpVxnwnz92W2p+BS4P1Aa2Bh4EES+0F4MNgfUD5xdaln0TgIZc++wkgwCszGzdu5O67zX8tV61axemnn87dd9/N5MmTAfjiiy9sk99NmjSJ//znP4wdOxaANWvWcMcdd7BgwQJuvfVWmjRpwp/+9Cc6dOjAjz+amfz6669n5cqVpfbQv39/6tatS4cOHYruW1m7dm3R+r1791JQUGC7XHTOOefQsWNH6tWrV3Rj9P3338/ll1/OVVddxaRJk7jrrrt45ZVXePXVVwH417/+xYUXXqizKiIiDl0w50OxqgnsCcbB44DbAevba1YBbwbj4MERe2dUIsiiRYtsywsXLqRly5ZFrxgoDBuF0tLSGDhwoG0yvDlz5pCQkECzZs045ZRTOHToEIsXLy7a5/fffz/mjbPt2rVj6dKlZb65trCP6tWrs2PHDlsvzZo1o3nz5gCccsopfPfdd36/n4iIFHsRe0iZjpktghJSmgCPYA8pk/BUSIFYO6NyCPPMhlufHWT79u2zLVevXp2JEyfy/PPP+227YcMGWrVqFfBn7N+/P+B9qlevzpYtW+jRo4ffut27dwd8PBGRWFMNcwI3q4uAucH6gIHASZblLOAlzOtLHhNbQQVCEhhCpVOnTrblzp07s3r1ao4cOVLi9kuWLOHUU0+1XZqxWrlyJYmJibRv377obEyrVq2K7hspyYoVK7jhhhuoVatWiWdV8vPzSUhIsNWWLFlCgwYNKCgoYP369SUe97fffqNTp068/vrrtt9PRCTWnQcscNR8wN5gHLwe4Lwd8A1gTTAOHhq69ONhTZo04bnnnqNVq1Zcc8013H777UX3n5Tk6aefpmvXrowbN460tDRatGjBZZddxrhx4wDzPpdZs2YxceJEOnbsyFlnncXkyZPJy8sr9ZhvvfUWW7duZcaMGXTt2pVmzZpx5ZVXFoWKP/74o2jG4dq1a5OUlMTcuXNZuHAhM2bM4KKLLqJp06Z06dKFxx9/nPbt2wMwduxYBg0axMCBA2nZsiXp6emcdtppQfz2REQizxTsIWUq5qWeoISUK7GHlP3AY3g6pICCiqdNmzaNqlWr8v333zN+/HjGjh1b9BhySX766Se6d+9Oq1at+Oqrr1i6dCnDhw9n8+bNRdtcd911bN68mQULFvDee+/x8ssvs23btlKPeejQIXr16sW2bdv45JNP+Omnnxg6dCiHD5vPqr377rvMnj2befPmkZ2dzbXXXgvAn/70J7788kumTJnCqlWr+O9//0vTpk2Lbrp95513eOyxxxg5ciSLFy+madOmTJgwIRhfm4hIxKmOedVloKXWw7FcbjUxHzG2zhDxHvA0rj52HAgjkofP5zMMwzB8Pp/fuqZNmxrTpk0zmjZt6nqfgY558+YZo0ePdr0PL41I/uepoaGhUdq4EAzDMaoF6/gXYpDuGEnu/85w7L/f1hF796iIiIh4xH+Bfpbll4Bbg3HgasADjtocIAIfrlRQERERCbMawG5HrStByhGdAeer9EYCpd+O6GkKKh51/vnnu92CiIiEwJ+Ajx21qsCBih44CXjQUfsG+KyiB3aXbqYVEREJk5nYQ8oYzKd6KhxSTsc/pIwh4kMK6IyKiIhIyJ0A7HDUOuD/FuSAJQD3YZ6SKbQC86meKBHVQcUwDAAqVYrqXzNmFP5zLPznKiISCS4H3nfUKgP5FT1wC+DvjtoEzFlmo0hUX/rZscPMr23atHG5EwmGwn+O2dnZLnciIlI2c7GHlBGYl3oqFFLiMB8NsoaUPzDnSomykAJRfkZl3759zJ8/n759+wLmFPIFBQUudyWBqlSpEm3atKFv377Mnz//mDPpioh4QV3AOZVmO2B5RQ+cCtzoqE0BSn5bSVSI6qACMGXKFAD69et3nC3F6+bPn1/0z1NExKv6Yc6PUugg5syzFf7P5P6A9d2yu4BxQMmvf4sacZgzv0Usn89HTk4OKSkp5ObmlrpdtWrVqFOnDnFxcWHsToLBMAyys7N1JkVEPO8bzPlQCv0b83U6FVIH+Kej9l9gZUUP7K6y/v2O+jMqhfLy8tiwYYPbbYiISBRqAGxx1E4Dfq3ogf8CnG1ZPox5o0sM3cUQM0FFREQkFP4BTLMs78K8R6VC7/vzAfc6ajOBJRU5aGRSUBERESmnZUCaZXkI5mz1FdIdcE5O/hRBmBUuMimoiIiIBOhEYKOj1gpYXZGDVgGGOmpfAF9W5KCRT0FFREQkADcAkyzLm4HGVPDhm7Mx70exeg4o/R7TmKGgIiIiUka/Y39C+E7g+YocMBF4yFH7HvikIgeNLgoqIiIix3ESkOGonVxCLSCnYE66YvU8sLMiB40+CioiIiLH8E/MedUKrQZaU4FJyOIxT8XUsNR+A94u7wGjm4KKiIhICeIwZ6ZvbKndAkysyEFPAgY6ai9j3ugiJVJQERERcWiB/xM8TfB/0icg12NPPZsx78qN6PnhQ09BRURExOJe4FnL8nLMFwqWWwPMUzFW04B1FTlo7FBQERERwbx1ZDtwgqU2EJhakYNeBbS1LO8FRhH1LxIMJgUVERGJeafg/16eRvi/v6fMamHeMGv1P+Dn8h4wdsW73YCIiIibHsIeUhZh3khb7pByMf4h5QkUUspJZ1RERCQmVcKc+LWKpXYt8N/yHjAZuN9R+wRzAjcpNwUVERGJOacDKxy1+sC28h7wHOAiR20kkFfeA0ohBRUREYkpj2OftX4ecEF5D1YZGOaofQV8Xt4DipOCioiIxIREIN9R+yvwXnkPmAZc4aiNBvaU94BSEgUVERGJeu2BHx212pTztToJwBAgyVJbBswoz8HkePTUj4iIRLVnsYeUjzCf6ilXSGkF/At7SBmPQkoI6YyKiIhEpcrAAUftL8DH5TlYHDAYqGuprQVeL1drEgAFFRERiTqdMOdDsapJOW8faYz5nh6rV6jgi3+krHTpR0REosp47CFlOuYJkXKFlH9gDynZwKMopISRzqiIiEhUqIr/tCW9gM/Kc7C6wG2O2pvAqvIcTCpCQUVERCJeN+BLRy0Fc+bZgPUBzrQs5wNPA4fL1ZpUkIKKiIhEtFeB6yzL04AB5TlQCnCPozYD89FjcY2CioiIRKRkYK+j1gNYUJ6DXQCc56iNAA6W52ASTAoqIiIScXoCcx21ZMrxap2qmJO3WX0GfFO+viT49NSPiIhElLewh5SJmE/1BBxSOuIfUp5BIcVjdEZFREQiQgr+jxifA3wb6IESsb+VEMznmWeXry8JLQUVERHxvEuATxy1qvjPPHtcpwFXO2pjgV3l60tCL6SXfrp168bMmTPJzMzEMAz69Onjt82jjz7K5s2bycvL47PPPqNFixahbElERCLMDOwhZQzmpZ6AQko8cB/2kPILkI5CiseFNKgkJyezfPlybrvNOWuO6YEHHuCOO+7glltuoVOnTuzbt485c+ZQuXLlULYlIiIRoBZgYE5rUqgDcHegBzoZ+DdQ3VJ7CXPKWokIRjiGYRhGnz59bLXNmzcb9957b9FySkqKsX//fqNfv35lPq7P5zMMwzB8Pl9Yfg8NDQ0NjdCPPmAYjpEU6HHiMLgRg3TLuN79303DHGX9++3aUz/NmjWjYcOGzJ1bfO92Tk4O3333HV26dCl1v6SkJHw+n22IiEj0+BTzck+hpzAv9eQHcpCGwCNAqqU2FfNlghJRXLuZtkGDBgBkZWXZ6llZWUXrSjJs2DDS09ND2ZqIiLigDrDdUWsHLA/0QP2AUyzLezBvmD1S3s7ETRE3j8qIESNISUkpGqmpqcffSUREPO1q7CHlEOZTxAGFlBMwb461hpR3gNEopEQw14LK1q1bAahfv76tXr9+/aJ1JcnPzyc3N9c2REQkcn2NmScKpQNJQEEgB7kEuMNRewL4tSKdiRe4FlQyMjLYsmULPXv2LKr5fD46derEwoUL3WpLRETCpAHm3ZLnWGqnAY8GcpDqmMmmk6X20dHaoQq1Jx4R0ntUkpOTbfOiNGvWjLS0NHbu3MnGjRsZM2YMDz/8MKtXryYjI4PHHnuMzZs3M2PGjFC2JSIiLvs78LpleQ9QGzgcyEG6Yb70x+ppYH+FWhMPCtmjR927dzdKMmXKlKJtHn30UWPLli3G/v37jc8++8xo2bJlSB5v0tDQ0NDwxliM/bHjIYEeozL2R47TMejh/u+lEdgo69/vuKM/RCyfz0dOTg4pKSm6X0VExMNSgU2OWmtgVSAHORP7DHAAo4Cc8vcl7ijr32+960dERELuBmCSZXkLcCIBPIxTCRgGJFhqi4EPg9GdeJmCioiIhNRKzDMnhe7CnNakzFoD1zpqLwDZFWpLIoSCioiIhERT4A9HrTmwrqwHiANux5wfpdBq4D8V7UwiiYKKiIgE3WBgvGV5LdCSAG6KbAIMctQm43+Ti0Q9BRUREQmaOGA90NhSuxXzZcVlNhA4ybKcdfQAEf3oh5SXgoqIiARFc2CNo9YU2FDWA9TDPBVj9UYJB5WYoqAiIiIVdg/wnGX5Z+D0QA5wBZBmWd4PPEuAM8BJNFJQERGRcovHvDJTx1IbBEwp6wFqYj4GZPUesKKCjUnUUFAREZFyaQP85qg1wpwjpUwuBM511J4E8ivWl0QXBRUREQnYg5gvJy70HdC5rDtXAx5w1OYAeh+tlEBBRUREyiwBc7b6apba34A3y3qAzkBvR+0ZYF+FW5MopaAiIiJlcjr+t47UB7aVZeckzNMwVt8An1W8L4lu8W43ICIi3jcce0iZhzlnSplCyun4h5QxKKRImeiMioiIlCoR/3tbrwLeLcvOCcB9QFVL7aey7ixiUlAREZESnQkscdTqADvKsnNz4B+O2gTMZ5lFAqCgIiIifkYC91uWZwF/KsuOccDNQANL7Q/gtSA1JjFHQUVERIpUBg44apcBH5Zl51TgRkdtCubLf0TKSUFFREQA6AQsctRqAbvLsnN/oJVleRcwDjgSjM4klimoiIgI44B/Wpbfxbxp9rhqA7c7av8FVganLxEFFRGRGFYVyHPULgY+LcvOfwY6WJYPAyOAgqC0JgIoqIiIxKxzga8ctRQg93g7+oB7HbWZ+D8iJBIEmvBNRCQGTcYeUl7HfGDnuCGlO/4h5SkUUiRkdEZFRCSGJAN7HbXzgfnH27EKMNRR+wL4MihtiZRKQUVEJEZcAHzuqFWnDO8DbA9c6qg9RxlOv4hUnIKKiEgM+A/mE8SFXsacl+2YKgEPO2rfA58Ery+R41FQERGJYinAHkftXMwXFx/TKUA/R20cZZw/XyR4FFRERKJUb8yp762q4j/zrE08cAdQ01JbiTk3iogL9NSPiEgUeh97SHke86meY4aUk4B/Yw8pL6OQIq7SGRURkShSC9jpqHUEfjjejoOAJpblzcAkwAhaayLloqAiIhIlLgM+cNQqA/nH2qk+cKujNg1YF7y+RCpCl35ERKLAbOwh5WnMSz3HDClXYQ8p+4DhKKSIp+iMiohIBKsDbHfUzgSWHWunWsCdjtr/gJ+D1pZI0CioiIhEqKuA6Zblw5hP9Rw61k69gK6O2hPH20nEPQoqIiIR6Eugm2X5USD9WDskA/c7arOA74LalkjQKaiIiESQ+sBWR60t8MuxduqKeSbFaiSQF7y+REJFQUVEJEL8DXjDspyLebvJ4dJ2SAIedNS+wv+FPyIepqAiIhIBfsR8N2ChYcBTx9ohDbjCURuN/3z6Ih6noCIi4mGNgExHrTWwqrQdEoAHMCdQKbQMmBHkxkTCREFFRMSjBgGvWJazMIPLkdJ2aIl5fchqPP7PL4tEEAUVEREP+hXzBcaF7gbGlLZxHDAYqGuprcOcYVYkwimoiIh4SBNgvaPWnGNMFnsicIOj9gqwMbh9ibhFQUVExCMGY16pKZSBGVJKfS/g34EWluXsowfQiwQliiioiIi4LA4zlDS11AYDE0rboS5wm6P2Jse4w1YkcimoiIi4qDmwxlFrCmwobYfLgLMsy/mYk7cVBL01EU9QUBERccndwCjL8i+Ys8yWKAW4x1GbwXHePigS+RRURETCLB7YAtSz1AYBU0rb4Xygu6M2AjgY9NZEPEdBRUQkjFoDKx21VGBzSRtXBYY4ap8B3wS/LxGvine7ARGRWDEUe0j5AfNG2hJDSgf8Q8qzKKRIzNEZFRGREEvAfMVOsqX2N8wHdfwkAg85aouA2SFpTcTzFFREREKoLfCTo9YAczp8P6cBVztqY4Fdwe9LJFIoqIiIhMijwL8tywuAHiVtGI/5CJDPUvsFmB6qzkQih+v3qDzyyCMYhmEbv/32m9ttiYiUWyLm5LDWkHI1pYSUZkc3tIaUl1BIETnKE2dUfv75Zy688MKi5YICzVwkIpHpTGCJo1YH2OHcMA7zHT2pltomYHLIWhOJSJ4IKgUFBWRllXjFVkQkYjwNPGBZng1cUtKGDYGbHbWpmPPoi4iNJ4JKy5YtyczM5MCBAyxcuJBhw4axcWPJr/5MSkqicuXKRcs+n6/E7UREwiUJ/7nX+gAzS9q4L3CqZTkHGAMcCUVnIpHP9XtUvvvuOwYOHEjv3r259dZbadasGV999RXVq1cvcfthw4aRk5NTNDIzM8PcsYhIsY74h5RalBBSTgDSsYeUdzDn0FdIESlVHB57IXiNGjVYv34999xzD6+++qrf+pLOqGRmZpKSkkJubm44WxWRGDcWuMOy/D5wZUkbXgJ0ctSeAA6Fpi+RSODz+cjJyTnu329PXPqx2rNnD6tWraJFixYlrs/Pzyc/Pz/MXYmIFKsC7HfUegNznBtWB+5z1D7GnJJWRMrE9Us/TsnJyTRv3pwtW7a43YqIiJ9z8A8pNSghpJyLf0h5GoUUkQC5HlSeeeYZzjvvPJo2bUqXLl14//33OXz4MG+99ZbbrYmI2LwMfG1Z/g/m9fMc60aVMe9FudBSm3+05kw4InJcrl/6OfHEE3nrrbeoXbs227dv5+uvv6Zz585kZ2e73ZqICGC+o2evo3YBMM+54ZmYj/tYjcKRZEQkEK4HlWuvvdbtFkRESnU+8IWjVh3YZy1Uwnw1svX/URcDH4a0NZGY4PqlHxERr3ode0iZjHmpxxZSWgMPYw8pL6CQIhIkrp9RERHxGh/+V2u6Yb8/hTjgn0BtS2015o0rIhI0CioiIhYXY059b1UNx32wTYBBjo0mY76rR0SCSkFFROSod7FP2DYO+4RuAAzAfONxoSzMtx17aupMkeihoCIiMa8msMtR6wR8by3UAwY7NnoDWBOytkQEBRURiXGX4v9enio43t9zBZBmWT4APAMcDmlrIoKCiojEsE8wX8NTaCQwxLpBDeBux07vAStC25eIFFNQEZGYUxtwTil5FrDUWuiJ+aiP1ZOAXjUmElYKKiISU/4K/M9RS8LyIuNqwAOODeYAC0Pbl4iUTBO+iUjMmI89pAzHnA6lKKR0xj+kPINCioiLdEZFRKJePcyniK1OB34uXEgCHnRs8A3wWWj7EpHjU1ARkajWH/tksXsxH0cuemCnLXCVY6cxwO7Q9iUiZaOgIiJR63ugg2X5Icz7YQFIAO7FvCel0E+Ys76JiGcoqIhI1GkEZDpqbYDfCxeaA/9wbDAB/+tDIuI6BRURiSrXAa9alrcBDYEjYN45e9PRQqH1wJRwdScigVJQEZGo8TNwmmX5HmB04UIqcKNjhymYQUVEPEtBRUQiXhP880YLYG3hwrVAa8vKXZhvHDwS8tZEpIIUVEQkot2CeXtJofWYLzc2wJyC9nbHDv8FVoalNREJAgUVEYlIcZhnTJpZarcBLxYu/Bn7Iz9HMB/5KQhHdyISLAoqIhJxTsZyWeeokzh6+ceH+dix1YfA4pC3JSIhoKAiIhHlLiw3yAK/AacWLpwHXODY4SngQMjbEpEQUVARkYgQD2wG6ltqNwCvAFQBhjp2+AL4MiytiUgIKaiIiOe1wjJZ21EncnRSt/bApY6VzwG5oe9LREJPQUVEPG0I5tWbQj9y9B7ZSphz4sdZVv4AfBy21kQkDBRURMSTEjCnO/FZan/n6AsGTwH6OXYYB+wIS2siEkYKKiLiOadhzjJr1QDIigfuwHz9caGVmHOjiEhUine7ARERq3TsIeUrzKs7WU2Bf2MPKS+jkCIS5XRGRUQ8oRKwH/v/KfUFpgMMwpwnv9BmYBJHp58VkWimoCIirmsHLHXU6gLZ9YFbHStex3+2NxGJWrr0IyKuegp7SPkU81JP9l+xh5R9wGMopIjEGJ1RERFXJAEHHbU+wMyamNPPWv0P/7trRSQmKKiISNh1AL531E4AdvUCujpWPAEcCkdXIuJFCioiElZjgDstyzOAK5KB+x0bzgK+C09PIuJdCioiEhZVMJ/qsboEmN0V6OVYMRLIC0dXIuJ1CioiEnJdgW8ctZQkyH3QUfwK+Dw8PYlIZNBTPyISUhOxh5Q3gbgzSggpo1FIERE/OqMiIiFRDfOJYqvzE2D+A0BlS3EZ5o0qIiIlUFARkaDrAcxz1Kq0hIN/cxRfBLaFpSURiVAKKiISVNOAf1iWJ8fBjbcC9SzFdUc3FBE5DgUVEQkKH5DjqKWdCCtucBRfATaGpycRiXwKKiJSYb2AOZZlA6j0dzjSwlLMBsajFwmKSED01I+IVMh07CHlX3UgPt0RUt4EXkAhRUQCpjMqIlIuNYFdjlq9y2D7WZbCIeBpoCBcXYlItFFQEZGA/QX40LK8MQWa3OPYaAbmo8ciIhWgoCIiAfkI+LNl+aLzYW53x0Yj8H81sohIOSioiEiZ1Ma8H7bQzqpQe4hjo7nA1+HrSUSin26mFZHjuhJ7SHmxQwkh5VkUUkQk6HRGRUSOaR7mTLMAeYmQ/JBjg0XA7LC2JCIxREFFREpUD8iyLE8/Ffr2dWz0PLAzfD2JSOxRUBERP9cAbx39+VA8NLkbtvosG/wKvBP+vkQk9iioiIjNIqDT0Z8/bwYXDnBsMBHYEt6eRCR2KaiICAANgc1HfzaATjfCD6mWDTYBk8PelojEOE889TN48GAyMjLYv38/ixYtokOHDm63JBJTBlIcUpY0NKfAt4WUqSikiIgrXA8qffv2ZdSoUTz66KOcddZZLF++nDlz5lC3bl23WxOJCSuAKUd/vqovtL/ZsjIHGA5khL0tEREA4nD5NWGLFi3ihx9+4Pbbbzcbiotj48aNjBs3jqeffvq4+/t8PnJyckhJSSE3NzfU7YpEjcbAhqM/rzkBWt7h2OAdzJtmRURCoKx/v109o5KYmEj79u2ZO3duUc0wDObOnUuXLl1K3CcpKQmfz2cbIhKYGygOKXf2LiGkPIFCioh4gqs309apU4dKlSqRlZVlq2dlZdGmTZsS9xk2bBjp6elh6E4k+sQBK4FWwNbq0PA+xwYfAz+EvS0RkVK5fo9KoEaMGEFKSkrRSE1NPf5OIkIz4AhmSBnTuYSQ8jQKKSLiOa6eUcnOzqagoID69evb6vXr12fr1q0l7pOfn09+fn442hOJGncAY4E9laHmMMfK+UeHiIgHuXpG5dChQyxevJiePXsW1eLi4ujZsycLFy50sTOR6BCP+djxWODVMx0hZS8wCoUUEfE01yd8GzVqFFOnTuXHH3/k+++/56677iI5OZkpU6Ycf2cRKVVLYBVwoBJUHQIHEi0rlwAz3elLRCQQrgeVd955h7p16zJ8+HAaNGjAsmXL6N27N9u2bXO7NZGI9QDmLScftoLL+jtWjge2h78nEZHycH0elYrSPCoixRKAbMAXB6f8E1bXtqxcA7zhTl8iIk5l/fvt+hkVEQmOU4FfgG8aw7nXO1ZOxnxXj4hIhFFQEYkC/wbSgZ7/B1+cbFmxDZhAhJ83FZFYpqAiEsEqAXnA7/UgfrBj5X+A1eHvSUQkmBRURCJUGrAMGHA5TGtnWXEAeAY4HP6eRESCTUFFJAI9CfSvAXF3O1a8Dyx3oSERkRCJuCn0RWJZEubtJjk94SRnSHkShRQRiTo6oyISIc4GZlWDuAccKz4FvnWhIRGRMFBQEYkAo4H4TlD3EseKZ4B9LjQkIhImCioiHlYFyE6E6g85VnyLeSZFRCTKKaiIeFQX4Pa2UP0qx4qxwC4XGhIRcYGCiogHjUuAR++B/smW4s/A/9zqSETEHQoqIh5SDZhxMvT6P8eKCUCWCw2JiLhMQUXEI86Lg5yboFfD4lr8Bjjyqns9iYi4TUFFxAP+3QiG3+QovgZH/nChGRERD1FQEXFRMtDzGhjeprhWeTccfB444lJTIiIeoqAi4pIutWHh7TDTUkt6Gw7+5lpLIiKeo6Ai4oJef4ZPOxQvxx0B40nIL3CvJxERL1JQEQkjnw9y77XP1dbsQ8hY7FpLIiKepqAiEiatzoNVF9hrSU9BxgF3+hERiQQKKiKhVgUYCqsspYvnwZwFkO9WTyIiEUJBRSSEqp0FeZfZa22fgzm57vQjIhJpFFREQqESxD0IefHFpZt/hFc/MmfCFxGRslFQEQm2NsA1YFhKd46DsTvcakhEJHIpqIgESzzE3wFHahaX/vI7rH/LfOGxiIgETkFFJBiaAtfZJ5P9chJckAmaGkVEpPwUVEQq6jrMoHJU2lb4y0Q4zyh1DxERKSMFFZHyqg/cai/Nfh3uXgtPuNKQiEj0UVARKY+/AqcXL9bOg5+egxMP612CIiLBpKAiEoiawF320n/ehaU/QSMX2hERiXYKKiJldRFwjr2U+ySclQ+rXWlIRCT6KaiIHE8ycL+9NHo2/HURpGCfL0VERIJLQUXkWLoAF9tL20bCY3nQxJWGRERii4KKSEmSgAftpSFfw1NzoRnwhwstiYjEIgUVEaczgCvtpYwxkL8b4tGlHhGRcFJQESmUgHkvSpXi0t+Xw+vvw03AJJfaEhGJZQoqIgAtgL/bSytehNO3QWNgkxs9iYiIgorEuDjM2WXrFZd6ZMAXU2H50dUiIuIeBRWJXScCN9hLX70K526AAcA0N3oSEREbBRWJTX8DWhYvttgBK1+ABMOcYXaLW32JiIiNgorEljrAP+2lGW9Bn9/hW/wmnhUREZcpqEjsuBRoX7xYuQB2PwVVCuAa4G23+hIRkVIpqEj0SwHusZcmfwDXLzV/rgdsD3dPIiJSJgoqEt16HB0Wu0dAjYPwOXBh+DsSEZEAKKhIdKoKDLGXnvgcHvzK/PkKYEaYWxIRkcApqEj0ORv4i720+VlouNf8uTawM9w9iYhIuSioSPRIBB6yl/75HYybZf78Eeb9tCIiEjkUVCQ6nAr0tZdWPQ8tj546+TPwSbh7EhGRClNQkcgWD9yF+WTPUVf8Bu9ZnjWuCewJa1MiIhIsCioSuU4CBtpLP06E9kenlZ2O30kWERGJMAoqEpluwHxXz1EdMmHRZIg3zOWLgLlu9CUiIkGloCKRpQFwi700dyr0zChe9gF7w9mTiIiETLybH56RkYFhGLYxZMiQ4+8oselqbCGlRi7kDy8OKa8BcSikiIhEE9fPqPzrX/9i0qRJRcu5ubkudiOeVAu40156ezr0/aV4uTvwZTh7EhGRsHA9qOTm5pKVleV2G+JVvYHO9tLeJyD5UPFyMpAXzp5ERCRsXL30AzB06FCys7NZsmQJ9913HwkJCcfcPikpCZ/PZxsShaoD6dhCynWfgJFeHFJewrzUo5AiIhK9XD2j8vzzz7NkyRJ27txJ165dGTFiBA0bNuTee+8tdZ9hw4aRnp4eviYl/M7BfGzHIvtpqL2/eLkrsDCcPYmIiGuMYI4RI0YYx9O6desS973uuuuM/Px8IykpqdTjJyUlGT6fr2g0atTIMAzD8Pl8Qf09NFwYlTFIt48W52MY2EcVt/vU0NDQ0Kjw8Pl8Zfr7HXf0h6CpU6cOtWvXPuY269at49ChQ371U089lV9++YXWrVuzatWqMn2ez+cjJyeHlJQU3YgbydoBl9tLr4yCQTnFy2OAu8PWkIiIhFJZ/34H/dJPdnY22dnZ5dq3Xbt2HD58mG3btgW5K/GsBGAo5gsFj0paCgc/sG/WAfgxjG2JiIg3uHaPSufOnenUqRPz5s0jNzeXLl26MHr0aN544w12797tVlsSTq2A/vbSBePh8+32WmUgP1w9iYiIp7gWVA4ePMg111xDeno6lStXJiMjg9GjRzNq1Ci3WpJwiQNuA+pYamvg0zfs99COAB4Ma2MiIuI1rgWVpUuX0qVLF7c+XtzSGLjeXqo5GXZtstfSgBXh6klERDzL9QnfJIb8H3CyZXkbXD0B3rHczn0QcwqVgvB2JiIiHqWgIqFXF/NSj9V/4NvVYD2n9m/gsfB1JSIiEUBBRULrcsxHjwsdhPojYeth+2anAb+GrSkREYkUCioSGjXwn/TkffjHcphmKe3CPOHiyC0iIiKAgoqEwgXAeY7aCFh20LxJttAQYGT4uhIRkQikoCLBUw14wFH7FE78FjY6yq2A1eHpSkREIpiCigRHJ+ASR+0ZuGEfTLKUNmM+oXwkbI2JiEgkU1CRikkEHnLUFgJz4HfMMyeF7gSeD1dfIiISFRRUpPzaAlc5amPhpF2Q4SifjH9NRETkeBRUJHDxwL1AsqX2M/A/+CcwzlJeDbQmyK/oFhGRmKGgIoE5GXOGWauXIG4rbABOtJRvBl4OW2MiIhKNFFSkbOKAm4CGltoG4FVogf8TPE3wf9JHREQkUAoqcnyNMEOK1WvAH3Af8IylvBz7RLQiIiIVoaAix3YN0MayvBt4HuKPQDZQy7JqIDA1fJ2JiEgMUFCRktUGbnfU3gZ+g1Pwfy9PI2BLOPoSEZGYEu92A+JBf8IeUo4AjwO/mVOmWEPKIszbVxRSREQkFHRGRYr5MB87tvoQWGz+i5ILVLGsuhb4b5haExGR2KSgIqZuQE9H7SngAJwOrHCsqgdsD0dfIiIS0xRUYl0VYKijNg9YYP74OPYZ8r/AP8+IiIiEioJKLDsLuMxRew7INV/hk+9Y9VfgvXD0JSIicpSCSiyqBAwDEiy1H4GPzB/bH120qg3sDENrIiIiVnrqJ9a0AR7GHlLGURRSnsUeUj7CfKpHIUVERNygMyqxIg64A/sMbb8Db5k/VgYOOHb5C/BxGFoTEREpjYJKLGgKXOeoTQIyzR87Yc6HYlUT2BPitkRERI5Hl36i3UDsIWUr8ChFIWU89pAyHfPki0KKiIh4gc6oRKv6wK2O2uvAWvPHqkCeY3Uv4LNQ9yUiIhIABZVodCVwhmU5D/Ox48PmYjfgS8cuKZgzz4qIiHiJgko0qQnc5ai9C/xUvPgq9itB04ABoe1KRESk3BRUosVFwDmO2pMUzdqWDOx1rO5B0QS0IiIinqSgEumqAQ84arOx3SHbE5jr2CQZ/3tUREREvEZP/USyzviHlJHYQspb2EPKRMynehRSREQkEuiMSiRKAh501L7GlkhS8H/E+Bzg21D2JSIiEmQKKpHmDMyneqzGALuLFy8BPnFsUhX/mWdFRES8TkElUiQA92EmjkIr8Hud8Qygj2V5DHB3SBsTEREJHQWVSNAC+Luj9iKwrXixFv4vDuyA/1uQRUREIomCipfFAbdgzjJbKAOYat+sD+aZFKvKFD2ZLCIiErEUVLwqFbjRUXsV2GAvfYo5hUqhEfjfZysiIhKpFFS8qD/QyrK8A3gBMIpLdYDtjt3aActD25mIiEhYKah4SR3gn47aW8Dv9tLVwDuW5UOY874VhLA1ERERNyioeMWlQHvLcgHwFH7p42vsM+U/AgwPcWsiIiJuUVBxmw+411H7AFhqLzUAtjg2Ow34NVR9iYiIeICCipt6HB1WI4CD9tI/MN9yXGgPUBs4HLLGREREvEFBxQ1VgKGO2ufAV/6bLgHOtCwPBZ4OVV8iIiIeo6ASbmcDf3HUngX22kupwCbHZq2BVaHqS0RExIMUVMIlEXjIUfsOmOW/6Q3AJMvyFuBE4EiIWhMREfEqBZVwOBXo66g9j/+c98BKzDMnhe4CxoaoLREREa9TUAmleMykkWKp/Qa87b9pU+APR605sC4kjYmIiEQGBZVQOQkY6KhNxP8ZY+A2zIlnC60FWmKbiFZERCQmKaiEwvVAY8tyJjAZv+QRB6x3bHor8FJouxMREYkYCirB1ADzbcdW0yjx+k1zYI2j1gTYGIq+REREIpSCSrBcjTlVbKFcYDQlPqpzL+YTyYV+As4IYWsiIiKRKj5UB37wwQf55ptv2LdvH7t27Spxm8aNG/PRRx+xb98+srKyGDlyJAkJCaFqKTRqAenYQ8p04Dn8Qko8kI09pAxCIUVERKQ0ITujkpSUxPTp01m4cCHXX3+93/r4+Hg+/vhjtm7dSteuXWnYsCHTpk3j0KFDPPSQc8IRj7oY6OKoPYH5OmOHNpgP/Fg1osR7a0VERMTCCOUYMGCAsWvXLr967969jYKCAqNevXpFtZtvvtnYvXu3kZiYWObj+3w+wzAMw+fzhfT3sI1kDNIdo2Pp2z8EhmEZi8LVp4aGhoaGhkdHWf9+h+zSz/F06dKFn376iW3bthXV5syZQ40aNTjttNNK3S8pKQmfz2cbYXUOcL+j9jTwvf+mlYA84HFL7W9A5xC1JiIiEm1cu5m2QYMGZGVl2WqFyw0aNCh1v2HDhpGenh7K1kpWGRjmqC0A5pW8+enACketPrCthG1FRESkZAGdURkxYgSGYRxztG7d+vgHqoARI0aQkpJSNFJTU0P6eQC0wz+kjKLUkDIce0iZhzlnikKKiIhIYAI6o/Lcc8/x2muvHXObdevKNun71q1b6dixo61Wv379onWlyc/PJz8/v0yfUWEJwBAgyVJbCnxQ8uaJgLOzq4B3Q9CaiIhILAgoqGRnZ5OdnR2UD164cCEPPfQQdevWZfv27QBcdNFF7Nmzh19//TUon1EhyfjfizIe2F7y5mcBix21OsCOYPclIiISQ0J2M23jxo1JS0ujSZMmJCQkkJaWRlpaGsnJyQB8+umn/Prrr7z++uucccYZ9OrVi8cff5zx48eH74zJsbS1/LwGc66UUkLKM9hDyieYl3oUUkRERCouJI8dTZkyxShJ9+7di7Zp0qSJ8fHHHxv79u0ztm3bZjzzzDNGQkJCSB5vCnhUxqADBqmlb1MZ+2PHBhiXeuCRLw0NDQ0NDa+Psv79jjv6Q8Ty+Xzk5OSQkpJCbm5u2D63E7DIUasF7A5bByIiIpGrrH+/XZtHJZKNwx5S3sW81LPblW5ERESil15KGICqmBO4WV0MfOpCLyIiIrFAQaWMzgW+ctRSMF+SLCIiIqGhSz9lMBl7SHkd81KPQoqIiEho6YzKMSQDex21Cyh1QloREREJMgWVUtQHnPPjVgf2udCLiIhIrNKln1Jcbfn5ZcxLPQopIiIi4aUzKqV44+j/fgf84GYjIiIiMUxBpRS7gRfcbkJERCTG6dKPiIiIeJaCioiIiHiWgoqIiIh4loKKiIiIeJaCioiIiHiWgoqIiIh4loKKiIiIeJaCioiIiHiWgoqIiIh4loKKiIiIeJaCioiIiHiWgoqIiIh4loKKiIiIeFbUvD3Z5/O53YKIiIiUUVn/bkd8UCn8RTMzM13uRERERALl8/nIzc0tdX0cYISvndBo1KjRMX/JSOfz+cjMzCQ1NTWqf8+y0HdRTN9FMX0XJn0PxfRdFPPyd+Hz+di8efMxt4n4MyrAcX/JaJGbm+u5f8ncou+imL6LYvouTPoeium7KObF76Is/ehmWhEREfEsBRURERHxLAWVCHDw4EHS09M5ePCg2624Tt9FMX0XxfRdmPQ9FNN3USzSv4uouJlWREREopPOqIiIiIhnKaiIiIiIZymoiIiIiGcpqIiIiIhnKahEmA8++ID169ezf/9+Nm/ezLRp02jYsKHbbYVd06ZNmTx5MuvWrSMvL481a9aQnp5OYmKi26254sEHH+Sbb75h37597Nq1y+12wmrw4MFkZGSwf/9+Fi1aRIcOHdxuyRXdunVj5syZZGZmYhgGffr0cbslVwwdOpTvv/+enJwcsrKyeP/992nVqpXbbbnilltuYfny5ezZs4c9e/bw7bff0rt3b7fbCpiCSoSZN28effv2pXXr1vz1r3+lefPm/O9//3O7rbBr06YN8fHx3HzzzZx22mncfffd3HLLLTz55JNut+aKpKQkpk+fzoQJE9xuJaz69u3LqFGjePTRRznrrLNYvnw5c+bMoW7dum63FnbJycksX76c2267ze1WXNW9e3fGjx9P586dueiii0hMTOTTTz+lWrVqbrcWdps2bWLo0KG0b9+es88+my+++IIPPviAU0891e3WAmZoRO649NJLjcOHDxuVKlVyvRe3x3333WesXbvW9T7cHAMGDDB27drleh/hGosWLTLGjRtXtBwXF2ds2rTJGDJkiOu9uTkMwzD69Onjeh9eGHXq1DEMwzC6devmei9eGDt27DAGDRrkeh+BDJ1RiWC1atXib3/7G99++y0FBQVut+O6GjVqsHPnTrfbkDBJTEykffv2zJ07t6hmGAZz586lS5cuLnYmXlKjRg2AmP//hvj4ePr160dycjILFy50u52AKKhEoKeeeoq9e/eyc+dOmjRpErPXoq2aN2/O7bffzsSJE91uRcKkTp06VKpUiaysLFs9KyuLBg0auNSVeElcXBxjxozh66+/5pdffnG7HVe0bduW3NxcDh48yEsvvcQVV1zBb7/95nZbAVFQ8YARI0ZgGMYxR+vWrYu2f+aZZzjzzDO56KKLOHz4MNOmTXOx++AK9LsAaNSoEbNnz2b69OlMnjzZpc6DrzzfhYgUGz9+PG3btuWaa65xuxXX/P7777Rr145OnToxYcIEpk6dyimnnOJ2WwHRFPoeUKdOHWrXrn3MbdatW8ehQ4f86qmpqWzatIkuXbqwaNGiULUYNoF+Fw0bNmT+/PksWrSIgQMHYhjR869zef69GDBgAGPGjKFWrVqhbs91iYmJ5OXlcdVVV/HBBx8U1V977TVq1qzJ5Zdf7l5zLjMMg8svv9z2vcSacePG0adPH8477zz++OMPt9vxjM8++4y1a9dyyy23uN1KmVVyuwGB7OxssrOzy7VvfLx5Uqxy5crBbMk1gXwXjRo1Yt68eSxevJjrrrsuqkIKVOzfi1hw6NAhFi9eTM+ePYv+IMfFxdGzZ09eeOEFl7sTN40bN44rrriCHj16KKQ4xMfHR9zfCwWVCNKxY0c6dOjA119/za5du2jevDmPPfYYa9asibiboyqqUaNGzJ8/n/Xr13PffffZHkd13rMQCxo3bswJJ5xAkyZNSEhIIC0tDYA1a9awb98+l7sLnVGjRjF16lR+/PFHvv/+e+666y6Sk5OZMmWK262FXXJyMi1atChabtasGWlpaezcuZONGze62Fl4jR8/nv79+9OnTx9yc3OpX78+AHv27OHAgQMudxdeTz75JLNmzWLDhg34fD769+9Pjx49uPjii91uLWCuP3qkUbbRtm1b4/PPPzeys7ON/fv3G+vWrTNefPFFo1GjRq73Fu4xYMAAozRu9+bGmDJlSonfRffu3V3vLdTjtttuM/744w/jwIEDxqJFi4yOHTu63pMbo3v37iX+OzBlyhTXewvnKM2AAQNc7y3cY/LkyUZGRoZx4MABIysry/jss8+MCy+80PW+Ah26R0VEREQ8S0/9iIiIiGcpqIiIiIhnKaiIiIiIZymoiIiIiGcpqIiIiIhnKaiIiIiIZymoiIiIiGcpqIiIiIhnKaiIiIiIZymoiIiIiGcpqIiIiIhnKaiIiIiIZ/0/TzuKrQNSkOEAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# actual vs predicted\n",
    "\n",
    "plt.plot(xs, ys, c=\"r\", label=\"actual\")\n",
    "plt.plot(xs, lin_reg_model(trained_params, xs), c=\"g\", label=\"predicted\")\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So that's it. You now have an elementary understanding of a training loop using jax, (and or, flax) and optax. Let's recap a bit. You initialise your params, create the optimiser, initialise the optimiser states and then during training, update the optimiser state and params using optax. \n",
    "\n",
    "Also: make sure to check what you can `jit` and what you can't. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}