Lesson_7/simple-snn/Simple_RNN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Simple RNN\n",
    "\n",
    "In ths notebook, we're going to train a simple RNN to do **time-series prediction**. Given some set of input data, it should be able to generate a prediction for the next time step!\n",
    "<img src='assets/time_prediction.png' width=40% />\n",
    "\n",
    "> * First, we'll create our data\n",
    "* Then, define an RNN in PyTorch\n",
    "* Finally, we'll train our network and see how it performs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import resources and create data "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from torch import nn\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAecAAAEyCAYAAADA/hjIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAG0xJREFUeJzt3X+QVOW95/HPlxnGMQFMBUg2MupQudwlSAFem8EGUvY6msVbG4hZ3YC4XpIodZOwJtkkJbgp40ol5F7dtcrVJOtNLC+J8cfqqqxFyroZ7YjaRpqo2QAhNQFcB1JhLhrE3OAww3f/OM1kZuxhzjDndD/T/X5VTZ053c885zvPnD6fPk/3nDZ3FwAACMeEahcAAAAGI5wBAAgM4QwAQGAIZwAAAkM4AwAQGMIZAIDAEM4AAASGcAYAIDCEMwAAgWms1oanTZvmra2t1do8AAAVtWPHjn929+lx2lYtnFtbW1UsFqu1eQAAKsrMXovblmltAAACQzgDABAYwhkAgMBU7TVnAED4jh8/rq6uLh07dqzapYwbzc3Namlp0cSJE0+7D8IZADCsrq4uTZ48Wa2trTKzapcTPHfX4cOH1dXVpZkzZ552P0xrAwCGdezYMU2dOpVgjsnMNHXq1DHPNBDOAIBTIphHJ4nxGjGczexeMztkZr8a5n4zszvNrNPMfmlmfzXmqgAAqGNxzpzvk7TsFPdfLmlW6WutpO+OvSwAo1EoSJs2RcuwOwVGb/HixYn3uX//fv34xz9OvN+kjPiGMHd/1sxaT9FkhaTN7u6SXjSz95nZh9z9dwnVCOAUCgWpvV3q6ZGamqSODimbDbFT4PS88MILifd5MpyvvvrqxPtOQhKvOc+Q9PqA9a7Sbe9iZmvNrGhmxe7u7gQ2DSCfjzK0ry9a5vOhdoq6kfCsy6RJkyRJ+XxeuVxOV155pWbPnq3Vq1crOi+MLgl94403qq2tTW1tbers7JQkrVmzRo888si7+lq/fr22bdumBQsW6I477hh229u3b9e8efN07Ngx/fGPf9T555+vX/2q7Ku8iUriX6nKvfLt5Rq6+z2S7pGkTCZTtg1QywqFKOdyueRORHM5qamxTz0npKZGKZdrSKTTQsNS5U8sUa7heWVzubH3KaUzAAhLyrMuL7/8snbu3Kmzzz5bS5Ys0fPPP6+lS5dKkqZMmaKXXnpJmzdv1pe+9CU9+eSTw/bz7W9/W7fffvsp20jSwoULtXz5cn3961/Xn/70J11zzTWaO3duYr/PcJII5y5J5wxYb5F0MIF+gZqS1jErq4I6fIPyWqKcP6+sNkkaW8cFZdVuHeqRqclcHWoYY49iqrxelJt1SfDv3NbWppaWFknSggULtH///v5wXrVqVf/yy1/+cmLbvPnmm7Vw4UI1NzfrzjvvTKzfU0liWnuLpGtL79q+SNIRXm8G3i21meJ8Xtm+57TBv6Vs33OJdJzPSz29DerzCerpbWCqHPHlctGTr4aGaJnUrEvJGWec0f99Q0ODent7+9cH/gvTye8bGxt14sQJSdEFQnp6eka9zTfeeENvv/22jh49WrErpcX5V6oHJBUk/Wsz6zKzz5rZ35rZ35aabJW0V1KnpH+Q9PnUqgXGsdSOWSl0nEqtKR+0EYhsNpoV2bix4rMjDz30UP8yW9pua2urduzYIUl64okndPz4cUnS5MmTdfTo0f6fPXDggNrb28v2u3btWm3cuFGrV6/WjTfemOav0C/Ou7VXjXC/S/pCYhUBNerkMSvxl1xT6DiVWlMbAAQnm63K3/edd97RokWLdOLECT3wwAOSpOuvv14rVqxQW1ub2tvb9d73vleSNG/ePDU2Nmr+/Plas2aNPvrRj6qx8d2RuHnzZjU2Nurqq69WX1+fFi9erKefflqXXHJJqr+LnXynW6VlMhkvFotV2TYAIJ7du3frIx/5SLXLGFFra6uKxaKmTZt2Wj9/11136dxzz9Xy5csTqafcuJnZDnfPxPl5PvgCAFD31q1bV+0SBiGcAQDj3v79+6tdQqL44AsAAAJDOAPD4HrVyUvt16/zcUXtYVobKIPrVScvtV+/zscVtYkzZ6AMrledvDQvwlLP44raRDgDZXARjuSNp4uwIBx/+MMf9J3vfKci28rn86l8AtbpYFobKIOLcCRvPF2EBeE4Gc6f/3z8i0+6u9xdEyaM7vwzn89r0qRJqXx+9Kid/CUq/XXhhRc6ACBsu3btGvXPvPCC+7e+FS3H6lOf+pQ3Nzf7/Pnz/atf/aofPXrUL7nkEr/gggt87ty5/vjjj7u7+759+3z27Nn+uc99zhcsWOD79+/373//+z5r1iy/+OKL/brrrvMvfOEL7u5+6NAh/+QnP+mZTMYzmYw/99xzvm/fPv/gBz/oZ599ts+fP9+fffbZYWtaunSpv/zyy/3rixcv9ldffXVQm3LjJqnoMTOScAYADGu04fzCC+5nnune0BAtxxrQ+/bt8/PPP79//fjx437kyBF3d+/u7vYPf/jDfuLECd+3b5+bmRcKBXd3P3DggJ933nl++PBh7+np8aVLl/aH86pVq3zbtm3u7v7aa6/57Nmz3d39G9/4ht92220j1nTffff5F7/4RXd337Nnj5fLs7GGM9PaAIDEpPyJkXJ33XTTTXr22Wc1YcIEHThwQL///e8lSeedd54uuugiSdJLL72kiy++WO9///slSVdddZV+85vfSJJ++tOfateuXf19vvXWW4M+BGMkV111lTZu3KjbbrtN9957r9asWZPQb/dnhDMAIDEn35938j/bkn5/3v3336/u7m7t2LFDEydOVGtra//HOJ78UAspCvHhnDhxQoVCQWeeeeZp1fCe97xHl112mZ544gk9/PDDSuNzIni3NgAgMUl/YuTQj3Y8cuSIPvCBD2jixIl65pln9Nprr5X9uba2Nv3sZz/Tm2++qd7eXj366KP9933sYx/TXXfd1b/+yiuvlN3WY489pg0bNpTt/7rrrtMNN9yghQsX9p+dJ4lwBgAkKpuVNmxIZjp76tSpWrJkiebOnauvfe1rWr16tYrFojKZjO6//37Nnj277M/NmDFDN910kxYtWqRLL71Uc+bM0VlnnSVJuvPOO1UsFjVv3jzNmTNH3/ve9yRJH//4x/XYY49pwYIF2rZtm377299qypQpZfu/8MILNWXKFH36058e+y9ZBh8ZCQAY1nj5yMhy3n77bU2aNEm9vb264oor9JnPfEZXXHFF7J+/5pprdMcdd2j69Onvuu/gwYPK5XL69a9/XfZftsb6kZGcOQMAatItt9yiBQsWaO7cuZo5c6Y+8YlPjOrnf/SjH5UN5s2bN2vRokX65je/Oer/pY6LN4QBAGrS7bffnkq/1157ra699tpU+j6JM2cAwClV6+XP8SqJ8SKcURP4eMf6xt8/Pc3NzTp8+DABHZO76/Dhw2pubh5TP0xrY9zj4x3rG3//dLW0tKirq0vd3d3VLmXcaG5uVktLy5j6IJwx7qVyRaK0L3OExPD3T9fEiRM1c+bMapdRd5jWxrjHxzvWN/7+qEX8nzNqQqGQwicGptIp0sDfH+PBaP7PmXAGAKACuAgJAADjGOEMAEBgCGcAAAJDOAMAEBjCGQCAwBDOAAAEhnAGACAwhDMAAIEhnAEACAzhDABAYAhnAAACQzgDABAYwhkAgMAQzgAABIZwBgAgMLHC2cyWmdkeM+s0s/Vl7j/XzJ4xs5fN7Jdm9tfJl4paUChImzZFy/HRMepVKrsU+yliahypgZk1SLpb0mWSuiRtN7Mt7r5rQLOvS3rY3b9rZnMkbZXUmkK9GMcKBam9XerpkZqapI4OKZsNuWPUq1R2KfZTjEKcM+c2SZ3uvtfdeyQ9KGnFkDYuaUrp+7MkHUyuRNSKfD46LvX1Rct8PvSOUa9S2aXYTzEKccJ5hqTXB6x3lW4b6BZJ15hZl6Kz5v+USHWoKblcdMLQ0BAtc7nQO0a9SmWXYj/FKIw4rS3JytzmQ9ZXSbrP3f+bmWUl/dDM5rr7iUEdma2VtFaSzj333NOpF+NYNhvN5OXz0XEpsRm91DpGvUpll2I/xSiY+9CcHdIgCttb3P3fltY3SJK7bxrQZqekZe7+eml9r6SL3P3QcP1mMhkvFotj/w0AABgHzGyHu2fitI0zrb1d0iwzm2lmTZJWStoypM3/k9Re2vhHJDVL6o5fMgAAOGnEcHb3XknrJD0labeid2XvNLNbzWx5qdlXJF1vZq9KekDSGh/plBwAAJQV5zVnuftWRW/0GnjbzQO+3yVpSbKlAQBQn7hCGAAAgSGcAQAIDOEMAEBgCGcAAAJDOAMAEBjCGQCAwBDOAAAEhnAGACAwhDMAAIEhnAEACAzhDABAYAhnAAACQzgDABAYwhkAgMAQzgAABIZwBgAgMIQzhlUoSJs2RcuwOwXGh9R2fx5XNaex2gUgTIWC1N4u9fRITU1SR4eUzYbYKTA+pLb787iqSZw5o6x8Pnqs9/VFy3w+1E6B8SG13Z/HVU0inFFWLhc9CW9oiJa5XKidAuNDars/j6uaZO5elQ1nMhkvFotV2TbiKRSiJ+G5XIKzZKl0CowPqe3+PK7GBTPb4e6ZWG0JZwAA0jeacGZaGwCAwBDOAAAEhnAGACAwhDMAAIEhnAEACAzhDABAYAhnAAACQzgDABAYwhkAgMAQzgAABIZwBgAgMIQzAACBIZwBAAgM4QwAQGAIZwAAAkM4AwAQGMIZAIDAEM4AAASGcAYAIDCxwtnMlpnZHjPrNLP1w7T5D2a2y8x2mtmPky0TAID60ThSAzNrkHS3pMskdUnabmZb3H3XgDazJG2QtMTd3zSzD6RVMAAAtS7OmXObpE533+vuPZIelLRiSJvrJd3t7m9KkrsfSrZMAADqR5xwniHp9QHrXaXbBvpLSX9pZs+b2YtmtqxcR2a21syKZlbs7u4+vYoBAKhxccLZytzmQ9YbJc2SlJO0StL3zex97/oh93vcPePumenTp4+2VgyjUJA2bYqW46NjAElK5aHK47+qRnzNWdGZ8jkD1lskHSzT5kV3Py5pn5ntURTW2xOpEsMqFKT2dqmnR2pqkjo6pGw25I4BJCmVhyqP/6qLc+a8XdIsM5tpZk2SVkraMqTN45L+jSSZ2TRF09x7kywU5eXz0eOnry9a5vOhdwwgSak8VHn8V92I4ezuvZLWSXpK0m5JD7v7TjO71cyWl5o9Jemwme2S9Iykr7n74bSKxp/lctET24aGaJnLhd4xgCSl8lDl8V915j705ePKyGQyXiwWq7LtWlMoRE9sc7mEZ55S6xhAklJ5qPL4T5yZ7XD3TKy2hDMAAOkbTThz+U4AAAJDOAMAEBjCGQCAwBDOAAAEhnAGACAwhDMAAIEhnAEACAzhDABAYAhnAAACQzgDABAYwhkAgMAQzgAABIZwBgAgMIQzAACBIZwBAAgM4QwAQGAIZwAAAkM4AwAQGMIZAIDAEM4AAASGcAYAIDCEMwAAgSGcAQAIDOEMAEBgCGcAAAJDOAMAEBjCGQCAwBDOFVYoSJs2RcuwOwVQzzhWVVdjtQuoJ4WC1N4u9fRITU1SR4eUzYbYKYB6xrGq+jhzrqB8Ptov+/qiZT4faqcA6hnHquojnCsol4ueMDY0RMtcLtROAdQzjlXVZ+5elQ1nMhkvFotV2XY1FQrRE8ZcLsEZnVQ6BVDPOFYlz8x2uHsmVlvCGQCA9I0mnJnWBgAgMIQzAACBIZwBAAgM4QwAQGAIZwAAAkM4AwAQmFjhbGbLzGyPmXWa2fpTtLvSzNzMYr1VHAAAvNuI4WxmDZLulnS5pDmSVpnZnDLtJku6QdLPky4SAIB6EufMuU1Sp7vvdfceSQ9KWlGm3UZJfy/pWIL1AQBQd+KE8wxJrw9Y7yrd1s/MLpB0jrs/mWBtAADUpTjhbGVu67/mp5lNkHSHpK+M2JHZWjMrmlmxu7s7fpUAANSROOHcJemcAestkg4OWJ8saa6kvJntl3SRpC3l3hTm7ve4e8bdM9OnTz/9qgEAqGFxwnm7pFlmNtPMmiStlLTl5J3ufsTdp7l7q7u3SnpR0nJ351MtAAA4DSOGs7v3Slon6SlJuyU97O47zexWM1uedoEAANSbxjiN3H2rpK1Dbrt5mLa5sZcFAED94gphAAAEhnAGACAwhDMAAIEhnAEACAzhDABAYAhnAAACQzgDABAYwhkAgMAQzgAABIZwBgAgMIQzAACBIZwBAAgM4QwAQGAIZwAAAkM4AwAQGML5FAoFadOmaBl2pwAQvtQOfzV4XG2sdgGhKhSk9napp0dqapI6OqRsNsROASB8qR3+avS4ypnzMPL56G/d1xct8/lQOwWA8KV2+KvR4yrhPIxcLnoS1tAQLXO5UDsFgPCldvir0eOquXtVNpzJZLxYLFZl23EVCtGTsFwuwVmSVDoFgPCldvgbJ8dVM9vh7plYbQlnAADSN5pwZlobAIDAEM4AAASGcAYAIDCEMwAAgSGcAQAIDOEMAEBgCGcAAAJDOAMAEBjCGQCAwBDOAAAEhnAGACAwhDMAAIEhnAEACAzhDABAYAhnAAACQzgDABAYwhkAgMAQzgAABIZwBgAgMIQzAACBiRXOZrbMzPaYWaeZrS9z/382s11m9ksz6zCz85IvFQCA+jBiOJtZg6S7JV0uaY6kVWY2Z0izlyVl3H2epEck/X3ShQIAUC/inDm3Sep0973u3iPpQUkrBjZw92fc/V9Kqy9Kakm2TAAA6keccJ4h6fUB612l24bzWUk/KXeHma01s6KZFbu7u+NXCQBAHYkTzlbmNi/b0OwaSRlJt5W7393vcfeMu2emT58ev0oAAOpIY4w2XZLOGbDeIung0EZmdqmk/yLpYnd/J5nyAACoP3HOnLdLmmVmM82sSdJKSVsGNjCzCyT9T0nL3f1Q8mUCAFA/Rgxnd++VtE7SU5J2S3rY3Xea2a1mtrzU7DZJkyT9LzN7xcy2DNMdAAAYQZxpbbn7Vklbh9x284DvL024LgAA6hZXCAMAIDCEMwAAgSGcAQAITE2Ec6EgbdoULcdHxwCApKRyqK7y8T/WG8JCVihI7e1ST4/U1CR1dEjZbMgdAwCSksqhOoDj/7g/c87no/Hr64uW+XzoHQMAkpLKoTqA4/+4D+dcLnpi09AQLXO50DsGACQllUN1AMd/cy97mezUZTIZLxaLifRVKERPbHK5hGceUusYAJCUVA7VKXRqZjvcPROrbS2EMwAAoRtNOI/7aW0AAGoN4QwAQGAIZwAAAkM4AwAQGMIZAIDAEM4AAASGcAYAIDCEMwAAgSGcAQAIDOEMAEBgCGcAAAJDOAMAEBjCGQCAwBDOAAAEhnAGACAwhDMAAIEhnAEACAzhDABAYAhnAAACQzgDABAYwhkAgMAQzgAABIZwBgAgMIQzAACBIZwBAAgM4QwAQGAIZwAAAkM4AwAQGMIZAIDAEM4AAASGcAYAIDCxwtnMlpnZHjPrNLP1Ze4/w8weKt3/czNrTbpQAADqxYjhbGYNku6WdLmkOZJWmdmcIc0+K+lNd/8LSXdI+rukCz2lQkHatClaAgAwRtWOlcYYbdokdbr7XkkyswclrZC0a0CbFZJuKX3/iKS7zMzc3ROstbxCQWpvl3p6pKYmqaNDymZT3ywAoDaFECtxprVnSHp9wHpX6baybdy9V9IRSVOHdmRma82saGbF7u7u06t4qHw+GsG+vmiZzyfTLwCgLoUQK3HC2crcNvSMOE4bufs97p5x98z06dPj1DeyXC56atPQEC1zuWT6BQDUpRBiJc60dpekcwast0g6OEybLjNrlHSWpDcSqXAk2Ww055DPRyPIlDYAYAxCiJU44bxd0iwzmynpgKSVkq4e0maLpL+RVJB0paSnK/J680nZLKEMAEhMtWNlxHB2914zWyfpKUkNku51951mdqukortvkfQDST80s05FZ8wr0ywaAIBaFufMWe6+VdLWIbfdPOD7Y5KuSrY0AADqE1cIAwAgMIQzAACBIZwBAAgM4QwAQGAIZwAAAkM4AwAQGMIZAIDAWCUv5DVow2bdkl6rysbDM03SP1e7iIAwHoMxHoMxHoMxHoOFPB7nuXusD5aoWjjjz8ys6O6ZatcRCsZjMMZjMMZjMMZjsFoZD6a1AQAIDOEMAEBgCOcw3FPtAgLDeAzGeAzGeAzGeAxWE+PBa84AAASGM2cAAAJDOAMAEBjCuYLMbJmZ7TGzTjNbX+b+NWbWbWavlL6uq0adlWBm95rZITP71TD3m5ndWRqrX5rZX1W6xkqKMR45MzsyYN+4uVy7WmFm55jZM2a228x2mtkXy7Spm30k5njUzT5iZs1m9pKZvVoaj/9aps0ZZvZQaf/4uZm1Vr7S09dY7QLqhZk1SLpb0mWSuiRtN7Mt7r5rSNOH3H1dxQusvPsk3SVp8zD3Xy5pVulrkaTvlpa16j6dejwkaZu7/7vKlFN1vZK+4u6/MLPJknaY2T8NebzU0z4SZzyk+tlH3pF0ibu/bWYTJT1nZj9x9xcHtPmspDfd/S/MbKWkv5P0qWoUezo4c66cNkmd7r7X3XskPShpRZVrqhp3f1bSG6doskLSZo+8KOl9ZvahylRXeTHGo664++/c/Rel749K2i1pxpBmdbOPxByPulH6m79dWp1Y+hr67uYVkv6x9P0jktrNzCpU4pgRzpUzQ9LrA9a7VP7B9e9LU3SPmNk5lSktSHHHq55kS9N4PzGz86tdTKWUpiMvkPTzIXfV5T5yivGQ6mgfMbMGM3tF0iFJ/+Tuw+4f7t4r6YikqZWt8vQRzpVT7hnb0Gd6/0dSq7vPk/RT/flZXz2KM1715BeKrss7X9L/kPR4leupCDObJOlRSV9y97eG3l3mR2p6HxlhPOpqH3H3PndfIKlFUpuZzR3SZFzvH4Rz5XRJGngm3CLp4MAG7n7Y3d8prf6DpAsrVFuIRhyveuLub52cxnP3rZImmtm0KpeVqtJriY9Kut/d/3eZJnW1j4w0HvW4j0iSu/9BUl7SsiF39e8fZtYo6SyNo5eOCOfK2S5plpnNNLMmSSslbRnYYMjrZcsVva5Ur7ZIurb0jtyLJB1x999Vu6hqMbN/dfL1MjNrU/TYPVzdqtJT+l1/IGm3u//3YZrVzT4SZzzqaR8xs+lm9r7S92dKulTSr4c02yLpb0rfXynpaR9HV93i3doV4u69ZrZO0lOSGiTd6+47zexWSUV33yLpBjNbruidmW9IWlO1glNmZg9IykmaZmZdkr6h6E0dcvfvSdoq6a8ldUr6F0mfrk6llRFjPK6U9Dkz65X0J0krx9OB5jQskfQfJf3f0uuKknSTpHOlutxH4oxHPe0jH5L0j6X/gpkg6WF3f3LI8fQHkn5oZp2Kjqcrq1fu6HH5TgAAAsO0NgAAgSGcAQAIDOEMAEBgCGcAAAJDOAMAEBjCGQCAwBDOAAAE5v8D/l6MKUP+AowAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(8,5))\n",
    "\n",
    "# how many time steps/data pts are in one batch of data\n",
    "seq_length = 20\n",
    "\n",
    "# generate evenly spaced data pts\n",
    "time_steps = np.linspace(0, np.pi, seq_length + 1)\n",
    "data = np.sin(time_steps)\n",
    "data.resize((seq_length + 1, 1)) # size becomes (seq_length+1, 1), adds an input_size dimension\n",
    "\n",
    "x = data[:-1] # all but the last piece of data\n",
    "y = data[1:] # all but the first\n",
    "\n",
    "# display the data\n",
    "plt.plot(time_steps[1:], x, 'r.', label='input, x') # x\n",
    "plt.plot(time_steps[1:], y, 'b.', label='target, y') # y\n",
    "\n",
    "plt.legend(loc='best')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Define the RNN\n",
    "\n",
    "Next, we define an RNN in PyTorch. We'll use `nn.RNN` to create an RNN layer, then we'll add a last, fully-connected layer to get the output size that we want. An RNN takes in a number of parameters:\n",
    "* **input_size** - the size of the input\n",
    "* **hidden_dim** - the number of features in the RNN output and in the hidden state\n",
    "* **n_layers** - the number of layers that make up the RNN, typically 1-3; greater than 1 means that you'll create a stacked RNN\n",
    "* **batch_first** - whether or not the input/output of the RNN will have the batch_size as the first dimension (batch_size, seq_length, hidden_dim)\n",
    "\n",
    "Take a look at the [RNN documentation](https://pytorch.org/docs/stable/nn.html#rnn) to read more about recurrent layers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "class RNN(nn.Module):\n",
    "    def __init__(self, input_size, output_size, hidden_dim, n_layers):\n",
    "        super(RNN, self).__init__()\n",
    "        \n",
    "        self.hidden_dim=hidden_dim\n",
    "\n",
    "        # define an RNN with specified parameters\n",
    "        # batch_first means that the first dim of the input and output will be the batch_size\n",
    "        self.rnn = nn.RNN(input_size, hidden_dim, n_layers, batch_first=True)\n",
    "        \n",
    "        # last, fully-connected layer\n",
    "        self.fc = nn.Linear(hidden_dim, output_size)\n",
    "\n",
    "    def forward(self, x, hidden):\n",
    "        # x (batch_size, seq_length, input_size)\n",
    "        # hidden (n_layers, batch_size, hidden_dim)\n",
    "        # r_out (batch_size, time_step, hidden_size)\n",
    "        batch_size = x.size(0)\n",
    "        \n",
    "        # get RNN outputs\n",
    "        r_out, hidden = self.rnn(x, hidden)\n",
    "        # shape output to be (batch_size*seq_length, hidden_dim)\n",
    "        r_out = r_out.view(-1, self.hidden_dim)  \n",
    "        \n",
    "        # get final output \n",
    "        output = self.fc(r_out)\n",
    "        \n",
    "        return output, hidden\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check the input and output dimensions\n",
    "\n",
    "As a check that your model is working as expected, test out how it responds to input data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input size:  torch.Size([1, 20, 1])\n",
      "Output size:  torch.Size([20, 1])\n",
      "Hidden state size:  torch.Size([2, 1, 10])\n"
     ]
    }
   ],
   "source": [
    "# test that dimensions are as expected\n",
    "test_rnn = RNN(input_size=1, output_size=1, hidden_dim=10, n_layers=2)\n",
    "\n",
    "# generate evenly spaced, test data pts\n",
    "time_steps = np.linspace(0, np.pi, seq_length)\n",
    "data = np.sin(time_steps)\n",
    "data.resize((seq_length, 1))\n",
    "\n",
    "test_input = torch.Tensor(data).unsqueeze(0) # give it a batch_size of 1 as first dimension\n",
    "print('Input size: ', test_input.size())\n",
    "\n",
    "# test out rnn sizes\n",
    "test_out, test_h = test_rnn(test_input, None)\n",
    "print('Output size: ', test_out.size())\n",
    "print('Hidden state size: ', test_h.size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Training the RNN\n",
    "\n",
    "Next, we'll instantiate an RNN with some specified hyperparameters. Then train it over a series of steps, and see how it performs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RNN(\n",
      "  (rnn): RNN(1, 32, batch_first=True)\n",
      "  (fc): Linear(in_features=32, out_features=1, bias=True)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "# decide on hyperparameters\n",
    "input_size=1 \n",
    "output_size=1\n",
    "hidden_dim=32\n",
    "n_layers=1\n",
    "\n",
    "# instantiate an RNN\n",
    "rnn = RNN(input_size, output_size, hidden_dim, n_layers)\n",
    "print(rnn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loss and Optimization\n",
    "\n",
    "This is a regression problem: can we train an RNN to accurately predict the next data point, given a current data point?\n",
    "\n",
    ">* The data points are coordinate values, so to compare a predicted and ground_truth point, we'll use a regression loss: the mean squared error.\n",
    "* It's typical to use an Adam optimizer for recurrent models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# MSE loss and Adam optimizer with a learning rate of 0.01\n",
    "criterion = nn.MSELoss()\n",
    "optimizer = torch.optim.Adam(rnn.parameters(), lr=0.01) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Defining the training function\n",
    "\n",
    "This function takes in an rnn, a number of steps to train for, and returns a trained rnn. This function is also responsible for displaying the loss and the predictions, every so often.\n",
    "\n",
    "#### Hidden State\n",
    "\n",
    "Pay close attention to the hidden state, here:\n",
    "* Before looping over a batch of training data, the hidden state is initialized\n",
    "* After a new hidden state is generated by the rnn, we get the latest hidden state, and use that as input to the rnn for the following steps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# train the RNN\n",
    "def train(rnn, n_steps, print_every):\n",
    "    \n",
    "    # initialize the hidden state\n",
    "    hidden = None      \n",
    "    \n",
    "    for batch_i, step in enumerate(range(n_steps)):\n",
    "        # defining the training data \n",
    "        time_steps = np.linspace(step * np.pi, (step+1)*np.pi, seq_length + 1)\n",
    "        data = np.sin(time_steps)\n",
    "        data.resize((seq_length + 1, 1)) # input_size=1\n",
    "\n",
    "        x = data[:-1]\n",
    "        y = data[1:]\n",
    "        \n",
    "        # convert data into Tensors\n",
    "        x_tensor = torch.Tensor(x).unsqueeze(0) # unsqueeze gives a 1, batch_size dimension\n",
    "        y_tensor = torch.Tensor(y)\n",
    "\n",
    "        # outputs from the rnn\n",
    "        prediction, hidden = rnn(x_tensor, hidden)\n",
    "\n",
    "        ## Representing Memory ##\n",
    "        # make a new variable for hidden and detach the hidden state from its history\n",
    "        # this way, we don't backpropagate through the entire history\n",
    "        hidden = hidden.data\n",
    "\n",
    "        # calculate the loss\n",
    "        loss = criterion(prediction, y_tensor)\n",
    "        # zero gradients\n",
    "        optimizer.zero_grad()\n",
    "        # perform backprop and update weights\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        # display loss and predictions\n",
    "        if batch_i%print_every == 0:        \n",
    "            print('Loss: ', loss.item())\n",
    "            plt.plot(time_steps[1:], x, 'r.') # input\n",
    "            plt.plot(time_steps[1:], prediction.data.numpy().flatten(), 'b.') # predictions\n",
    "            plt.show()\n",
    "    \n",
    "    return rnn\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loss:  0.7399086356163025\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAE1lJREFUeJzt3X+M5Pd91/HnK2tfg5RfInelxnfuGXGVMFFpwurIKlJZdCm65A/fH6TRpSqNq6SngkxBVAhDK7c4QhdaQaVSK+FQrCYpxHYDbZdwkUWvWbWq1q732iTENi5X09aLLbxNExcU2uOub/6YuWQ9N3u73vnefGf283xIq5nvzGfm8/nu9/t9zWc+8/2RqkKS1JbX9N0ASdL0Gf6S1CDDX5IaZPhLUoMMf0lqkOEvSQ0y/CWpQYa/JDXI8JekBt3SdwO2c/DgwTp69GjfzZCkuXLx4sU/qKpDO5Wb2fA/evQo6+vrfTdDkuZKkt/bTTmHfSSpQYa/JDXI8JekBhn+ktQgw1+SGtRJ+Cd5KMlLSb60zfNJ8tNJLiX5YpK3dVGvJGlvuur5/yxw8gbPvws4Nvw7A3yko3ql2bO2BmfPDm6lGdXJfv5V9atJjt6gyCngEzW4ZuTjSd6U5LaqerGL+qWZsbYGJ07A5ctw4ABcuABLS323SrrOtMb8bwee3zK9MXzsFZKcSbKeZH1zc3NKTZNGTNJzX10dBP/Vq4Pb1dXp1i/t0rSO8M2Yx667cnxVnQPOASwuLnpleU3fpD335eXB6669fnl5uvVLuzStnv8GcGTL9GHghSnVLe3epD33paVBYH/oQ3sL7i6+OUi7MK2e/wpwb5KHgb8GvOx4v2bSpD13GAT+XnvrXdQv7UIn4Z/kU8AycDDJBvBjwK0AVfVR4DzwbuAS8DXg+7uoV+rctZ776uogeKc95NJ3/WpGBjvgzJ7FxcXyrJ6S9OokuVhVizuV8whf7T+t7y3T+vxrV2b2fP7SnrS+t0zr869ds+ev/aX1vWVan3/tmuGv/eXa3jILC23uLdP6/GvXHPbR/tL63jKtz792zb19JGkfcW8fSdK2DH9JapDhL0kNMvwlqUGGv2aPR6j2y/9/E9zVU7PFI1T75f+/Gfb8NVs8QrVf/v+bYfhrtniEar/8/zfDYR/NFo9Q7Zf//2Z4hK8k7SMe4StJ2pbhL0kNMvwlqUGGvyQ1yPCXpAYZ/pLUIMNfkhrUSfgnOZnk2SSXktw35vk7knwuyW8l+WKSd3dRr2aUJwZrm8t/Lkx8hG+SBeBB4LuADeDJJCtV9fSWYj8KPFpVH0lyF3AeODpp3ZpBnhisbS7/udFFz/84cKmqnquqy8DDwKmRMgW8YXj/jcALHdSrWeSJwdrm8p8bXYT/7cDzW6Y3ho9t9ePA9ybZYNDr/3vj3ijJmSTrSdY3Nzc7aJqmzhODtc3lPze6OLFbxjw2esKg9wE/W1X/MskS8Mkkb6mqP33Fi6rOAedgcG6fDtqmafPEYG1z+c+NLsJ/AziyZfow1w/rfAA4CVBVa0leCxwEXuqgfs2apSU3+pa5/OdCF8M+TwLHktyZ5ABwGlgZKfP7wAmAJH8JeC3guI4k9WTi8K+qK8C9wGPAMwz26nkqyQNJ7h4W+2HgB5J8AfgUcE/N6rmkJakBnVzMparOM/ghd+tj92+5/zTwji7qkiRNziN8JalBhr8kNcjwl6QGGf6S1CDDX5IaZPhLUoMMf0lqkOGv63k+dvXJ9W8qOjnIS/uI52NXn1z/psaev17J87GrT65/U2P465U8H7v65Po3NQ776JU8H7v65Po3NZnVk2suLi7W+vp6382QpLmS5GJVLe5UzmEfSWqQ4S9JDTL8JalBhr8kNcjwl6QGGf6S1CDDX5IaZPhLUoMMf0lqkOEvSQ3qJPyTnEzybJJLSe7bpsx7kzyd5Kkk/76LeiVJezPxid2SLAAPAt8FbABPJlmpqqe3lDkG/BPgHVX1lSTfPGm9kqS966Lnfxy4VFXPVdVl4GHg1EiZHwAerKqvAFTVSx3UK0naoy7C/3bg+S3TG8PHtvo24NuS/HqSx5OcHPdGSc4kWU+yvrm52UHTGuVl8NQy1/9d6eJ8/hnz2Oh5om8BjgHLwGHg15K8paq++ooXVZ0DzsHglM4dtK09XgZPLXP937Uuev4bwJEt04eBF8aU+aWq+n9V9T+AZxl8GKhrXgZPLXP937Uuwv9J4FiSO5McAE4DKyNlfhH4GwBJDjIYBnqug7o1ysvgqWWu/7s28bBPVV1Jci/wGLAAPFRVTyV5AFivqpXhc38zydPAVeAfVdWXJ61bY3gZPLXM9X/XvIyjJO0jXsZRkrQtw1+SGmT4S1KDDH9JapDhL0kNMvwlqUGGvyQ1yPCXpAYZ/pLUIMNfkhpk+EtSgwx/SWqQ4S9JDTL8JalBhr8kNcjwl6QGGf6zaG0Nzp4d3Eqarka2v4kv46iOra3BiRODi08fODC4JJ2XopOmo6Htz57/rFldHax4V68ObldX+26R1I6Gtj/Df9YsLw96HAsLg9vl5b5bJLWjoe3PYZ9Zs7Q0+Kq5ujpY8fbpV05pJjW0/aWq+m7DWIuLi7W+vt53MyRpriS5WFWLO5Vz2EeSGtRJ+Cc5meTZJJeS3HeDcu9JUkl2/FSSJN08E4d/kgXgQeBdwF3A+5LcNabc64EfAp6YtE5J0mS66PkfBy5V1XNVdRl4GDg1ptyHgJ8A/riDOiVJE+gi/G8Hnt8yvTF87OuSvBU4UlWfudEbJTmTZD3J+ubmZgdNkySN00X4Z8xjX9+FKMlrgJ8CfninN6qqc1W1WFWLhw4d6qBpkqRxugj/DeDIlunDwAtbpl8PvAVYTfK7wNuBFX/0laT+dBH+TwLHktyZ5ABwGli59mRVvVxVB6vqaFUdBR4H7q4qd+KXpJ5MHP5VdQW4F3gMeAZ4tKqeSvJAkrsnfX9JUvc6Ob1DVZ0Hzo88dv82ZZe7qFOStHce4StJDTL8JalBhr8kNcjwl6QGGf6S1CDDX5IaZPhLUoMM/5thbQ3Onh3cSmrLnGz/XsO3a2trcOIEXL48uAD0hQv7+jqgkraYo+3fnn/XVlcHC/7q1cHt6mrfLZI0LXO0/Rv+XVteHnziLywMbpeX+26RpGmZo+3fYZ+uLS0Nvuqtrg4W/Ix+5ZN0E8zR9p+q2rlUDxYXF2t93bM+S9KrkeRiVe14vRSHfSSpQYa/JDXI8JekBhn+ktQgw1+SGmT4S1KDDH9JapDhL0kNMvwlqUGGvyQ1qJPwT3IyybNJLiW5b8zz/zDJ00m+mORCkm/tol5J0t5MHP5JFoAHgXcBdwHvS3LXSLHfAhar6tuBTwM/MWm9kqS966Lnfxy4VFXPVdVl4GHg1NYCVfW5qvracPJx4HAH9UqS9qiL8L8deH7L9Mbwse18APjsuCeSnEmynmR9c3Ozg6ZJksbpIvwz5rGx54lO8r3AIvCT456vqnNVtVhVi4cOHeqgaZKkcbq4mMsGcGTL9GHghdFCSd4J/Ajw16vqTzqoV5K0R130/J8EjiW5M8kB4DSwsrVAkrcC/wa4u6pe6qBOSdIEJg7/qroC3As8BjwDPFpVTyV5IMndw2I/CbwO+Pkkn0+yss3bSZKmoJNr+FbVeeD8yGP3b7n/zi7qmZq1tbm4BqekfWhK+eMF3EetrcGJE3D5Mhw4MLgYsx8AkqZhivnj6R1Gra4O/vFXrw5uV1f7bpGkVkwxfwz/UcvLg0/chYXB7fJy3y2S1Iop5o/DPqOWlgZftRzzlzRtU8yfVI09Hqt3i4uLtb6+3nczJGmuJLlYVYs7lXPYR5IaZPhLUoMMf0lqkOEvSQ0y/CWpQYa/JDXI8JekBhn+ktQgw1+SGmT4S1KDDH9JapDhL0kNMvwlqUGGvyQ1yPCXpAYZ/pLUIMNfkhpk+EtSgzoJ/yQnkzyb5FKS+8Y8/01JHhk+/0SSo13UK0nam4nDP8kC8CDwLuAu4H1J7hop9gHgK1X1F4GfAv7FpPXe0NoanD07uJUkXeeWDt7jOHCpqp4DSPIwcAp4ekuZU8CPD+9/GviZJKmbcfX4tTU4cQIuX4YDB+DCBVha6rwaSZpnXQz73A48v2V6Y/jY2DJVdQV4GXjz6BslOZNkPcn65ubm3lqzujoI/qtXB7erq3t7H0nax7oI/4x5bLRHv5syVNW5qlqsqsVDhw7trTXLy4Me/8LC4HZ5eW/vI0n7WBfDPhvAkS3Th4EXtimzkeQW4I3AH3ZQ9/WWlgZDPaurg+B3yEeSrtNF+D8JHEtyJ/A/gdPA94yUWQHeD6wB7wF+5aaM91+ztGToS9INTBz+VXUlyb3AY8AC8FBVPZXkAWC9qlaAjwGfTHKJQY//9KT1SpL2roueP1V1Hjg/8tj9W+7/MfDdXdQlSZqcR/hKUoMMf0lqkOEvSQ0y/CWpQYa/JDXI8JekBhn+ktQgw1+SGmT4S1KDDH9JapDhL0kNMvwlqUGGvyQ1yPCXpAYZ/tKMWVuDs2cHt/P4es2HTs7nL+kb1tb2fhXRtTU4cQIuXx5cgvrChVf3Hn2//tp7eBXV2bcvw9+VT5PoM7xXVwevvXp1cLu6Ol+v98Njfuy78O9i5VO7+g7v5eVBvdfqX15+de3v+/V+eMyPfRf+k658mn+TbPx9h/fS0iDw9tr+vl+/Hz48WrHvwn/SlU/zbdKNv+/wvvYekwRWn6+f9w8PaOebw74L/y42PvWrz577LIT3vJvnD4+Whp32XfiDG98867vnDq4/fevzw6OlYad9Gf7q17z33DXfJvnwmIVhp2mZKPyT/FngEeAo8LvAe6vqKyNlvgP4CPAG4Crwz6vqkUnq1eyy56551vewE0xv2GjSnv99wIWq+nCS+4bT/3ikzNeA76uq/57kzwMXkzxWVV+dsG7dJPbc1bI+h52mOWw0afifApaH9z8OrDIS/lX121vuv5DkJeAQMLPhPy8/2NwM9tylyUyy/k5z2GjS8P9zVfUiQFW9mOSbb1Q4yXHgAPA7E9Z708zTDzY3gz13qT/T3FV9x/BP8svAt4x56kdeTUVJbgM+Cby/qv50mzJngDMAd9xxx6t5+87M0w8225nkm4s9d6k/0+w87Rj+VfXO7Z5L8r+S3Dbs9d8GvLRNuTcA/xn40ap6/AZ1nQPOASwuLtZObbsZ5v0gsUm/udhzl/o1rc7TpMM+K8D7gQ8Pb39ptECSA8AvAJ+oqp+fsL6brovw6/M3gy6+udhzl/a/ScP/w8CjST4A/D7w3QBJFoEfrKoPAu8FvhN4c5J7hq+7p6o+P2HdN80k4df3EYLz/s1F0nRMFP5V9WXgxJjH14EPDu//HPBzk9QzT/o+QtBhG0m74RG+HZuFIwQdtpG0E8O/Y7NwhKAk7cTwvwn6PEJQknbD8J9BDttIutle03cDJEnTZ/hLUoMMf0lqkOEvSQ0y/CWpQYa/JDUoVb2cPHNHSTaB3+u7HR05CPxB343oiPMym5yX2dTHvHxrVR3aqdDMhv9+kmS9qhb7bkcXnJfZ5LzMplmeF4d9JKlBhr8kNcjwn45zfTegQ87LbHJeZtPMzotj/pLUIHv+ktQgw78jSU4meTbJpST3jXn+niSbST4//PtgH+3cjSQPJXkpyZe2eT5Jfno4r19M8rZpt3G3djEvy0le3rJc7p92G3cjyZEkn0vyTJKnkvz9MWXmYrnscl7mZbm8NslvJPnCcF7+2Zgy35TkkeFyeSLJ0em3dIyq8m/CP2AB+B3gLwAHgC8Ad42UuQf4mb7busv5+U7gbcCXtnn+3cBngQBvB57ou80TzMsy8Jm+27mL+bgNeNvw/uuB3x6zjs3FctnlvMzLcgnwuuH9W4EngLePlPm7wEeH908Dj/Td7qqy59+R48Clqnquqi4DDwOnem7TnlXVrwJ/eIMip4BP1MDjwJuS3Dad1r06u5iXuVBVL1bVbw7v/2/gGeD2kWJzsVx2OS9zYfi//j/DyVuHf6M/pJ4CPj68/2ngRJJMqYnbMvy7cTvw/JbpDcavzH9r+HX800mOTKdpN8Vu53deLA2/tn82yV/uuzE7GQ4bvJVBL3OruVsuN5gXmJPlkmQhyeeBl4D/UlXbLpequgK8DLx5uq28nuHfjXGf4qOf/v8JOFpV3w78Mt/oCcyj3czvvPhNBofD/xXgXwO/2HN7bijJ64D/APyDqvqj0afHvGRml8sO8zI3y6WqrlbVdwCHgeNJ3jJSZCaXi+HfjQ1ga0/+MPDC1gJV9eWq+pPh5L8F/uqU2nYz7Di/86Kq/uja1/aqOg/cmuRgz80aK8mtDMLy31XVfxxTZG6Wy07zMk/L5Zqq+iqwCpwceerryyXJLcAbmYGhSMO/G08Cx5LcmeQAgx91VrYWGBl7vZvBOOe8WgG+b7h3yduBl6vqxb4btRdJvuXa+GuS4wy2iS/326rrDdv4MeCZqvpX2xSbi+Wym3mZo+VyKMmbhvf/DPBO4L+NFFsB3j+8/x7gV2r462+fvIB7B6rqSpJ7gccY7PnzUFU9leQBYL2qVoAfSnI3cIXBp/49vTV4B0k+xWBvi4NJNoAfY/BDFlX1UeA8gz1LLgFfA76/n5bubBfz8h7g7yS5Avxf4PQsbJhjvAP428B/HY4vA/xT4A6Yu+Wym3mZl+VyG/DxJAsMPqAerarPjGz7HwM+meQSg23/dH/N/QaP8JWkBjnsI0kNMvwlqUGGvyQ1yPCXpAYZ/pLUIMNfkhpk+EtSgwx/SWrQ/wfs8H9r8NjzbAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loss:  0.006314641796052456\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAFCBJREFUeJzt3X+wXGddx/H3t2lvmVHKJOkPAiWmQtFWywheCysD3GlafsmQ1lIoqISxNXSAGdERDcPAOPSPBhi1OjBqLGqAQahUaC0VSC9ewPGCvRE0tAykLQVCM22agsqgvTT9+seeDMnN2dxNzt7ds/u8XzN39uzuc+55nuyeT5777HOejcxEklSWk0ZdAUnS8Bn+klQgw1+SCmT4S1KBDH9JKpDhL0kFMvwlqUCGvyQVyPCXpAKdPOoK9HL66afnhg0bRl0NSRoru3bteigzz1iuXGvDf8OGDSwsLIy6GpI0ViLiW/2Uc9hHkgpk+EtSgQx/SSqQ4S9JBTL8JalAhr8kFWgiw39+Hq67rnsrSTpaa+f5n6j5edi4ERYXYWoKZmeh0xl1rSSpXSau5z831w3+gwe7t3Nzo66RJLXPxIX/zEy3x79qVfd2ZmbUNZKk9hlI+EfEiyPi6xFxd0RsrXn+1Ij4aPX8lyJiwyCOW6fT6Q71XHutQz6S1EvjMf+IWAW8D7gE2AvcERG3ZOZdhxW7CvheZj4tIq4E3gW8qumxe+l0DH1JOpZB9PwvBO7OzHszcxH4CLBpSZlNwI5q+2PAxoiIARy7ntN9JOmYBjHb58nAdw67vxd4dq8ymfloRPwXsBZ4aADHP5LTfSRpWYPo+df14PMEyhARWyJiISIW9u/ff2K1cbqPJC1rEOG/F3jKYffPBu7vVSYiTgaeADy89Bdl5vbMnM7M6TPOWPa7COo53UeSljWIYZ87gHMj4hzgu8CVwGuWlLkF2AzMA68APpuZR/X8B+LQdJ+5uW7wO+QjSUdpHP7VGP6bgE8Dq4C/zsw7I+KdwEJm3gK8H/hgRNxNt8d/ZdPjHpPTfSTpmAayvENm3gbctuSxdxy2/X/AFYM4liSpuYm7wleSxtmwZqpP3MJukjSuhjlT3Z6/JLXEMGeqG/6S1BLDnKnusI8ktcQwZ6ob/pLUIsOaqe6wjyQVyPCXpAIZ/pJUIMNfkgpk+EtSgQx/SSqQ4S9JBTL8JalAhr8kFcjwl6QCGf6SVCDDX5IKZPhLUoEMf0kqkOEvSQUy/CWpQIZ/nfl5uO667q0kDdOQ8sdv8lpqfh42bux+e/LUVPc71YbxtTqSNMT8see/1Nxc9x/+4MHu7dzcqGskqRRDzB/Df6mZme7/uKtWdW9nZkZdI0mlGGL+OOyzVKfT/VNrbq77D++Qj6RhGWL+RGau2C9vYnp6OhcWFkZdDUkaKxGxKzOnlyvXaNgnItZExM6I2FPdrq4p8wsRMR8Rd0bEf0bEq5occxic7CNp0jUd9tkKzGbmtojYWt3/gyVlfgi8NjP3RMSTgF0R8enM/H7DY68IJ/tIKkHTD3w3ATuq7R3ApUsLZOY3MnNPtX0/8CBwRsPjrhgn+0gqQdPwPysz9wFUt2ceq3BEXAhMAfc0PO6KcbKPpBIsO+wTEbcDT6x56m3Hc6CIWAd8ENicmY/1KLMF2AKwfv364/n1A+NkH0klaDTbJyK+Dsxk5r4q3Ocy82dqyp0GzAHXZebf9/O7ne0jScdvKLN9gFuAzdX2ZuDmmopMAR8HPtBv8EvSuBqX2YJNZ/tsA26MiKuAbwNXAETENHBNZl4NvBJ4PrA2Il5X7fe6zPxKw2NLUquM02zBRuGfmQeAjTWPLwBXV9sfAj7U5DiSNA7qZgu2Nfxd20eSBmScZgu6to8kDcg4zRY0/CVpgDqddof+IQ77SFKBDH9JKpDhL0kFMvwlqUCGvyQVyPCXpAIZ/pJUIMNfkgpk+EtSgQx/SSqQ4b8SxmVBb0mDNybnv2v7DNo4LegtabDG6Py35z9odQt6SyrDGJ3/hv+gjdOC3pIGa4zOf4d9Bm2cFvSWNFhjdP5HZo66DrWmp6dzYWFh1NWQpLESEbsyc3q5cg77SFKBDH9JKpDhL0kFMvwlqUCGvyQVyPCXpAIZ/pJUIMNfkgpk+EtSgQx/SSpQo/CPiDURsTMi9lS3q49R9rSI+G5EvLfJMcfBmCznLalgTXv+W4HZzDwXmK3u93It8LmGx2u9Q8t5v/3t3Vv/A5DGSymdt6bhvwnYUW3vAC6tKxQRvwicBXym4fFab4yW85a0REmdt6bhf1Zm7gOobs9cWiAiTgL+CHjLcr8sIrZExEJELOzfv79h1UZjjJbzlrRESZ23Zdfzj4jbgSfWPPW2Po/xBuC2zPxORByzYGZuB7ZDd0nnPn9/q4zRct6SljjUeTv0LYyT3HlbNvwz8+Jez0XEAxGxLjP3RcQ64MGaYh3geRHxBuAngamI+EFmHuvzgbHW6Rj60jgqqfPW9Ju8bgE2A9uq25uXFsjMXzu0HRGvA6YnOfgljbdSOm9Nx/y3AZdExB7gkuo+ETEdETc0rZwkaWX4NY6SNEH8GkdJUk+GvyQVyPBvo1IuMZTaqJDzr+lsHw3aoUsMD000np0tY+qB1AYFnX/2/NumpEsMpbYp6Pwz/NvG9SGk0Sno/HPYp21KusRQapuCzj/n+UvSBHGevySpJ8Nfkgpk+EtSgQx/SSqQ4S9JBTL8JalAhr8kFcjwl6QCGf6SVCDDX5IKZPhLUoEMf0kqkOEvSQUy/FuokG+Rk1aE509/XM+/ZQr6Fjlp4Dx/+mfPv2UK+hY5aeA8f/pn+LdMQd8iJw2c50//HPZpmYK+RU4aOM+f/vk1jpI0Qfwax5I53UEl8/3fl0bDPhGxBvgosAG4D3hlZn6vptx64AbgKUACL83M+5ocWz043UEl8/3ft6Y9/63AbGaeC8xW9+t8AHhPZp4HXAg82PC46sXpDiqZ7/++NQ3/TcCOansHcOnSAhFxPnByZu4EyMwfZOYPGx5XvTjdQSXz/d+3prN9zsrMfQCZuS8izqwp83Tg+xHxD8A5wO3A1sw82PDYquN0B5XM93/flg3/iLgdeGLNU287jmM8D3gm8G26nxG8Dnh/zbG2AFsA1q9f3+ev11E6Hd/0Kpfv/74sG/6ZeXGv5yLigYhYV/X611E/lr8X+HJm3lvt8wngOdSEf2ZuB7ZDd6pnf02QJB2vpmP+twCbq+3NwM01Ze4AVkfEGdX9i4C7Gh5XktRA0/DfBlwSEXuAS6r7RMR0RNwAUI3t/x4wGxG7gQD+quFxJUkNNPrANzMPABtrHl8Arj7s/k7gGU2OJUkaHK/wlaQCGf6SVCDDX5IKZPhLUoEMf0kqkOE/gVzRVtJy/CavCeOKtpL6Yc9/wriiraR+GP4TxhVtJfXDYZ8J44q2Gnfz875/h8Hwn0CuaKtx5WdWw+Owj6TW8DOr4TH8JbWGn1kNj8M+OpqDrhqRTgdmr9/N3E0HmLl8LZ3OBaOu0sQy/HUkB101SvPzdN68kc7iInxhCi7w/bdSHPbRkRx01Sj5/hsaw19HctBVo+T7b2gc9tGRvFBAo+T7b2giM0ddh1rT09O5sLAw6mpI0liJiF2ZOb1cOYd9JKlAhr+O4pLQ0uRzzF9HcKanVAZ7/jqCM+2kMhj+OoIz7aQyOOyjIzjTTiqD4a+juCS0NPkc9pGkAhn+klSgRuEfEWsiYmdE7KluV/co9+6IuDMivhYRfxYR0eS4kqRmmvb8twKzmXkuMFvdP0JE/DLwXOAZwM8DvwS8oOFxJUkNNA3/TcCOansHcGlNmQQeB0wBpwKnAA80PK6klvIK8fHQdLbPWZm5DyAz90XEmUsLZOZ8RPwzsA8I4L2Z+bWGx5XUQl4hPj6W7flHxO0R8dWan039HCAingacB5wNPBm4KCKe36PslohYiIiF/fv3H0871CZ2/Yo1NweLj2T3CvFH0ivEW2zZnn9mXtzruYh4ICLWVb3+dcCDNcUuA76YmT+o9vkn4DnA52uOtR3YDt0lnftrglrFrl/RZtbuZuqxp7LIKUw99iNm1t4D+D28bdR0zP8WYHO1vRm4uabMt4EXRMTJEXEK3Q97HfaZVHNzzD/yLK47+BbmH3mWiwMVpnPgVmZPeiHX8g5mT3ohnQO3jrpK6qHpmP824MaIuIpuyF8BEBHTwDWZeTXwMeAiYDfdD38/lZn/2PC4aqn5tS9j42O/zSJTTD22yOzae7DfX5CZGTqnXktn8YvV4lDvGXWN1EOj8M/MA8DGmscXgKur7YPA65scR+Nj7sAFLJ6UHHwsWDxpFXMHLjD8S+LiUGPDtX00UDMzMHVqVEP+4aqgJXJxqLFg+Gug7PhJ48Hw18DZ8ZPaz4XdJKlAhr8kFcjwl6QCGf6SVCDDX5IKZPhLUoEMf0kqkOGv9nFJ6JGa376b6140x/z23aOuilaQF3mpXVwSeqTmt+9m4+ufyiLnMfWZRWbZTWeLSzJPInv+aheXhB6puZsOsMgUBzmZRU5h7qYDo66SVog9f7WKS0KP1szla5n6zCKLJFP8iJnL1466Slohhr9axSWhR6uz5QJm2c3cTQeYuXytQz4TzPBXq7gk9Oh1tlxAZ8uoa6GVZvirVVwSWhoOw1+t45LQ0spzto8kFcjwl6QCGf6SVCDDX5On9OUhSm+/+uIHvpospS8PUXr71Td7/posc3Pd4Dt4sHtb2vIQLo+hPtnz12SZmen2eA/1fAu7SszlMdQvw1+TpdNh/vov/Xh5gk5ZyxO4PIb6ZfhroszPw8Y3X9Dt+H8BZi8oa8jb5THUL8NfE6VuyL+k8Hd5DPXL8NdEGciQ//z8aNOz4fFdHkP9aBT+EXEF8IfAecCFmbnQo9yLgT8FVgE3ZOa2JseVemnc8x31VMlRH1/FaDrV86vArwKf71UgIlYB7wNeApwPvDoizm94XKmnTgfe+tYTzMxBTBVtcpFV6VNVNTSNev6Z+TWAiDhWsQuBuzPz3qrsR4BNwF1Nji2tiKbjRk177oVPVdXwDGPM/8nAdw67vxd4dl3BiNgCbAFYv379ytdMWqrpuFHTT5wLn6qq4Vk2/CPiduCJNU+9LTNv7uMYdX8WZF3BzNwObAeYnp6uLSOttHk6zNFhBo5/jnzDnnvpU1U1PMuGf2Ze3PAYe4GnHHb/bOD+hr9TWhGNP29t+JdD6VNVNTzDGPa5Azg3Is4BvgtcCbxmCMeVjttAwrfBXEuH/DUsjWb7RMRlEbGX7l/Hn4yIT1ePPykibgPIzEeBNwGfBr4G3JiZdzartrQyDoXvqlUnHr5NJvsc+sPh2mud5amVFZntHFqfnp7OhYXaywakFdXkGiun6WvUImJXZk4vV84rfKUlmlwh65i9xoXr+UsDNIhhI2kY7PlLA+TCahoXhr80YC6spnHgsI8kFcjwl6QCGf6SVCDDX5IKZPhLUoEMf0kqUGuXd4iI/cC3Rl2PGqcDD426EgNiW9rJtrTTuLTlpzLzjOUKtTb82yoiFvpZN2Mc2JZ2si3tNEltAYd9JKlIhr8kFcjwP37bR12BAbIt7WRb2mmS2uKYvySVyJ6/JBXI8D9MRKyKiC9HxK3V/S9ExFeqn/sj4hM99jt4WLlbhlvrejVt2RgR/17V8V8i4mk99ntrRNwdEV+PiBcNt9b1TqQtEbEhIv73sNflL4Zf86PVtOWiqi1fjYgdEVG70m5EbI6IPdXP5uHWul6DtrTxfLkvInZXdVqoHlsTETurf/OdEbG6x76te236kpn+VD/A7wIfBm6tee4m4LU99vvBqOu+XFuAbwDnVdtvAP62Zp/zgf8ATgXOAe4BVo1pWzYAXx113Y/VFrqdr+8AT6+eeydwVc0+a4B7q9vV1fbqcWxL9Vwbz5f7gNOXPPZuYGu1vRV417i8Nv382POvRMTZwK8AN9Q893jgIqC25982PdqSwGnV9hOA+2t23QR8JDMfycxvAncDF65kXZfToC2tU9OWtcAjmfmN6v5O4PKaXV8E7MzMhzPze1W5F690fY+lQVvGySZgR7W9A7i0pkzrXpt+Gf4/dj3w+8BjNc9dBsxm5n/32PdxEbEQEV+MiLo3yLDVteVq4LaI2Av8BrCtZr8n0+29HbK3emyUTrQtAOdUwxKfi4jnrXA9+7G0LQ8Bp0TEoQuHXgE8pWa/cXhd+m0LtO98gW6H4jMRsSsitlSPnZWZ+wCq2zNr9mvja9MXwx+IiJcBD2bmrh5FXg383TF+xfrsXvn3GuD6iHjqoOvYr2O05XeAl2bm2cDfAH9ct3vNYyObDtawLfvovi7PpBqeiIjTasoNRV1bsjtucCXwJxHxb8D/AI/W7V7zWKtel+NoC7TofDnMczPzWcBLgDdGxPP73K9Vr83x8Gscu54LvDwiXgo8DjgtIj6Umb8eEWvpDn1c1mvnzLy/ur03IuaAZ9IdLx+FurZ8EvjZzPxSVeajwKdq9t3Lkb21sxntkMoJtyUzHwEeqbZ3RcQ9wNOBhaHU/Gg932PA8wAi4oVVHZfaC8wcdv9sYG5Fa3tsTdrStvNlaZ0ejIiP0z3nH4iIdZm5LyLWAQ/W7Nq216Z/o/7QoW0/dF/IWw+7fw2w4xjlVwOnVtunA3uA80fdjsPbQvc/+Yf48YdxVwE31ZT/OY78wPdeWvCB7wm25YxDdQd+GvgusGbU7Vj6HgPOrG5PBWaBi2rKrwG+Wb3XVlfb49qW1p0vwE8Ajz9s+1/pjtu/hyM/8H33OL02y/3Y81/elSwZU67GNa/JzKuB84C/jIjH6A6jbcvMu4Zfzd4y89GI+C3gpqqe3wN+EyAiXg5MZ+Y7MvPOiLgRuIvun+xvzMyDI6t4jX7bAjwfeGdEPAocpPt6PTyqeh/DW6phlJOAP8/Mz8KR77HMfDgirgXuqPZ557i2hXaeL2cBH48I6HYuPpyZn4qIO4AbI+Iq4NvAFTC2r81RvMJXkgrkB76SVCDDX5IKZPhLUoEMf0kqkOEvSQUy/CWpQIa/JBXI8JekAv0/ztWjazBD4SkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loss:  0.00963638350367546\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAD8CAYAAACW/ATfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAE2lJREFUeJzt3X2MXFd5x/Hvkw0LglJAsRE0tnFoHYk0qUQ0hG4DdFtDa6IqbgtFCarUl6guVUMpJVSpqNIo+cO8CKFWjWgNRGmpiqHQl1XlKlDTBUQ31GNK4tiQYtzQbFwREygSQmSx/fSPmUWTZdZ752Xn7t3z/UjW7p1778w5c2Z/PnPuufdGZiJJ2twuqrsAkqT1Z9hLUgEMe0kqgGEvSQUw7CWpAIa9JBXAsJekAhj2klQAw16SCnBxXS+8ZcuW3LlzZ10vL0mNdPTo0a9n5tZB96st7Hfu3Em73a7r5SWpkSLiq8Ps5zCOJBXAsJekAhj2klQAw16SCmDYS1IBDHtJKoBhr0ZZWID9+zs/JVVX2zx7aVALC7B7NywtwfQ0HD4MMzN1l0pqBnv2aoz5+U7QnzvX+Tk/X3eJpOYw7NUYs7OdHv3UVOfn7GzdJZKaw2EcNcbMTGfoZn6+E/QO4UjVGfaaqIWF0cJ6ZsaQl4Zh2GtiPMAq1WfNMfuIuDsiHouIB1dZHxHxZxFxMiIeiIirx19MbQYeYJXqU+UA7T3AngusfzWwq/tvH/De0YulzWhTHGB1or8aas1hnMz8dETsvMAme4G/zswE7ouIZ0fE8zPzf8dURm0SjT/A6jiUGmwcY/aXAo/0LC92H/uBsI+IfXR6/+zYsWMML62mqf0A6yhHiPuNQw1TmVGPUktDGEfYR5/Hst+GmXkAOADQarX6biOtm1F75svjUMv7DzMO5bcD1WQcJ1UtAtt7lrcBp8fwvNJ4jXqEeHkc6s47hw9pj1KrJuPo2c8BN0fEQeClwLccr9eGNDvLwtTLmD9/LbNTn2VmmJ75qONQ4/h2IA1hzbCPiA8Bs8CWiFgE/gR4CkBm/gVwCLgOOAl8B/iN9SqsNIoFZtgdh1kimI7kMFNMfACl8Uep1VRVZuPcuMb6BH53bCWS1sn8PCydneJcwtLZ4Y+vjqz2o9QqkRdCUzE2xTx/aUheLkHNMsK0xU0zguLUTQ3BsFdzjGHaYuNHUJy6qSE5jKPmcNqi74GGZtirORx09z3Q0BzGUXNsmkH3EfgeaEjRmTk5ea1WK9vtdi2vLUlNFRFHM7M16H4O40hSAQx7SSqAYS9JBTDsJakAhr0my9v61c82KJJTLzU5nv1ZP9ugWPbsNZCROoWe/Vk/26BY9uxV2cidQm/cUT/boFiGvSob+X7bm+Dsz8ZfcHITtIGGY9irsrF0Cht82clNM9zd4DbQ8Ax7VVZ6p3DkbzZSjQx7DaTkTqHD3Woyw16qqPRvNmo2w14aQMnfbNRszrOXCuMJtGWyZy8VZNPMKNLA7NlLBfEE2nIZ9lJBvIVtuRzGkRpmlLN4nVFUrkphHxF7gD8FpoD3Z+bbV6zfAfwV8OzuNrdm5qExl1UbQeOvF1C/Ud7CcYy5jzyjyM9AI60Z9hExBdwFvApYBI5ExFxmnujZ7I+Bj2TmeyPiCuAQsHMdyqs6eXRvZKO+hbWfxetnoLGqjNlfA5zMzFOZuQQcBPau2CaBH+7+/izg9PiKqA3Do3sjG/UtrH3M3c9AY1UZxrkUeKRneRF46Yptbgc+HhFvBJ4BvLLfE0XEPmAfwI4dOwYtq+rm9QJGNupbWPuY++wsC1MvY/78tcxOfZYZPwONUSXso89juWL5RuCezHx3RMwAH4yIKzPz/JN2yjwAHABotVorn0MbXe1J03zjeAvrPIt3gRl2x2GWCKYjOcwUfgqaoUrYLwLbe5a38YPDNDcBewAycyEingZsAR4bRyG1gXi9gJE1+S2cn4els1OcS1g665U/m6TKmP0RYFdEXBYR08ANwNyKbf4H2A0QES8CngacGWdBJdWv9mMGGtqaPfvMPBsRNwP30plWeXdmHo+IO4B2Zs4BbwHeFxFvpjPE8+uZ6TCNtMk4ktdcUVcmt1qtbLfbtby2JDVVRBzNzNag+3m5BEkqgGEvSQUw7CWpAIa9JBXAsJekAhj2klQAw16SCmDYS1IBDPvCLBw4xv6fn2fhwLG6i6JSLSzA/v2dn5oYb0tYkIUDx9j92z/KEi9i+uNLHOYYM/uuqrtYKok3P6mNPfuCzH/scZaY5hwXs8RTmP/Y43UXSaXx5ie1MewLMvuaS5hmiSm+xzTfY/Y1l9RdJJXGy2bWxmGcgszsu4rDHGP+Y48z+5pLHMLR5HnZzNp41UtJahCveilJWpVhL0kFMOwlqQCGvSQVwLCXpAIY9pJUAMNe0kR5aZx6eFKVpInx0jj1sWcvaWK8NE59DHtJE+OlcerjMI6kifHSOPUx7CVN1MyMIV+HSsM4EbEnIh6KiJMRcesq27wuIk5ExPGI+NvxFlOSNIo1e/YRMQXcBbwKWASORMRcZp7o2WYX8EfAtZn5zYh47noVWJI0uCo9+2uAk5l5KjOXgIPA3hXb/BZwV2Z+EyAzHxtvMSVJo6gS9pcCj/QsL3Yf63U5cHlEfDYi7ouIPf2eKCL2RUQ7ItpnzpwZrsSl84wUlc6/gaFUOUAbfR5beceTi4FdwCywDfhMRFyZmf/3pJ0yDwAHoHPzkoFLWzrPSFHp/BsYWpWe/SKwvWd5G3C6zzb/lJnfy8z/Bh6iE/4ap/l5Fp64mv3n3srCE1d7RorK41lZQ6vSsz8C7IqIy4BHgRuA16/Y5h+BG4F7ImILnWGdU+MsqGDhkl9g9/k3scQ00+eXOHzJV7BPo6Isn5W13LP3rKzK1gz7zDwbETcD9wJTwN2ZeTwi7gDamTnXXfdzEXECOAe8NTMfX8+Cl2j+8atYuig5dz5YumiK+cevMuxVFs/KGpo3HG8QhyslDXvDcc+gbRA7NZKGZdg3jKeaSxqGV72UpAIY9pJUAMNekgpg2EtSAQz7CfOyHpLq4GycCXKevKS62LOfIC/rIakuhv0EebNlSXVxGGeCPANWUl0M+wnzDFhJdXAYZ0DOppHURPbsB+BsGklNZc9+AM6mkdRUhv0AnE0jqakcxhmAs2kkNZVhPyBn00hqIodxJs3pPFK9Cv0btGc/SU7nkepV8N+gPftJcjqPVK+C/wYN+0lyOo9Ur4L/Bh3GmSSn80j1KvhvMDKzlhdutVrZbrdreW1JaqqIOJqZrUH3cxhHkgpg2EtSASqFfUTsiYiHIuJkRNx6ge1eGxEZEQN/xZiUQqfYSircmgdoI2IKuAt4FbAIHImIucw8sWK7ZwK/B3xuPQo6DgVPsZVUuCo9+2uAk5l5KjOXgIPA3j7b3Qm8E/juGMs3VgVPsZU2Db+dD6fK1MtLgUd6lheBl/ZuEBEvBrZn5j9HxC2rPVFE7AP2AezYsWPw0o5oeYrtcs++oCm20qbgt/PhVenZR5/Hvj9fMyIuAt4DvGWtJ8rMA5nZyszW1q1bq5dyTJan2N55px8SqYn8dj68Kj37RWB7z/I24HTP8jOBK4H5iAB4HjAXEddn5oabSO9VK6Xm8tv58KqE/RFgV0RcBjwK3AC8fnllZn4L2LK8HBHzwC0bMeglNVvBJ8CObM2wz8yzEXEzcC8wBdydmccj4g6gnZlz611ISVrmt/PhVLo2TmYeAg6teOy2VbadHb1YkqRx8gxaSSqAYS9JBTDsJakAhr0kFcCwl6QCGPaSVADDXpIKYNhLUgEM+0F5fVWpbA3NgEpn0KrL66tKZWtwBtizH4TXV5XK1uAMMOwHsXx91akpr68qlajBGeAwziC8vqpUtgZnQGTm2lutg1arle22l7yXpEFExNHMbA26n8M4klQAw16SCmDYS1IBDHtJKkDjwr6hJ69JUq0aNfWywSevSVKtGtWzb/DJa5JUq0aFfYNPXpOkWjVqGKfBJ69JUq0aFfbQCXhDXpIG06hhHEnScAx7SSpApbCPiD0R8VBEnIyIW/us/4OIOBERD0TE4Yh4wfiLKkka1pphHxFTwF3Aq4ErgBsj4ooVm/0n0MrMnwA+Crxz3AWVJA2vSs/+GuBkZp7KzCXgILC3d4PM/LfM/E538T5g23iLKUkaRZWwvxR4pGd5sfvYam4C/qXfiojYFxHtiGifOXOmeiklSSOpEvbR57G+dzyJiF8FWsC7+q3PzAOZ2crM1tatW6uXUpI0kirz7BeB7T3L24DTKzeKiFcCbwN+OjOfGE/xJEnjUKVnfwTYFRGXRcQ0cAMw17tBRLwY+Evg+sx8bPzFlCSNYs2wz8yzwM3AvcAXgY9k5vGIuCMiru9u9i7gh4C/i4gvRMTcKk8nSapBpcslZOYh4NCKx27r+f2VYy6XJGmMyjuD1rufSKpTTRnUuAuhjcS7n0iqU40ZVFbP3rufSKpTjRlUVth79xNJdaoxg8oaxvHuJ1LxFhZqjIAaMygy+54Mu+5arVa22+1aXltSmTbDYbuIOJqZrUH3K2sYR1LRSj5sZ9hLKkbJh+3KGrOXVLSSD9sZ9pKKMjNTVsgvcxhHkgpg2EtSAQx7SSqAYS9JBTDsJakAhr0kFcCwl6QCGPaSVADDXpIKYNhLUgEMe0kqgGEvSQUw7CWpAIa9JBXAsJekAhj2klSASmEfEXsi4qGIOBkRt/ZZ/9SI+HB3/eciYue4CypJGt6aYR8RU8BdwKuBK4AbI+KKFZvdBHwzM38MeA/wjnEX9PsWFmD//s5PSVIlVW5LeA1wMjNPAUTEQWAvcKJnm73A7d3fPwr8eUREZuYYy9oJ+N27O7eFn57u3EyyxPuLSdKAqgzjXAo80rO82H2s7zaZeRb4FnDJyieKiH0R0Y6I9pkzZwYv7fx8J+jPnev8nJ8f/DkkqUBVwj76PLayx15lGzLzQGa2MrO1devWKuV7stnZTo9+aqrzc3Z28OeQpAJVGcZZBLb3LG8DTq+yzWJEXAw8C/jGWErYa2amM3QzP98JeodwJKmSKmF/BNgVEZcBjwI3AK9fsc0c8GvAAvBa4JNjH69fNjNjyEvSgNYM+8w8GxE3A/cCU8DdmXk8Iu4A2pk5B3wA+GBEnKTTo79hPQstSRpMlZ49mXkIOLTisdt6fv8u8CvjLZokbTwLC80cSa4U9pKkZs/+9nIJklRRk2d/G/aSVFGTZ387jCNJFTV59rdhL0kDaOrsb4dxJKkAhr0kFcCwl6QJqusq7Y7ZS9KE1DlP3569JE1InfP0DXtJmpA65+k7jCNJE1LnPH3DXpImqK55+g7jSFIBDHtJKoBhL0kFMOwlqQCGvSQVwLCXpAJEZtbzwhFngK+u88tsAb6+zq8xKdZlY7IuG9NmrssLMnProE9SW9hPQkS0M7NVdznGwbpsTNZlY7IuP8hhHEkqgGEvSQXY7GF/oO4CjJF12Zisy8ZkXVbY1GP2kqSOzd6zlyTR0LCPiDdFxIMRcTwifn/FulsiIiNiyyr7nouIL3T/zU2mxKvrV5eIuD0iHu0p53Wr7LsnIh6KiJMRcetkS963PKPU5eGIONbdpj3ZkvctT9/PWES8sfueH4+Id66y74Zvl+7jVeqy4dslIj7c8/l6OCK+sMq+G75dBqjL4O2SmY36B1wJPAg8nc4lmv8V2NVdtx24l878/S2r7P/tuuuwVl2A24Fb1th3CvgK8EJgGrgfuKKJdenu//BqbbaB6vIz3d+f2t3uuQ1ulzXr0pR2WbHNu4HbmtouVeoybLs0sWf/IuC+zPxOZp4FPgX8Unfde4A/BJpyIOJCdVnLNcDJzDyVmUvAQWDvOpWzilHqstGsVpffAd6emU8AZOZjffZtSrtUqctGc8HPWEQE8DrgQ332bUq7AGvWZShNDPsHgVdExCUR8XTgOmB7RFwPPJqZ96+x/9Mioh0R90XEL657aS+sb126626OiAci4u6IeE6ffS8FHulZXuw+VpdR6gKd/6A/HhFHI2LfJAp8AavV5XLg5RHxuYj4VES8pM++TWmXKnWBZrTLspcDX8vML/fZtyntsuxCdYEh2qVxd6rKzC9GxDuATwDfpvN17CzwNuDnKjzFjsw8HREvBD4ZEccy8yvrV+LVXaAu7wXupNOgd9L5OvebK3aPfk+5fqW9sBHrAnBtt12eC3wiIr6UmZ+eTOmf7AJ1uRh4DvCTwEuAj0TEC7P7vbqrKe1SpS7QjHZZdiOr94Sb0i7LLlQXGKJdmtizJzM/kJlXZ+YrgG/QGb+6DLg/Ih4GtgGfj4jn9dn3dPfnKWAeePGEit1Xn7p8OTO/lpnnMvM88D46X0FXWuTJPYFtwOn1L/HqRqhLb7s8BvzDattNSr+60HnP/z47/gM4T+e6Jb0a0S5Uq0tT2oWIuBj4ZeDDq+zalHapUpfh2mVcBxwm+Y/uwSRgB/Al4Dkr1j9Mn4MXdHoyywektnTf3NoO0qxWF+D5PevfDBzss9/FwCk6/8ktH3D68YbW5RnAM3t+/3dgzwasyxuAO7qPX05nWCAa2i5V6tKIduku7wE+dYH9GtEuFesyVLvUVtER36TPACe6Dba7z/rvhz3QAt7f/f2ngGPd/Y4BN23EugAf7JbvAWBuOTCBHwEO9ex7HfBfdGYZvK2pdaEzQ+L+7r/jG7gu08Df0Blv/Tzwsw1ulzXr0pR26T5+D/CGFds2rl2q1GXYdvEMWkkqQCPH7CVJgzHsJakAhr0kFcCwl6QCGPaSVADDXpIKYNhLUgEMe0kqwP8DZQQSRz9g4CcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loss:  0.004345584195107222\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAFAJJREFUeJzt3X2QZFV5x/Hvw+iQSizC2wIrsC7RTSkRYrAljvFl42JFKCsLCRjUkrUC2RjLVPIHiVhEy4plQZJK+Yap1AaVRStBfEFIXGNgdDRVjoTeBAV8AwziwhaLKyZSVlhYnvzRd3BoemZ65vZM9+3z/VRN9dvpe+7Z2/3b0+e+nMhMJEllOWTYKyBJWnuGvyQVyPCXpAIZ/pJUIMNfkgpk+EtSgQx/SSqQ4S9JBTL8JalATxv2Cizk6KOPzo0bNw57NSSpUXbv3v3DzFy3VLmRDf+NGzfSbreHvRqS1CgR8f1+yjnsI0kFMvwlqUCGvyQVyPCXpAIZ/pJUoIGEf0S8OiK+ExF3RcQlPV4/NCI+Ub1+c0RsHES9kqSVqR3+ETEBfAg4EzgZeF1EnNxV7ELgocx8DvBe4K/q1ruo2Vm47LLOrSTpKQZxnP/pwF2Z+T2AiLgG2Ap8c16ZrcC7qvufAq6IiMjVmENydha2bIEDB2ByEqanYWpq4NVIUpMNYtjneOAH8x7vqZ7rWSYzHwP+Bziqe0ERsT0i2hHRfvDBB1e2NjMzneA/eLBzOzOzsuVI0hgbRPhHj+e6e/T9lCEzd2RmKzNb69YteXZyb5s3d3r8ExOd282bV7YcSRpjgxj22QOcOO/xCcD9C5TZExFPA34R+NEA6n6qqanOUM/MTCf4HfKRpKcYRPjfAmyKiJOA+4Dzgdd3lbkB2AbMAucCX1yV8f45U1OGviQtonb4Z+ZjEfFW4AvABPCRzLwjIv4SaGfmDcCHgY9FxF10evzn161XkrRyA7mqZ2buAnZ1PffOeff/DzhvEHVJkurzDF9JKpDhL0kjZK3OUR3ZyVwkqTRreY6qPX9JGhFreY6q4S9JI2Itz1F12EeSRsRanqNq+EvSCFmrc1Qd9pGkAhn+klQgw1+SCmT4S1KBxjL8ncVRkhY3dkf7OIujJC1t7Hr+zuIoSUsbu/B3FkdJWtrYDfs4i6MkLW3swh+cxVGSljJ2wz6SpKUZ/pJUIMNfkgpk+EtSgQx/SSqQ4d+L14eQNCxrlD9jeahnLV4fQtKwrGH+2PPv5vUhJA3LGuaP4d/N60NIGpY1zB+Hfbp5fQhJw7KG+ROZuWoLr6PVamW73R72akhSo0TE7sxsLVXOYR9JKlCt8I+IIyPixoi4s7o9okeZF0TEbETcERHfiIjfq1OnJKm+uj3/S4DpzNwETFePu/0UuCAzfwV4NfC+iDi8Zr2SpBrqhv9WYGd1fydwdneBzPxuZt5Z3b8f2Aesq1mvJKmGuuF/bGbuBahuj1mscEScDkwCdy/w+vaIaEdE+8EHH6y5apKkhSx5qGdE3AQc1+OlS5dTUUSsBz4GbMvMx3uVycwdwA7oHO2znOVLkvq3ZPhn5hkLvRYRD0TE+szcW4X7vgXKHQZ8DviLzPzaitdWkjQQdYd9bgC2Vfe3Add3F4iISeA64OrM/GTN+iRppDXlupB1z/C9HLg2Ii4E7gXOA4iIFvDmzLwIeC3wcuCoiHhT9b43ZeatNeuWpJHSpOtC1gr/zNwPbOnxfBu4qLr/ceDjdeqRpCbodV22UQ1/z/CVpAFp0nUhvbCbJA1Ik64LafhL0gBNTY126M9x2EeSCmT4S1KBDH9JKpDh30NTTtKQpJVyh2+XJp2kIUkrZc+/S6+TNCRp3Bj+XZp0koYkrZTDPl2adJKGJK2U4d9DU07SkKSVcthHkgpk+K8GjxWVytWQ77/DPoPmsaJSuRr0/bfnP2geKyqVq0Hff8N/0DxWVCpXg77/DvsMmseKSuVq0Pc/MnPY69BTq9XKdrs97NWQpEaJiN2Z2VqqnMM+klQgw1+SCmT4S1KBDH9JKpDhL0kFMvwlqUCGvyQVyPCXpAIZ/pJUIMNfkgpUK/wj4siIuDEi7qxuj1ik7GERcV9EXFGnTklSfXV7/pcA05m5CZiuHi/k3cCXa9YnSRqAuuG/FdhZ3d8JnN2rUES8EDgW+Lea9UnSqmrIRFy11b2k87GZuRcgM/dGxDHdBSLiEOBvgTcCWxZbWERsB7YDbNiwoeaqSdLyNGgirtqW7PlHxE0RcXuPv6191vEWYFdm/mCpgpm5IzNbmdlat25dn4uXpMFo0ERctS3Z88/MMxZ6LSIeiIj1Va9/PbCvR7Ep4GUR8RbgGcBkRDycmYvtH2i02dlGzOUgqcvcRFxzPf8RnoirtrrDPjcA24DLq9vruwtk5hvm7kfEm4DWuAd/KT8bpXHToIm4aqsb/pcD10bEhcC9wHkAEdEC3pyZF9VcfuP0+tk4zh8gadxMTZXxna0V/pm5nx47cTOzDTwl+DPzKuCqOnWOupJ+NkpqLidwH7CSfjZKai7DfxWU8rNRUnN5bR9JKpDhP4pKOcVQGkWFfP8c9hk1HisqDU9B3z97/qOmpFMMpVFT0PfP8B81c8eKTkx4rKi01gr6/jnsM2o8VlQanoK+f5GZw16HnlqtVrbb7WGvhiQ1SkTszszWUuUc9pGkAhn+klQgw1+SCmT4S1KBDH9JKpDhL0kFMvwlqUCGvyQVyPCXpAIZ/pJUIMNfkgpk+EtSgQx/SSqQ4S9JBTL8R1AhU4hKq8LvT3+czGXEFDSFqDRwfn/6Z89/xBQ0hag0cH5/+mf4j5iCphCVBs7vT/8c9hkxBU0hKg2c35/+Gf4jaGqq5od2dtZPv4o1xSxTzACbAT//C6kV/hFxJPAJYCNwD/DazHyoR7kNwJXAiUACZ2XmPXXq1gLc46WS+fnvW90x/0uA6czcBExXj3u5GvibzHwecDqwr2a9Woh7vFQyP/99qxv+W4Gd1f2dwNndBSLiZOBpmXkjQGY+nJk/rVmvFuIeL5XMz3/f6o75H5uZewEyc29EHNOjzC8DP46IzwAnATcBl2Tmwe6CEbEd2A6wYcOGmqtWKPd4qWR+/vsWmbl4gYibgON6vHQpsDMzD59X9qHMPKLr/ecCHwZ+DbiXzj6CXZn54cXqbbVa2W63+2qEJKkjInZnZmupckv2/DPzjEUqeSAi1le9/vX0HsvfA/xXZn6ves9ngRfT+Q9BkjQEdcf8bwC2Vfe3Adf3KHMLcERErKsevxL4Zs16JUk11A3/y4FXRcSdwKuqx0REKyKuBKjG9i8GpiPiNiCAf6hZrySphlo7fDNzP7Clx/Nt4KJ5j28ETq1TlyRpcLy2jyQVyPCXpAIZ/pJUIMNfkgpk+EtSgQx/SSqQ4S9JBTL8JalAhr8kFcjwl6QCGf5jaHYWLruscytJvTiB+5hxClNJ/bDnP2acwlRN5y/XtWHPf8zMTWE61/Nf0RSms7NOg6ehmJ2FLb95kAMHgsnJZPpLE34EV4nhP2ZqT2HquJGGaObq73PgkeM5yAQHHnmUmav3MDX1rGGv1lhy2GcMTU3B29++wsx23EhDtJkvM8kBJniUSR5lM18e9iqNLcNfTzY3bjQxUWPcSFqZqQs2MT15Fu+OdzE9eRZTF2wa9iqNrcjMYa9DT61WK9vt9rBXo0yO+WuY/PzVEhG7M7O1ZDnDX5LGR7/h77CPJBXI8JekAhn+klQgw1+SCmT4S1KBDH9JKpDhL0kFMvwlqUCGvyQVyPCXpALVCv+IODIiboyIO6vbIxYo99cRcUdEfCsiPhARUadeSVI9dXv+lwDTmbkJmK4eP0lEvAT4DeBU4PnAi4BX1KxXklRD3fDfCuys7u8Ezu5RJoGfAyaBQ4GnAw/UrFeryGn0pPFXdyavYzNzL0Bm7o2IY7oLZOZsRHwJ2AsEcEVmfqtmvVolA5nIy0vyls3t3whLhn9E3AQc1+OlS/upICKeAzwPOKF66saIeHlmfqVH2e3AdoANGzb0s3gNWK+JvJb1/XUayLK5/RtjyWGfzDwjM5/f4+964IGIWA9Q3e7rsYhzgK9l5sOZ+TDweeDFC9S1IzNbmdlat27dylulFas9kZfTQJZtZobZR07jsoN/xuwjp7n9R1jdYZ8bgG3A5dXt9T3K3Av8QURcRmfY5xXA+2rWq1VSewL4uf895np+TgNZlNmjXsOWx/+EA0wy+fgBpo+6G/v9o6lu+F8OXBsRF9IJ+fMAIqIFvDkzLwI+BbwSuI3Ozt9/zcx/rlmvVtHUVI1f6rX/91CTzew/hQOHJAcfDw4cMsHM/lMM/xFVK/wzcz+wpcfzbeCi6v5B4A/r1KOGqfW/h5ps82aYPDSqH37hD78RVrfnL0lP8Idfcxj+kgbKH37N4LV9JKlAhr8kFcjwl6QCGf6SVCDDX5IKZPhLUoEMf0kqkOGvgXM+AGn0eZKXBsr5AMaA//5FMPw1UM4H0HD++xfDYR8NlPMBNJz//sWw56+Bcj6AhvPfvxiRmcNeh55arVa22+1hr4aGwTHnoZrdcRszn97P5t89iqntpwx7dbRMEbE7M1tLlbPnr9HjZSGHZnYWtvzpKZ2O/7/D9CluinHlmL+kJzjkXw7DX9ITau+wV2M47CPpCc7EVQ7DX9KTuMulDA77SFKBDH+Nn9IvLlR6+9UXh300cmod5l/65QlKb7/6Zs9fI2Uuu97xjs7tsjuvpR+rWHr71TfDXyOldnaVfqxi6e1X3xz20UipfWmZ0o9VLL396pvX9tHI8dI+0sp5bR81lseZ1+N/nuqH4S91G3Z61qjfg33Ur1rhHxHnAe8Cngecnpk9x2ki4tXA+4EJ4MrMvLxOvdKqGXZ61qy/9kxqKkbdo31uB34H+MpCBSJiAvgQcCZwMvC6iDi5Zr3S6hj2oZI16/dgH/WrVs8/M78FEBGLFTsduCszv1eVvQbYCnyzTt3SqhjETFZ1ho1q1u/BPurXWoz5Hw/8YN7jPcCvr0G9KlStIfu66Vl32GgA6e0Oc/VjyfCPiJuA43q8dGlmXt9HHb1+FvQ8vjQitgPbATZs2NDHoqUnG8iQfZ30HMSgu+mtNbBk+GfmGTXr2AOcOO/xCcD9C9S1A9gBneP8a9arAg19h6cToKsh1mLY5xZgU0ScBNwHnA+8fg3qVYGGnr0DGLYZ9pGmKkPdQz3PAT4IrAM+FxG3ZuZvRcQz6RzSeVZmPhYRbwW+QOdQz49k5h2111zqYSR2eNYYthn2kaYqR92jfa4Druvx/P3AWfMe7wJ21alL6lfdIfNh9ryHPmylYniGrzTPIHreQzzSU+qb4S/NU7fnPQJHekp9Mfyleer2vD3SU01h+Evz1O15O2yjpjD8pS51et4O26gpDH9pwBy2URM4h68kFcjwl6QCGf6SVCDDX5IKZPhLUoEMf0kqUGSO5mXzI+JB4PvLfNvRwA9XYXWGwbaMJtsymmzLzzwrM9ctVWhkw38lIqKdma1hr8cg2JbRZFtGk21ZPod9JKlAhr8kFWjcwn/HsFdggGzLaLIto8m2LNNYjflLkvozbj1/SVIfRjb8I+IjEbEvIm7v8drFEZERcXT1+LkRMRsRj0TExYss86qI+O+IuLX6e8FqtmFevctpyxsi4hvV31cj4lcXWOZJEXFzRNwZEZ+IiMnVbkdV72q0pQnbZWvVjlsjoh0RL11gmS+MiNsi4q6I+EBExGq3o6p3NdoyExHfmbddjlntdlT19t2Wec+/KCIORsS5Cyxz5LfLvOeXastgtktmjuQf8HLgNOD2rudPBL5A5xyAo6vnjgFeBLwHuHiRZV4FnDvibXkJcER1/0zg5gWWeS1wfnX/74E/anBbmrBdnsHPhklPBb69wDL/A5gCAvg8cGaD2zIDtEZ5u1TPTwBfBHYt9DlqwnZZRlsGsl1GtuefmV8BftTjpfcCfw7kvLL7MvMW4NE1Wr1lWWZbvpqZD1UPvwac0P2mqtfySuBT1VM7gbMHuc4LGXRbhmmZbXk4q28e8AvzX5sTEeuBwzJztip7NaO5XZZsyzAtpy2VPwY+DezrtbymbJfKom0ZpJEN/14i4reB+zLz6zUW857qJ+97I+LQQa3bcvXZlgvp9FK6HQX8ODMfqx7vAY4f8Cr2rWZb5oz8domIcyLi28DngN/v8fbj6WyLOSO7Xfpoy5yPVkML71iroZJeFmpLRBwPnEPn1+9CGrFd+mzLnNrbpTHhHxE/D1wKvLPGYt4OPJfOENGRwNsGsGrL1k9bIuI36QRmr3XstbGH0nsbQFugIdslM6/LzOfS6TW+u9cier1tcGvYvwG0BeANmXkK8LLq742rsa5LWaIt7wPelpkHF1tEj+dGcbv00xYY0HZpTPgDzwZOAr4eEffQGUL4z4g4rt8FZObe7HgE+Chw+qqs6dIWbUtEnApcCWzNzP093v9D4PCImJuG8wTg/lVf697qtqUx22VO9VP+2d076uj0KOcPbY3sdpmzSFvIzPuq258A/8hobpcWcE31/LnA30VE95BOU7ZLP20Z2HZpzBy+mXkbnR27AFT/QK3M7PsCSBGxPjP3Vj+Tzgaesgd+LSzWlojYAHwGeGNmfneB92dEfInOB+QaYBtw/aqveO91qdWW6j1N2C7PAe6u/u1PAyaB/V3v3xsRP4mIFwM3AxcAH1yzBjx5XWq1pepYHF6VfzrwGuCmNWvAPEt890+a9/xVwL9k5me73t+I7UIfbRnodqm7x3i1/oB/AvbS2Ym7B7iw6/V7+NnRC8dVZf4X+HF1/7DqtV3AM6v7XwRuoxMuHweeMYJtuRJ4CLi1+mvPKze/Lb9E5wiGu4BPAoc2uC1N2C5vA+6o2jELvHReuVvn3W9V7bgbuILqqJqmtYXOjuDdwDeqsu8HJkatLV3PX8W8I2Satl36acsgt4tn+EpSgZo05i9JGhDDX5IKZPhLUoEMf0kqkOEvSQUy/CWpQIa/JBXI8JekAv0/j5q2QtV3Y9QAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loss:  0.0010318030836060643\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAFANJREFUeJzt3X+M5Hd93/HnizPnSIXw6y4R+A6f215+WKCSZOt0lba+xqExqPK1DWlMS6GE1iqK04QmVX1NSpBRRQKNSCss0ktiCFSJ45CkuVZXuejiE221JrcnjMGGK1c7jRdb8UEoSYrCYufdP2YOD+vd2+/cd/Y7O9/v8yGtdmb2OzOfz35nXvvZ93y+n2+qCklSvzxr3g2QJM2e4S5JPWS4S1IPGe6S1EOGuyT1kOEuST1kuEtSDxnuktRDhrsk9dAV83riffv21aFDh+b19JK0kM6ePfu5qtq/3XZzC/dDhw6xuro6r6eXpIWU5P802c6yjCT1kOEuST1kuEtSDxnuktRDhrsk9dC24Z7kziRPJPnkFj9Pkn+f5HySB5J8++ybKUmaRpOR+/uBGy/x81cBh8dftwDvbd8saZdaWYF3vGP0XdrFtp3nXlUfSXLoEpscBT5Qo/P13Zfk+UleXFWPz6iN0u6wsgI33ADr67B3L5w6BcvL826VtKlZ1NyvAh6duL42vu0ZktySZDXJ6oULF2bw1FKHTp8eBftTT42+nz497xZJW5pFuGeT2zY963ZVHa+qpapa2r9/26NnpdlrU1Y5cmQ0Yt+zZ/T9yJHu2yA1NIvlB9aAgxPXDwCPzeBxpdlaWWHlyDFOf+W7OPLsYyyffsd0ZZXl5VEp5vTpUbBfTknG0o46MotwPwHcmuQu4DuBL1pv105ZWbn8bF35wGe4Yf0k6+xl7/o6pz7wIZanfZDl5XZhvFlpx3DXDtg23JP8KnAE2JdkDfgp4NkAVfXzwEng1cB54EvAG3eqsRq2toPe01zPOnt5iitYpzjN9Uwbq23+uABPl3YuduJySzvSNprMlnntNj8v4Idm1iJpC20HvUdefzV73/cU6+tPsXfvszjy+qunev6ZVFRmUdqRGpjbkr/StNoOepeX4dS9ey47V2dWUWlb2pEaMNzVqTZljeVlOPVzn+D0b3yeI9/3IpaXXz7187fJ1V1TUWldG9IQGO7qTOuyxsoKyz96A8vr6/Df98LLu51psisqKs62UUMuHKbOtD4GaBccRLS8DMeOzTFPd8HvQIvBcFdnWh8DNKuDiBaZvwM1ZFlGnWld1tgVdZH2WpXMe/I70M7LaCZj95aWlsoTZGtoLJmrrSRnq2ppu+0sy0gdsmSurhjuUocsmasr1tylDlkyV1cMd6ljHqCqLliW0VRcilxaDI7c1dhMZnp46Pz8uQ8GwXBXY60XznIe4Py5DwbDsowaaz3Tw3mA8+c+GAxH7mqs9UyPXbOs4oC5DwbDI1TVLeu9rbX+FboPFlrTI1QNd2mBWDKXyw9IPWTJXE0Z7tICcfkCNeUHqtICcfkCNWW4SwvG5QvUhGUZSeohw12Seshwl6QeMtwHxlUdpWHwA9UB8QAYaTgcuQ+IB8BIw9Eo3JPcmORckvNJbtvk5y9Ncm+SjyV5IMmrZ99UtTWTA2Cs68jXwELYtiyTZA9wB/BKYA04k+REVT00sdlPAndX1XuTXAucBA7tQHvVQusDYKzryNfAwmgycr8OOF9VD1fVOnAXcHTDNgV8/fjy84DHZtdEzdLyMhw7dpnvR+s68jWwMJqE+1XAoxPX18a3TXob8Loka4xG7T+82QMluSXJapLVCxcuXEZzNVcubNILraoqvgYWRpPZMtnkto3rBL8WeH9V/WySZeCDSV5WVX/2NXeqOg4ch9GSv5fTYM2RC5ssvNZVFV8DC6NJuK8BByeuH+CZZZc3ATcCVNVKkq8D9gFPzKKR2kVc2GShtT4PLvgaWBBNyjJngMNJrkmyF7gZOLFhm98HbgBI8q3A1wHWXaRdxqrKcGw7cq+qJ5PcCtwD7AHurKoHk9wOrFbVCeDHgF9I8hZGJZt/VPM6xZOkLVlVGQ5PsydJC8TT7EnSgBnuktRDhrsk9ZDhLkk9ZLgvGNdsktSE67kvENdsktSUI/cF4ppNkpoy3BeIRxdKasqyzALx6EJJTRnuC8Y1myQ1YVlmaJxuo3nzNdgJR+5D4nQbzZuvwc44ch8Sp9toBloNvH0NdsaR+5BcnG5zcdTkdBtNqfXA29dgZwz3IXG6jVpqfSYnX4OdMdyHxuk2amEmA29fg50w3CU15sB7cRjukqbiwHsxOFtGknrIcJekHjLcJamHDHdJ6iHDXZJ6yHCXpB4y3DvmgniSuuA89w65IJ6krjhy75AL4knqiuHeIc+BKqkrjcI9yY1JziU5n+S2Lbb5e0keSvJgkl+ZbTP74eK6HG9/uyUZSTtr25p7kj3AHcArgTXgTJITVfXQxDaHgWPAd1XVF5J8w041eNG5LoekLjQZuV8HnK+qh6tqHbgLOLphm38C3FFVXwCoqidm20xJ0jSahPtVwKMT19fGt036JuCbkvzPJPcluXGzB0pyS5LVJKsXLly4vBZLkrbVJNyzyW214foVwGHgCPBa4BeTPP8Zd6o6XlVLVbW0f//+adsqcKK85HugkSbz3NeAgxPXDwCPbbLNfVX1FeCRJOcYhf2ZmbRSI06U19D5Hmisycj9DHA4yTVJ9gI3Ayc2bPOfgL8BkGQfozLNw7NsqHCivOR7oLFtw72qngRuBe4BPgXcXVUPJrk9yU3jze4BPp/kIeBe4F9U1ed3qtGD5UR5DZ3vgcZStbF83o2lpaVaXV2dy3MvtJUVT2Cphdb6JTzw90CSs1W1tO12hrukrlgyb69puLv8gKTOWDLvjuEuqTOWzLvjkr+SOnNxfaUBl8w7Y7hL6pTrK3XDsowk9ZDhLkk9ZLhLUg8Z7pLUQ4a7JPWQ4S5JPWS4S1IPGe5T8jwBkhaBBzFNwUWPJC0KR+5TcNEjSYvCcJ+Cix5JWhSWZabgokeSFoXhPiUXPZK0CCzLSFIPGe6S1EOGe9ecKC/N10Deg9bcu+REeWm+BvQedOTeJSfKS/M1oPeg4d4lJ8pL8zWg96BlmS45UV6arwG9B1NVc3nipaWlWl1dnctzS1pcKyuDyOYtJTlbVUvbbefIXdLCGNDnoa1Zc5e0MAb0eWhrjcI9yY1JziU5n+S2S2z3miSVZNt/GSRpWgP6PLS1bcsySfYAdwCvBNaAM0lOVNVDG7Z7LvDPgI/uREMlaUCfh7bWpOZ+HXC+qh4GSHIXcBR4aMN2bwfeCfz4TFsoSRNcvK+ZJmWZq4BHJ66vjW/7qiTfBhysqv8yw7ZJki5Tk3DPJrd9df5kkmcB7wZ+bNsHSm5Jsppk9cKFC81bKUmaSpNwXwMOTlw/ADw2cf25wMuA00l+D/grwInNPlStquNVtVRVS/v377/8VkuSLqlJuJ8BDie5Jsle4GbgxMUfVtUXq2pfVR2qqkPAfcBNVeURSpI0J9uGe1U9CdwK3AN8Cri7qh5McnuSm3a6gZKk6TU6QrWqTgInN9z21i22PdK+WZKkNjxCVZJ6yHCXpB4aXLgP5AxbkgZuUKtCuqKcpKEY1MjdFeUkDcWgwt0V5SQNxaDKMq4oJ2koBhXu4IpykoZhUGUZSRoKw12Seshwn5YT5aVhW5AMGFzNvRUnykvDtkAZ4Mh9Gk6Ul4ZtgTLAcJ+GE+WlYVugDLAsMw0nykvDtkAZkKrafqsdsLS0VKurnqxJkqaR5GxVPeM0phtZlpGkHjLcJQ3KgsxkbM2au6TBWKCZjK05cpc0GAs0k7E1w13SYCzQTMbWLMtIGowFmsnYmuEuaVCGsuy3ZRlJ6iHDXZJ6yHCXpB4y3CWphwx3SeqhRuGe5MYk55KcT3LbJj//50keSvJAklNJrp59U0eGcuiwJLWx7VTIJHuAO4BXAmvAmSQnquqhic0+BixV1ZeSvBl4J/ADs27skA4dlqQ2mozcrwPOV9XDVbUO3AUcndygqu6tqi+Nr94HHJhtM0eGdOiwJLXRJNyvAh6duL42vm0rbwL+a5tGbWVIhw5LUhtNjlDNJrdteoaPJK8DloDrt/j5LcAtAC996UsbNvFpQzp0WJLaaBLua8DBiesHgMc2bpTke4CfAK6vqi9v9kBVdRw4DqMzMU3dWoZz6LAktdGkLHMGOJzkmiR7gZuBE5MbJPk24D8AN1XVE7NvpiRpGtuGe1U9CdwK3AN8Cri7qh5McnuSm8abvQt4DvDrSe5PcmKLh5MkdaDRqpBVdRI4ueG2t05c/p4Zt0uS1IJHqEpSDw0v3D3EVdI8dZRBwzpZh4e4SpqnDjNoWCN3D3GVNE8dZtCwwt1DXCXNU4cZNKyyjIe4SpqnDjMoVZd1oGhrS0tLtbq6OpfnlqRFleRsVS1tt92wyjKSNBCGuyT1kOEuST1kuEvSFBblOMhhzZaRpBYW6ThIR+6S1NAiHQdpuEtSQ4t0HKRlGUlqaJGOgzTcJWkKi3KqT8syktRDhrsk9ZDhLkk9ZLhLUg8Z7pLUQ4a7JPWQ4S5JPWS4S1IPGe6S1EOGuyT1kOEuST1kuEtSDzUK9yQ3JjmX5HyS2zb5+ZVJfm38848mOTTrhkqSmts23JPsAe4AXgVcC7w2ybUbNnsT8IWq+ovAu4GfmXVDJUnNNRm5Xwecr6qHq2oduAs4umGbo8Avjy9/CLghSWbXzAmLcgJDSZqjJuu5XwU8OnF9DfjOrbapqieTfBF4EfC5WTTyqxbpBIaSNEdNRu6bjcDrMrYhyS1JVpOsXrhwoUn7vtYincBQkuaoSbivAQcnrh8AHttqmyRXAM8D/nDjA1XV8apaqqql/fv3T9/aRTqBoSTNUZOyzBngcJJrgM8CNwN/f8M2J4A3ACvAa4DfqapnjNxbW6QTGErSHG0b7uMa+q3APcAe4M6qejDJ7cBqVZ0Afgn4YJLzjEbsN+9YixflBIaSNEeNTpBdVSeBkxtue+vE5T8Fvn+2TZMkXS6PUJWkHjLcJamHDHdJ6iHDXZJ6yHCXpB4y3CWpQ10tj9VoKqQkqb0ul8dy5C5JHelyeSzDXZI60uXyWJZlJKkjXS6PZbhLUoe6Wh7Lsowk9ZDhLkk9ZLhLUg8Z7pLUQ4a7JPWQ4S5JPZSdONVpoydO/hg4N5cnn719wOfm3YgZsS+7k33ZnebRl6urav92G81znvu5qlqa4/PPTJJV+7L72Jfdyb50w7KMJPWQ4S5JPTTPcD8+x+eeNfuyO9mX3cm+dGBuH6hKknaOZRlJ6qGZhHuSO5M8keSTE7e9Isl9Se5PsprkuvHtz0vyn5N8PMmDSd64xWOeTnJufP/7k3zDLNo64768IMlvJXkgye8medkWj3lNko8m+UySX0uyd4H78v4kj0zsl1fMsS9/KclKkk+MX1NfP/GzY0nOj19D37vFY+6m/dK2L53vl2n6keRFSe5N8idJ3nOJx3xhkg+P98mHk7xgp/uxg315W5LPTuyTV3fRl6+qqtZfwF8Hvh345MRt/w141fjyq4HT48v/CviZ8eX9wB8Cezd5zNPA0izat4N9eRfwU+PL3wKc2uIx7wZuHl/+eeDNC9yX9wOv2SX75Qxw/fjyDwJvH1++Fvg4cCVwDfC/gT27fL+07Uvn+2XKfvw54K8C/xR4zyUe853AbePLt13MigXty9uAH+9yn0x+zWTkXlUfYRTSX3MzcHH08TzgsYnbn5skwHPG93tyFu2YhSn7ci1wany/TwOHknzj5B3H/fxu4EPjm34Z+Nuzb/kzzbov87RFX74Z+Mj48oeB7xtfPgrcVVVfrqpHgPPAdZN33IX75bL7Mi/T9KOq/l9V/Q/gT7d52KOM9gXs0n0yRV/maidr7j8KvCvJo8C/BY6Nb38P8K2MQuUTwI9U1Z9t8RjvG/8786/Hb8Z52aovHwf+LsC4vHE1cGDDfV8E/N+quvgHbA24asdbvLU2fbno34zLN+9OcuVON/gSPgncNL78/cDB8eWrgEcnttvsd77b9kubvly0G/bLVv1o6hur6nGA8fdOyrFbaNsXgFvH++TOrkpMF+1kuL8ZeEtVHQTeAvzS+PbvBe4HXgK8AnjPZH1xwj+oqpcDf2389Q93sK3b2aovPw28IMn9wA8DH+OZ/4Vs9kdpnlOU2vQFRn8MvgX4y8ALgX+54y3e2g8CP5TkLPBcYH18e5Pf+W7bL236Artnv2zVj0XUti/vBf4Co5x7HPjZ2Tbv0nYy3N8A/Ob48q/z9L+SbwR+s0bOA48welF+jar67Pj7HwO/wnz/Fd20L1X1R1X1xqp6BfB6Rp8hPLLhvp8Dnp/k4lIPB3i6FDIPbfpCVT0+3ndfBt7HHPdLVX26qv5mVX0H8KuM6tEwGt1OjrI2+53vqv3Ssi+7Zr9coh9N/UGSFwOMvz8x6zY21bYvVfUHVfXUuDLxC3S8T3Yy3B8Drh9f/m7gM+PLvw/cADCu6X4z8PDkHZNckWTf+PKzgb/F6F+kedm0L0mePzHD4h8DH6mqP5q8Y40+WbkXeM34pjcAv73jLd7aZfdlvN3FN14Y1UPntl8ynkGV5FnATzL6UBTgBHBzkiuTXAMcBn538r67bb+06cv4frtiv1yiH02dYLQvYPfuk6b3f/HE1b9D1/tkFp/KMvqr9jjwFUYjjTcx+jT5LKNa7keB7xhv+xJGMzY+Me7s6yYe5/56+tPos8ADwIPAv2OTGQI78TVlX5YZheOnGY2GXzDxOCeBl4wv/3lGb8jzjEbLVy5wX35nYt/9R+A5c+zLjwD/a/z104wPyhtv/xOMRlrnGM8O2uX7pW1fOt8vl9GP32P0oeWfjLe/dnz7LzKeGcfos5BT49fiKeCFu3SfNOnLB8f75AFGf7Re3EVfLn55hKok9ZBHqEpSDxnuktRDhrsk9ZDhLkk9ZLhLUg8Z7pLUQ4a7JPWQ4S5JPfT/AaPtywNn2gEJAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# train the rnn and monitor results\n",
    "n_steps = 75\n",
    "print_every = 15\n",
    "\n",
    "trained_rnn = train(rnn, n_steps, print_every)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Time-Series Prediction\n",
    "\n",
    "Time-series prediction can be applied to many tasks. Think about weather forecasting or predicting the ebb and flow of stock market prices. You can even try to generate predictions much further in the future than just one time step!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}