FINISHED 07b Exquisite Corpse.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PREP: DOWNLOAD SPACY FOR ENGLISH"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to install a library that is fantastic at quickly parsing through text, called SPACY. Then we'll need to download the English package or Spacy to know which language to use. This is very cool, but it's going to take a while..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !conda install -c conda-forge spacy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# IF THIS CELL RUNS FINE, THE INSTALL WORKED!\n",
    "# import spacy\n",
    "\n",
    "# NOW WE NEED TO DOWNLOAD THE ENGLISH LANGUAGE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !python -m spacy download en"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BUILDING OUR EXQUISITE CORPSE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>line</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>Curiosity, earnest research to learn the hidden laws of nature, gladness akin to rapture, as they were unfolded to me, are among the earliest sensations I can remember.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101</th>\n",
       "      <td>How pleased you would be to remark the improvement of our Ernest.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>102</th>\n",
       "      <td>The possession of these treasures gave me extreme delight; I now continually studied and exercised my mind upon these histories, whilst my friends were employed in their ordinary occupations.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>103</th>\n",
       "      <td>I read of men concerned in public affairs, governing or massacring their species.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>104</th>\n",
       "      <td>Will you hear it?  Most willingly.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                                                                                                                                 line\n",
       "100                          Curiosity, earnest research to learn the hidden laws of nature, gladness akin to rapture, as they were unfolded to me, are among the earliest sensations I can remember.\n",
       "101                                                                                                                                 How pleased you would be to remark the improvement of our Ernest.\n",
       "102   The possession of these treasures gave me extreme delight; I now continually studied and exercised my mind upon these histories, whilst my friends were employed in their ordinary occupations.\n",
       "103                                                                                                                 I read of men concerned in public affairs, governing or massacring their species.\n",
       "104                                                                                                                                                                Will you hear it?  Most willingly."
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas\n",
    "pandas.set_option('display.max_colwidth', 300)\n",
    "\n",
    "train_lines = pandas.read_csv('data/sentences.csv', encoding='utf-8')\n",
    "train_lines[100:105]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import Helpers\n",
    "helper = Helpers.Exquisite_Corpse"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>line</th>\n",
       "      <th>tokens</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>Curiosity, earnest research to learn the hidden laws of nature, gladness akin to rapture, as they were unfolded to me, are among the earliest sensations I can remember.</td>\n",
       "      <td>[ , curiosity, ,, earnest, research, to, learn, the, hidden, laws, of, nature, ,, gladness, akin, to, rapture, ,, as, they, were, unfolded, to, me, ,, are, among, the, earliest, sensations, i, can, remember, .]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101</th>\n",
       "      <td>How pleased you would be to remark the improvement of our Ernest.</td>\n",
       "      <td>[how, pleased, you, would, be, to, remark, the, improvement, of, our, ernest, .]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>102</th>\n",
       "      <td>The possession of these treasures gave me extreme delight; I now continually studied and exercised my mind upon these histories, whilst my friends were employed in their ordinary occupations.</td>\n",
       "      <td>[ , the, possession, of, these, treasures, gave, me, extreme, delight, ;, i, now, continually, studied, and, exercised, my, mind, upon, these, histories, ,, whilst, my, friends, were, employed, in, their, ordinary, occupations, .]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>103</th>\n",
       "      <td>I read of men concerned in public affairs, governing or massacring their species.</td>\n",
       "      <td>[i, read, of, men, concerned, in, public, affairs, ,, governing, or, massacring, their, species, .]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>104</th>\n",
       "      <td>Will you hear it?  Most willingly.</td>\n",
       "      <td>[will, you, hear, it, ?,  , most, willingly, .]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                                                                                                                                 line  \\\n",
       "100                          Curiosity, earnest research to learn the hidden laws of nature, gladness akin to rapture, as they were unfolded to me, are among the earliest sensations I can remember.   \n",
       "101                                                                                                                                 How pleased you would be to remark the improvement of our Ernest.   \n",
       "102   The possession of these treasures gave me extreme delight; I now continually studied and exercised my mind upon these histories, whilst my friends were employed in their ordinary occupations.   \n",
       "103                                                                                                                 I read of men concerned in public affairs, governing or massacring their species.   \n",
       "104                                                                                                                                                                Will you hear it?  Most willingly.   \n",
       "\n",
       "                                                                                                                                                                                                                                     tokens  \n",
       "100                      [ , curiosity, ,, earnest, research, to, learn, the, hidden, laws, of, nature, ,, gladness, akin, to, rapture, ,, as, they, were, unfolded, to, me, ,, are, among, the, earliest, sensations, i, can, remember, .]  \n",
       "101                                                                                                                                                        [how, pleased, you, would, be, to, remark, the, improvement, of, our, ernest, .]  \n",
       "102  [ , the, possession, of, these, treasures, gave, me, extreme, delight, ;, i, now, continually, studied, and, exercised, my, mind, upon, these, histories, ,, whilst, my, friends, were, employed, in, their, ordinary, occupations, .]  \n",
       "103                                                                                                                                     [i, read, of, men, concerned, in, public, affairs, ,, governing, or, massacring, their, species, .]  \n",
       "104                                                                                                                                                                                         [will, you, hear, it, ?,  , most, willingly, .]  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import spacy\n",
    "encoder = spacy.load('en')\n",
    "\n",
    "train_lines['tokens'] = helper.text_to_tokens(train_lines['line'], encoder)\n",
    "train_lines[['line','tokens']][100:105]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "lexicon sample (4221 total items):\n",
      "[('actually', 206), ('thrush', 1270), ('supple', 2318), ('longer', 2319), ('private', 2320)]\n"
     ]
    }
   ],
   "source": [
    "import pickle\n",
    "import os\n",
    "\n",
    "lexicon = helper.make_lexicon(token_seqs=train_lines['tokens'], min_freq=2)\n",
    "\n",
    "filename = 'data/sentences_lexicon.pkl'\n",
    "\n",
    "if not os.path.exists(filename):\n",
    "    open(filename, 'w+').close()\n",
    "\n",
    "with open(filename, 'wb') as f:\n",
    "    pickle.dump(lexicon, f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "filename = 'data/sentence_lexicon.pkl'\n",
    "\n",
    "if not os.path.exists(filename):\n",
    "    open(filename, 'w+').close()\n",
    "    \n",
    "with open(filename, 'wb') as f:\n",
    "    pickle.dump(lexicon, f)\n",
    "    f.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1640"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lexicon[\"fellows\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'dict'>\n",
      "4221\n"
     ]
    }
   ],
   "source": [
    "print(type(lexicon))\n",
    "print(len(lexicon))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "lexicon_lookup = helper.get_lexicon_lookup(lexicon)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>tokens</th>\n",
       "      <th>line_ids</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>500</th>\n",
       "      <td>[everything, nourishes, what, is, strong, already, .]</td>\n",
       "      <td>[185, 1, 3881, 4058, 748, 351, 262]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>501</th>\n",
       "      <td>[i, covered, it, carefully, with, dry, wood, and, leaves, and, placed, wet, branches, upon, it, ;, and, then, ,, spreading, my, cloak, ,, i, lay, on, the, ground, and, sank, into, sleep, .]</td>\n",
       "      <td>[868, 1702, 1204, 859, 1189, 1070, 561, 4070, 2577, 4070, 1389, 3865, 3092, 3978, 1204, 691, 4070, 2681, 1330, 1, 2921, 150, 1330, 868, 1114, 3510, 1896, 2962, 4070, 1277, 2839, 1759, 262]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>502</th>\n",
       "      <td>[  , he, made, no, answer, ,, and, they, were, again, silent, till, they, had, gone, down, the, dance, ,, when, he, asked, her, if, she, and, her, sisters, did, not, very, often, walk, to, meryton, .]</td>\n",
       "      <td>[2275, 2294, 2500, 3128, 1777, 1330, 4070, 3863, 3124, 2137, 3391, 2628, 3863, 2284, 1018, 1900, 1896, 3118, 1330, 3053, 2294, 501, 1463, 1168, 2680, 4070, 1463, 2196, 2810, 281, 2034, 1602, 3168, 3700, 3307, 262]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>503</th>\n",
       "      <td>[the, rain, continued, the, whole, evening, without, intermission, ;, jane, certainly, could, not, come, back, .]</td>\n",
       "      <td>[1896, 1453, 3572, 1896, 2508, 2217, 3579, 1, 691, 741, 2207, 2052, 281, 2748, 904, 262]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>504</th>\n",
       "      <td>[in, a, doleful, voice,   , bennet, began, the, projected, conversation, :, oh, .]</td>\n",
       "      <td>[1929, 873, 1, 3613, 2275, 267, 3574, 1896, 1, 1316, 2607, 319, 262]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>505</th>\n",
       "      <td>[i, shall, tell, colonel, forster, it, will, be, quite, a, shame, if, he, does, not, .]</td>\n",
       "      <td>[868, 3519, 3915, 3729, 524, 1204, 1433, 1575, 372, 873, 2768, 1168, 2294, 3798, 281, 262]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>506</th>\n",
       "      <td>[and, you, may, be, certain, when, i, have, the, honour, of, seeing, her, again, ,, i, shall, speak, in, the, very, highest, terms, of, your, modesty, ,, economy, ,, and, other, amiable, qualification, .]</td>\n",
       "      <td>[4070, 1633, 1926, 1575, 1339, 3053, 868, 3921, 1896, 4007, 1954, 318, 1463, 2137, 1330, 868, 3519, 1635, 1929, 1896, 2034, 184, 2707, 1954, 3850, 3048, 1330, 1, 1330, 4070, 293, 1308, 1, 262]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>507</th>\n",
       "      <td>[their, eyes, were, immediately, wandering, up, in, the, street, in, quest, of, the, officers, ,, and, nothing, less, than, a, very, smart, bonnet, indeed, ,, or, a, really, new, muslin, in, a, shop, window, ,, could, recall, them, .]</td>\n",
       "      <td>[2098, 3512, 3124, 2048, 4066, 43, 1929, 1896, 3516, 1929, 2130, 1954, 1896, 4048, 1330, 4070, 1017, 785, 2389, 873, 2034, 1501, 1, 609, 1330, 4135, 873, 3583, 2993, 1, 1929, 873, 800, 2687, 1330, 2052, 509, 1208, 262]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>508</th>\n",
       "      <td>[  , the, astonishment, which, i, had, at, first, experienced, on, this, discovery, soon, gave, place, to, delight, and, rapture, .]</td>\n",
       "      <td>[2275, 1896, 223, 1892, 868, 2284, 2926, 2392, 2859, 3510, 1948, 3576, 3781, 1634, 2955, 3700, 2967, 4070, 3421, 262]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>509</th>\n",
       "      <td>[yet, she, appeared, confident, in, innocence, and, did, not, tremble, ,, although, gazed, on, and, execrated, by, thousands, ,, for, all, the, kindness, which, her, beauty, might, otherwise, have, excited, was, obliterated, in, the, minds, of, the, spectators, by, the, imagination, of, the, eno...</td>\n",
       "      <td>[1587, 2680, 1185, 40, 1929, 3675, 4070, 2810, 281, 1866, 1330, 2703, 337, 3510, 4070, 1, 1003, 540, 1330, 2988, 2506, 1896, 672, 1892, 1463, 16, 465, 4095, 3921, 2911, 991, 3849, 1929, 1896, 1652, 1954, 1896, 1, 1003, 1896, 2910, 1954, 1896, 1834, 2680, 991, 4122, 3700, 3921, 2544, 262]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                                                                                                                                                                                                                                          tokens  \\\n",
       "500                                                                                                                                                                                                                                                        [everything, nourishes, what, is, strong, already, .]   \n",
       "501                                                                                                                [i, covered, it, carefully, with, dry, wood, and, leaves, and, placed, wet, branches, upon, it, ;, and, then, ,, spreading, my, cloak, ,, i, lay, on, the, ground, and, sank, into, sleep, .]   \n",
       "502                                                                                                     [  , he, made, no, answer, ,, and, they, were, again, silent, till, they, had, gone, down, the, dance, ,, when, he, asked, her, if, she, and, her, sisters, did, not, very, often, walk, to, meryton, .]   \n",
       "503                                                                                                                                                                                            [the, rain, continued, the, whole, evening, without, intermission, ;, jane, certainly, could, not, come, back, .]   \n",
       "504                                                                                                                                                                                                                           [in, a, doleful, voice,   , bennet, began, the, projected, conversation, :, oh, .]   \n",
       "505                                                                                                                                                                                                                      [i, shall, tell, colonel, forster, it, will, be, quite, a, shame, if, he, does, not, .]   \n",
       "506                                                                                                 [and, you, may, be, certain, when, i, have, the, honour, of, seeing, her, again, ,, i, shall, speak, in, the, very, highest, terms, of, your, modesty, ,, economy, ,, and, other, amiable, qualification, .]   \n",
       "507                                                                   [their, eyes, were, immediately, wandering, up, in, the, street, in, quest, of, the, officers, ,, and, nothing, less, than, a, very, smart, bonnet, indeed, ,, or, a, really, new, muslin, in, a, shop, window, ,, could, recall, them, .]   \n",
       "508                                                                                                                                                                         [  , the, astonishment, which, i, had, at, first, experienced, on, this, discovery, soon, gave, place, to, delight, and, rapture, .]   \n",
       "509  [yet, she, appeared, confident, in, innocence, and, did, not, tremble, ,, although, gazed, on, and, execrated, by, thousands, ,, for, all, the, kindness, which, her, beauty, might, otherwise, have, excited, was, obliterated, in, the, minds, of, the, spectators, by, the, imagination, of, the, eno...   \n",
       "\n",
       "                                                                                                                                                                                                                                                                                             line_ids  \n",
       "500                                                                                                                                                                                                                                                               [185, 1, 3881, 4058, 748, 351, 262]  \n",
       "501                                                                                                      [868, 1702, 1204, 859, 1189, 1070, 561, 4070, 2577, 4070, 1389, 3865, 3092, 3978, 1204, 691, 4070, 2681, 1330, 1, 2921, 150, 1330, 868, 1114, 3510, 1896, 2962, 4070, 1277, 2839, 1759, 262]  \n",
       "502                                                                             [2275, 2294, 2500, 3128, 1777, 1330, 4070, 3863, 3124, 2137, 3391, 2628, 3863, 2284, 1018, 1900, 1896, 3118, 1330, 3053, 2294, 501, 1463, 1168, 2680, 4070, 1463, 2196, 2810, 281, 2034, 1602, 3168, 3700, 3307, 262]  \n",
       "503                                                                                                                                                                                                          [1896, 1453, 3572, 1896, 2508, 2217, 3579, 1, 691, 741, 2207, 2052, 281, 2748, 904, 262]  \n",
       "504                                                                                                                                                                                                                              [1929, 873, 1, 3613, 2275, 267, 3574, 1896, 1, 1316, 2607, 319, 262]  \n",
       "505                                                                                                                                                                                                        [868, 3519, 3915, 3729, 524, 1204, 1433, 1575, 372, 873, 2768, 1168, 2294, 3798, 281, 262]  \n",
       "506                                                                                                  [4070, 1633, 1926, 1575, 1339, 3053, 868, 3921, 1896, 4007, 1954, 318, 1463, 2137, 1330, 868, 3519, 1635, 1929, 1896, 2034, 184, 2707, 1954, 3850, 3048, 1330, 1, 1330, 4070, 293, 1308, 1, 262]  \n",
       "507                                                                        [2098, 3512, 3124, 2048, 4066, 43, 1929, 1896, 3516, 1929, 2130, 1954, 1896, 4048, 1330, 4070, 1017, 785, 2389, 873, 2034, 1501, 1, 609, 1330, 4135, 873, 3583, 2993, 1, 1929, 873, 800, 2687, 1330, 2052, 509, 1208, 262]  \n",
       "508                                                                                                                                                                             [2275, 1896, 223, 1892, 868, 2284, 2926, 2392, 2859, 3510, 1948, 3576, 3781, 1634, 2955, 3700, 2967, 4070, 3421, 262]  \n",
       "509  [1587, 2680, 1185, 40, 1929, 3675, 4070, 2810, 281, 1866, 1330, 2703, 337, 3510, 4070, 1, 1003, 540, 1330, 2988, 2506, 1896, 672, 1892, 1463, 16, 465, 4095, 3921, 2911, 991, 3849, 1929, 1896, 1652, 1954, 1896, 1, 1003, 1896, 2910, 1954, 1896, 1834, 2680, 991, 4122, 3700, 3921, 2544, 262]  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_lines['line_ids'] = helper.tokens_to_ids(all_tokens=train_lines['tokens'], lexicon=lexicon)\n",
    "train_lines[['tokens','line_ids']][500:510]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[   0    0    0 ..., 1463 3681  262]\n",
      " [   0    0    0 ..., 1896 2754  262]\n",
      " [   0    0    0 ...,  745  636  262]\n",
      " ..., \n",
      " [   0    0    0 ..., 3052 1463  262]\n",
      " [   0    0    0 ...,    1  344  262]\n",
      " [   0    0    0 ..., 1896 2007  262]]\n",
      "SHAPE: (4000, 142)\n"
     ]
    }
   ],
   "source": [
    "from keras.preprocessing.sequence import pad_sequences\n",
    "\n",
    "max_length = max( [len(ids) for ids in train_lines['line_ids']])\n",
    "\n",
    "train_padded_ids = pad_sequences(train_lines['line_ids'], maxlen=max_length)\n",
    "print(train_padded_ids)\n",
    "\n",
    "print(\"SHAPE:\", train_padded_ids.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input word</th>\n",
       "      <th>output word</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-</td>\n",
       "      <td>but</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>but</td>\n",
       "      <td>still</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>still</td>\n",
       "      <td>he</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>he</td>\n",
       "      <td>would</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>would</td>\n",
       "      <td>be</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>be</td>\n",
       "      <td>her</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>her</td>\n",
       "      <td>husband</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>husband</td>\n",
       "      <td>.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  input word output word\n",
       "0          -         but\n",
       "1        but       still\n",
       "2      still          he\n",
       "3         he       would\n",
       "4      would          be\n",
       "5         be         her\n",
       "6        her     husband\n",
       "7    husband           ."
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pandas.DataFrame( list(zip([\"-\"] + train_lines['tokens'].loc[0], \n",
    "                      train_lines['tokens'].loc[0])),\n",
    "                 columns=['input word', 'output word'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     input words  output words\n",
      "0              0             0\n",
      "1              0             0\n",
      "2              0             0\n",
      "3              0             0\n",
      "4              0             0\n",
      "5              0             0\n",
      "6              0             0\n",
      "7              0             0\n",
      "8              0             0\n",
      "9              0             0\n",
      "10             0             0\n",
      "11             0             0\n",
      "12             0             0\n",
      "13             0             0\n",
      "14             0             0\n",
      "15             0             0\n",
      "16             0             0\n",
      "17             0             0\n",
      "18             0             0\n",
      "19             0             0\n",
      "20             0             0\n",
      "21             0             0\n",
      "22             0             0\n",
      "23             0             0\n",
      "24             0             0\n",
      "25             0             0\n",
      "26             0             0\n",
      "27             0             0\n",
      "28             0             0\n",
      "29             0             0\n",
      "..           ...           ...\n",
      "111            0             0\n",
      "112            0             0\n",
      "113            0             0\n",
      "114            0             0\n",
      "115            0             0\n",
      "116            0             0\n",
      "117            0             0\n",
      "118            0             0\n",
      "119            0             0\n",
      "120            0             0\n",
      "121            0             0\n",
      "122            0             0\n",
      "123            0             0\n",
      "124            0             0\n",
      "125            0             0\n",
      "126            0             0\n",
      "127            0             0\n",
      "128            0             0\n",
      "129            0             0\n",
      "130            0             0\n",
      "131            0             0\n",
      "132            0             0\n",
      "133            0          3789\n",
      "134         3789          3126\n",
      "135         3126          2294\n",
      "136         2294          1521\n",
      "137         1521          1575\n",
      "138         1575          1463\n",
      "139         1463          3681\n",
      "140         3681           262\n",
      "\n",
      "[141 rows x 2 columns]\n"
     ]
    }
   ],
   "source": [
    "print(pandas.DataFrame(list(zip(train_padded_ids[0,:-1], train_padded_ids[0, 1:])), columns=['input words','output words']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.models import Model\n",
    "from keras.layers import Input, Dense, TimeDistributed\n",
    "from keras.layers.embeddings import Embedding\n",
    "from keras.layers.recurrent import GRU\n",
    "\n",
    "def create_model(seq_input_len, n_input_nodes, n_embedding_nodes, n_hidden_nodes, stateful=False, batch_size=None):\n",
    "\n",
    "    input_layer = Input(batch_shape=(batch_size, seq_input_len), name='input_layer')\n",
    "    \n",
    "    embedding_layer = Embedding(input_dim=n_input_nodes,\n",
    "                               output_dim=n_embedding_nodes,\n",
    "                               mask_zero=True, name='embedding_layer')(input_layer)\n",
    "    \n",
    "    gru_layer1 = GRU(n_hidden_nodes,\n",
    "                    return_sequences=True,\n",
    "                    stateful=stateful,\n",
    "                    name='hidden_layer1')(embedding_layer)\n",
    "    \n",
    "    gru_layer2 = GRU(n_hidden_nodes,\n",
    "                    return_sequences=True,\n",
    "                    stateful=stateful,\n",
    "                    name='hidden_layer2')(gru_layer1)\n",
    "    \n",
    "    output_layer = TimeDistributed(Dense(n_input_nodes, activation=\"softmax\"),\n",
    "                                  name='output_layer')(gru_layer2)\n",
    "    \n",
    "    model = Model(inputs=input_layer, outputs=output_layer)\n",
    "    \n",
    "    model.compile(loss='sparse_categorical_crossentropy', optimizer='adam')\n",
    "    \n",
    "    return model\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = create_model(seq_input_len=train_padded_ids.shape[-1] - 1,\n",
    "                    n_input_nodes = len(lexicon) + 1,\n",
    "                    n_embedding_nodes = 250,\n",
    "                    n_hidden_nodes = 400)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4221"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(lexicon)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "4000/4000 [==============================] - ETA: 10:54 - loss: 8.34 - ETA: 7:43 - loss: 8.3466 - ETA: 6:38 - loss: 8.344 - ETA: 6:07 - loss: 8.341 - ETA: 5:53 - loss: 8.336 - ETA: 5:46 - loss: 8.325 - ETA: 5:38 - loss: 8.300 - ETA: 5:29 - loss: 8.248 - ETA: 5:21 - loss: 8.151 - ETA: 5:16 - loss: 8.028 - ETA: 5:10 - loss: 7.889 - ETA: 5:04 - loss: 7.782 - ETA: 4:59 - loss: 7.671 - ETA: 4:55 - loss: 7.582 - ETA: 4:52 - loss: 7.521 - ETA: 4:48 - loss: 7.458 - ETA: 4:45 - loss: 7.404 - ETA: 4:42 - loss: 7.348 - ETA: 4:39 - loss: 7.303 - ETA: 4:36 - loss: 7.268 - ETA: 4:34 - loss: 7.238 - ETA: 4:32 - loss: 7.204 - ETA: 4:30 - loss: 7.172 - ETA: 4:27 - loss: 7.146 - ETA: 4:25 - loss: 7.113 - ETA: 4:22 - loss: 7.087 - ETA: 4:20 - loss: 7.070 - ETA: 4:18 - loss: 7.053 - ETA: 4:16 - loss: 7.036 - ETA: 4:14 - loss: 7.020 - ETA: 4:12 - loss: 7.003 - ETA: 4:10 - loss: 6.987 - ETA: 4:08 - loss: 6.969 - ETA: 4:06 - loss: 6.953 - ETA: 4:04 - loss: 6.933 - ETA: 4:03 - loss: 6.918 - ETA: 4:01 - loss: 6.904 - ETA: 4:00 - loss: 6.894 - ETA: 3:58 - loss: 6.882 - ETA: 3:57 - loss: 6.867 - ETA: 3:55 - loss: 6.854 - ETA: 3:54 - loss: 6.842 - ETA: 3:52 - loss: 6.836 - ETA: 3:50 - loss: 6.824 - ETA: 3:49 - loss: 6.810 - ETA: 3:47 - loss: 6.802 - ETA: 3:45 - loss: 6.794 - ETA: 3:43 - loss: 6.788 - ETA: 3:42 - loss: 6.778 - ETA: 3:40 - loss: 6.768 - ETA: 3:39 - loss: 6.760 - ETA: 3:37 - loss: 6.751 - ETA: 3:36 - loss: 6.744 - ETA: 3:34 - loss: 6.734 - ETA: 3:32 - loss: 6.724 - ETA: 3:31 - loss: 6.717 - ETA: 3:29 - loss: 6.708 - ETA: 3:27 - loss: 6.701 - ETA: 3:26 - loss: 6.693 - ETA: 3:24 - loss: 6.685 - ETA: 3:23 - loss: 6.678 - ETA: 3:21 - loss: 6.671 - ETA: 3:20 - loss: 6.664 - ETA: 3:18 - loss: 6.658 - ETA: 3:17 - loss: 6.650 - ETA: 3:15 - loss: 6.644 - ETA: 3:13 - loss: 6.641 - ETA: 3:12 - loss: 6.635 - ETA: 3:10 - loss: 6.629 - ETA: 3:09 - loss: 6.623 - ETA: 3:07 - loss: 6.619 - ETA: 3:06 - loss: 6.615 - ETA: 3:04 - loss: 6.609 - ETA: 3:03 - loss: 6.602 - ETA: 3:01 - loss: 6.596 - ETA: 3:00 - loss: 6.593 - ETA: 2:58 - loss: 6.591 - ETA: 2:57 - loss: 6.588 - ETA: 2:55 - loss: 6.585 - ETA: 2:53 - loss: 6.581 - ETA: 2:52 - loss: 6.574 - ETA: 2:50 - loss: 6.569 - ETA: 2:49 - loss: 6.563 - ETA: 2:47 - loss: 6.558 - ETA: 2:46 - loss: 6.553 - ETA: 2:45 - loss: 6.548 - ETA: 2:43 - loss: 6.543 - ETA: 2:42 - loss: 6.539 - ETA: 2:40 - loss: 6.534 - ETA: 2:39 - loss: 6.530 - ETA: 2:37 - loss: 6.525 - ETA: 2:36 - loss: 6.521 - ETA: 2:34 - loss: 6.517 - ETA: 2:33 - loss: 6.514 - ETA: 2:31 - loss: 6.510 - ETA: 2:30 - loss: 6.507 - ETA: 2:28 - loss: 6.503 - ETA: 2:27 - loss: 6.498 - ETA: 2:25 - loss: 6.495 - ETA: 2:24 - loss: 6.491 - ETA: 2:22 - loss: 6.487 - ETA: 2:21 - loss: 6.483 - ETA: 2:19 - loss: 6.482 - ETA: 2:18 - loss: 6.480 - ETA: 2:16 - loss: 6.475 - ETA: 2:15 - loss: 6.472 - ETA: 2:13 - loss: 6.470 - ETA: 2:12 - loss: 6.469 - ETA: 2:10 - loss: 6.469 - ETA: 2:09 - loss: 6.467 - ETA: 2:08 - loss: 6.462 - ETA: 2:06 - loss: 6.461 - ETA: 2:05 - loss: 6.458 - ETA: 2:03 - loss: 6.456 - ETA: 2:02 - loss: 6.454 - ETA: 2:00 - loss: 6.451 - ETA: 1:59 - loss: 6.449 - ETA: 1:57 - loss: 6.447 - ETA: 1:56 - loss: 6.444 - ETA: 1:55 - loss: 6.443 - ETA: 1:53 - loss: 6.442 - ETA: 1:52 - loss: 6.439 - ETA: 1:50 - loss: 6.436 - ETA: 1:49 - loss: 6.433 - ETA: 1:47 - loss: 6.430 - ETA: 1:46 - loss: 6.428 - ETA: 1:44 - loss: 6.427 - ETA: 1:43 - loss: 6.426 - ETA: 1:42 - loss: 6.422 - ETA: 1:40 - loss: 6.420 - ETA: 1:39 - loss: 6.419 - ETA: 1:37 - loss: 6.417 - ETA: 1:36 - loss: 6.415 - ETA: 1:34 - loss: 6.412 - ETA: 1:33 - loss: 6.412 - ETA: 1:31 - loss: 6.409 - ETA: 1:30 - loss: 6.408 - ETA: 1:28 - loss: 6.406 - ETA: 1:27 - loss: 6.404 - ETA: 1:26 - loss: 6.403 - ETA: 1:24 - loss: 6.400 - ETA: 1:23 - loss: 6.398 - ETA: 1:21 - loss: 6.396 - ETA: 1:20 - loss: 6.395 - ETA: 1:18 - loss: 6.392 - ETA: 1:17 - loss: 6.391 - ETA: 1:16 - loss: 6.389 - ETA: 1:14 - loss: 6.388 - ETA: 1:13 - loss: 6.386 - ETA: 1:11 - loss: 6.386 - ETA: 1:10 - loss: 6.385 - ETA: 1:08 - loss: 6.382 - ETA: 1:07 - loss: 6.380 - ETA: 1:05 - loss: 6.378 - ETA: 1:04 - loss: 6.375 - ETA: 1:03 - loss: 6.373 - ETA: 1:01 - loss: 6.372 - ETA: 1:00 - loss: 6.371 - ETA: 58s - loss: 6.370 - ETA: 57s - loss: 6.36 - ETA: 55s - loss: 6.36 - ETA: 54s - loss: 6.36 - ETA: 52s - loss: 6.36 - ETA: 51s - loss: 6.36 - ETA: 50s - loss: 6.36 - ETA: 48s - loss: 6.36 - ETA: 47s - loss: 6.35 - ETA: 45s - loss: 6.35 - ETA: 44s - loss: 6.35 - ETA: 42s - loss: 6.35 - ETA: 41s - loss: 6.35 - ETA: 40s - loss: 6.35 - ETA: 38s - loss: 6.35 - ETA: 37s - loss: 6.35 - ETA: 35s - loss: 6.34 - ETA: 34s - loss: 6.34 - ETA: 32s - loss: 6.34 - ETA: 31s - loss: 6.34 - ETA: 30s - loss: 6.34 - ETA: 28s - loss: 6.34 - ETA: 27s - loss: 6.34 - ETA: 25s - loss: 6.33 - ETA: 24s - loss: 6.33 - ETA: 22s - loss: 6.33 - ETA: 21s - loss: 6.33 - ETA: 20s - loss: 6.33 - ETA: 18s - loss: 6.33 - ETA: 17s - loss: 6.33 - ETA: 15s - loss: 6.32 - ETA: 14s - loss: 6.32 - ETA: 12s - loss: 6.32 - ETA: 11s - loss: 6.32 - ETA: 10s - loss: 6.32 - ETA: 8s - loss: 6.3258 - ETA: 7s - loss: 6.324 - ETA: 5s - loss: 6.323 - ETA: 4s - loss: 6.322 - ETA: 2s - loss: 6.321 - ETA: 1s - loss: 6.320 - 286s 71ms/step - loss: 6.3194\n",
      "Epoch 2/10\n",
      "4000/4000 [==============================] - ETA: 4:36 - loss: 6.035 - ETA: 4:36 - loss: 5.981 - ETA: 4:36 - loss: 5.974 - ETA: 4:34 - loss: 6.007 - ETA: 4:33 - loss: 6.026 - ETA: 4:32 - loss: 6.043 - ETA: 4:30 - loss: 6.029 - ETA: 4:30 - loss: 6.048 - ETA: 4:28 - loss: 6.045 - ETA: 4:27 - loss: 6.036 - ETA: 4:26 - loss: 6.046 - ETA: 4:24 - loss: 6.057 - ETA: 4:23 - loss: 6.069 - ETA: 4:22 - loss: 6.071 - ETA: 4:20 - loss: 6.075 - ETA: 4:19 - loss: 6.064 - ETA: 4:17 - loss: 6.061 - ETA: 4:16 - loss: 6.054 - ETA: 4:14 - loss: 6.067 - ETA: 4:13 - loss: 6.064 - ETA: 4:11 - loss: 6.065 - ETA: 4:10 - loss: 6.065 - ETA: 4:08 - loss: 6.067 - ETA: 4:07 - loss: 6.072 - ETA: 4:05 - loss: 6.071 - ETA: 4:04 - loss: 6.069 - ETA: 4:03 - loss: 6.069 - ETA: 4:01 - loss: 6.067 - ETA: 4:00 - loss: 6.076 - ETA: 3:59 - loss: 6.078 - ETA: 3:57 - loss: 6.079 - ETA: 3:56 - loss: 6.080 - ETA: 3:54 - loss: 6.075 - ETA: 3:53 - loss: 6.072 - ETA: 3:51 - loss: 6.071 - ETA: 3:50 - loss: 6.073 - ETA: 3:49 - loss: 6.076 - ETA: 3:48 - loss: 6.075 - ETA: 3:47 - loss: 6.076 - ETA: 3:45 - loss: 6.078 - ETA: 3:44 - loss: 6.077 - ETA: 3:42 - loss: 6.078 - ETA: 3:41 - loss: 6.079 - ETA: 3:39 - loss: 6.076 - ETA: 3:38 - loss: 6.075 - ETA: 3:37 - loss: 6.080 - ETA: 3:35 - loss: 6.080 - ETA: 3:34 - loss: 6.080 - ETA: 3:32 - loss: 6.081 - ETA: 3:31 - loss: 6.081 - ETA: 3:30 - loss: 6.083 - ETA: 3:28 - loss: 6.084 - ETA: 3:27 - loss: 6.083 - ETA: 3:26 - loss: 6.082 - ETA: 3:25 - loss: 6.082 - ETA: 3:23 - loss: 6.081 - ETA: 3:22 - loss: 6.081 - ETA: 3:21 - loss: 6.080 - ETA: 3:19 - loss: 6.079 - ETA: 3:18 - loss: 6.081 - ETA: 3:16 - loss: 6.082 - ETA: 3:15 - loss: 6.084 - ETA: 3:14 - loss: 6.082 - ETA: 3:12 - loss: 6.083 - ETA: 3:11 - loss: 6.083 - ETA: 3:09 - loss: 6.083 - ETA: 3:08 - loss: 6.084 - ETA: 3:06 - loss: 6.085 - ETA: 3:05 - loss: 6.084 - ETA: 3:03 - loss: 6.084 - ETA: 3:02 - loss: 6.083 - ETA: 3:01 - loss: 6.082 - ETA: 2:59 - loss: 6.081 - ETA: 2:58 - loss: 6.082 - ETA: 2:56 - loss: 6.081 - ETA: 2:55 - loss: 6.081 - ETA: 2:54 - loss: 6.080 - ETA: 2:52 - loss: 6.078 - ETA: 2:51 - loss: 6.080 - ETA: 2:49 - loss: 6.080 - ETA: 2:48 - loss: 6.079 - ETA: 2:47 - loss: 6.079 - ETA: 2:45 - loss: 6.080 - ETA: 2:44 - loss: 6.080 - ETA: 2:42 - loss: 6.080 - ETA: 2:41 - loss: 6.078 - ETA: 2:39 - loss: 6.078 - ETA: 2:38 - loss: 6.078 - ETA: 2:37 - loss: 6.078 - ETA: 2:35 - loss: 6.077 - ETA: 2:34 - loss: 6.076 - ETA: 2:32 - loss: 6.078 - ETA: 2:31 - loss: 6.079 - ETA: 2:30 - loss: 6.077 - ETA: 2:28 - loss: 6.080 - ETA: 2:27 - loss: 6.080 - ETA: 2:25 - loss: 6.081 - ETA: 2:24 - loss: 6.080 - ETA: 2:22 - loss: 6.081 - ETA: 2:21 - loss: 6.081 - ETA: 2:20 - loss: 6.081 - ETA: 2:18 - loss: 6.081 - ETA: 2:17 - loss: 6.081 - ETA: 2:15 - loss: 6.081 - ETA: 2:14 - loss: 6.080 - ETA: 2:13 - loss: 6.081 - ETA: 2:11 - loss: 6.080 - ETA: 2:10 - loss: 6.079 - ETA: 2:08 - loss: 6.076 - ETA: 2:07 - loss: 6.077 - ETA: 2:06 - loss: 6.076 - ETA: 2:04 - loss: 6.076 - ETA: 2:03 - loss: 6.077 - ETA: 2:01 - loss: 6.077 - ETA: 2:00 - loss: 6.077 - ETA: 1:58 - loss: 6.076 - ETA: 1:57 - loss: 6.077 - ETA: 1:56 - loss: 6.078 - ETA: 1:54 - loss: 6.079 - ETA: 1:53 - loss: 6.078 - ETA: 1:51 - loss: 6.078 - ETA: 1:50 - loss: 6.077 - ETA: 1:49 - loss: 6.077 - ETA: 1:47 - loss: 6.078 - ETA: 1:46 - loss: 6.078 - ETA: 1:44 - loss: 6.078 - ETA: 1:43 - loss: 6.078 - ETA: 1:42 - loss: 6.078 - ETA: 1:40 - loss: 6.078 - ETA: 1:39 - loss: 6.079 - ETA: 1:37 - loss: 6.079 - ETA: 1:36 - loss: 6.078 - ETA: 1:34 - loss: 6.079 - ETA: 1:33 - loss: 6.079 - ETA: 1:32 - loss: 6.079 - ETA: 1:30 - loss: 6.079 - ETA: 1:29 - loss: 6.079 - ETA: 1:27 - loss: 6.078 - ETA: 1:26 - loss: 6.078 - ETA: 1:25 - loss: 6.078 - ETA: 1:23 - loss: 6.079 - ETA: 1:22 - loss: 6.079 - ETA: 1:20 - loss: 6.080 - ETA: 1:19 - loss: 6.079 - ETA: 1:17 - loss: 6.079 - ETA: 1:16 - loss: 6.079 - ETA: 1:15 - loss: 6.078 - ETA: 1:13 - loss: 6.079 - ETA: 1:12 - loss: 6.080 - ETA: 1:10 - loss: 6.079 - ETA: 1:09 - loss: 6.080 - ETA: 1:08 - loss: 6.080 - ETA: 1:06 - loss: 6.079 - ETA: 1:05 - loss: 6.080 - ETA: 1:03 - loss: 6.080 - ETA: 1:02 - loss: 6.081 - ETA: 1:00 - loss: 6.081 - ETA: 59s - loss: 6.081 - ETA: 58s - loss: 6.08 - ETA: 56s - loss: 6.08 - ETA: 55s - loss: 6.08 - ETA: 53s - loss: 6.08 - ETA: 52s - loss: 6.08 - ETA: 50s - loss: 6.08 - ETA: 49s - loss: 6.08 - ETA: 48s - loss: 6.08 - ETA: 46s - loss: 6.08 - ETA: 45s - loss: 6.08 - ETA: 43s - loss: 6.08 - ETA: 42s - loss: 6.08 - ETA: 41s - loss: 6.08 - ETA: 39s - loss: 6.08 - ETA: 38s - loss: 6.08 - ETA: 36s - loss: 6.07 - ETA: 35s - loss: 6.08 - ETA: 33s - loss: 6.08 - ETA: 32s - loss: 6.08 - ETA: 31s - loss: 6.08 - ETA: 29s - loss: 6.08 - ETA: 28s - loss: 6.08 - ETA: 26s - loss: 6.08 - ETA: 25s - loss: 6.07 - ETA: 24s - loss: 6.08 - ETA: 22s - loss: 6.08 - ETA: 21s - loss: 6.08 - ETA: 19s - loss: 6.08 - ETA: 18s - loss: 6.08 - ETA: 16s - loss: 6.08 - ETA: 15s - loss: 6.08 - ETA: 14s - loss: 6.08 - ETA: 12s - loss: 6.08 - ETA: 11s - loss: 6.08 - ETA: 9s - loss: 6.0810 - ETA: 8s - loss: 6.081 - ETA: 7s - loss: 6.081 - ETA: 5s - loss: 6.081 - ETA: 4s - loss: 6.080 - ETA: 2s - loss: 6.080 - ETA: 1s - loss: 6.079 - 283s 71ms/step - loss: 6.0798\n",
      "Epoch 3/10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4000/4000 [==============================] - ETA: 4:28 - loss: 6.156 - ETA: 4:31 - loss: 6.084 - ETA: 4:30 - loss: 6.013 - ETA: 4:30 - loss: 6.003 - ETA: 4:28 - loss: 6.010 - ETA: 4:27 - loss: 5.993 - ETA: 4:27 - loss: 5.992 - ETA: 4:26 - loss: 6.002 - ETA: 4:26 - loss: 6.008 - ETA: 4:25 - loss: 6.001 - ETA: 4:23 - loss: 5.995 - ETA: 4:23 - loss: 5.998 - ETA: 4:21 - loss: 6.001 - ETA: 4:19 - loss: 6.001 - ETA: 4:18 - loss: 6.003 - ETA: 4:17 - loss: 6.013 - ETA: 4:15 - loss: 6.014 - ETA: 4:14 - loss: 6.019 - ETA: 4:13 - loss: 6.026 - ETA: 4:12 - loss: 6.023 - ETA: 4:10 - loss: 6.027 - ETA: 4:09 - loss: 6.033 - ETA: 4:07 - loss: 6.035 - ETA: 4:06 - loss: 6.035 - ETA: 4:05 - loss: 6.037 - ETA: 4:04 - loss: 6.037 - ETA: 4:02 - loss: 6.035 - ETA: 4:01 - loss: 6.036 - ETA: 4:00 - loss: 6.035 - ETA: 3:58 - loss: 6.036 - ETA: 3:57 - loss: 6.040 - ETA: 3:56 - loss: 6.041 - ETA: 3:55 - loss: 6.038 - ETA: 3:53 - loss: 6.043 - ETA: 3:52 - loss: 6.043 - ETA: 3:50 - loss: 6.042 - ETA: 3:49 - loss: 6.044 - ETA: 3:47 - loss: 6.043 - ETA: 3:46 - loss: 6.041 - ETA: 3:45 - loss: 6.042 - ETA: 3:43 - loss: 6.042 - ETA: 3:42 - loss: 6.041 - ETA: 3:41 - loss: 6.044 - ETA: 3:39 - loss: 6.043 - ETA: 3:38 - loss: 6.042 - ETA: 3:37 - loss: 6.044 - ETA: 3:35 - loss: 6.045 - ETA: 3:34 - loss: 6.045 - ETA: 3:32 - loss: 6.044 - ETA: 3:31 - loss: 6.043 - ETA: 3:29 - loss: 6.039 - ETA: 3:28 - loss: 6.038 - ETA: 3:27 - loss: 6.038 - ETA: 3:25 - loss: 6.037 - ETA: 3:24 - loss: 6.037 - ETA: 3:22 - loss: 6.035 - ETA: 3:21 - loss: 6.036 - ETA: 3:19 - loss: 6.035 - ETA: 3:18 - loss: 6.034 - ETA: 3:17 - loss: 6.035 - ETA: 3:15 - loss: 6.033 - ETA: 3:14 - loss: 6.034 - ETA: 3:12 - loss: 6.035 - ETA: 3:11 - loss: 6.034 - ETA: 3:10 - loss: 6.034 - ETA: 3:08 - loss: 6.035 - ETA: 3:07 - loss: 6.034 - ETA: 3:05 - loss: 6.036 - ETA: 3:04 - loss: 6.037 - ETA: 3:02 - loss: 6.038 - ETA: 3:01 - loss: 6.039 - ETA: 3:00 - loss: 6.038 - ETA: 2:58 - loss: 6.038 - ETA: 2:57 - loss: 6.038 - ETA: 2:55 - loss: 6.037 - ETA: 2:54 - loss: 6.037 - ETA: 2:53 - loss: 6.038 - ETA: 2:51 - loss: 6.037 - ETA: 2:50 - loss: 6.038 - ETA: 2:48 - loss: 6.039 - ETA: 2:47 - loss: 6.041 - ETA: 2:46 - loss: 6.041 - ETA: 2:44 - loss: 6.042 - ETA: 2:43 - loss: 6.044 - ETA: 2:41 - loss: 6.043 - ETA: 2:40 - loss: 6.042 - ETA: 2:39 - loss: 6.043 - ETA: 2:37 - loss: 6.044 - ETA: 2:36 - loss: 6.044 - ETA: 2:34 - loss: 6.044 - ETA: 2:33 - loss: 6.045 - ETA: 2:31 - loss: 6.045 - ETA: 2:30 - loss: 6.043 - ETA: 2:29 - loss: 6.042 - ETA: 2:27 - loss: 6.043 - ETA: 2:26 - loss: 6.044 - ETA: 2:24 - loss: 6.045 - ETA: 2:23 - loss: 6.046 - ETA: 2:22 - loss: 6.048 - ETA: 2:20 - loss: 6.049 - ETA: 2:19 - loss: 6.050 - ETA: 2:17 - loss: 6.049 - ETA: 2:16 - loss: 6.049 - ETA: 2:15 - loss: 6.049 - ETA: 2:13 - loss: 6.048 - ETA: 2:12 - loss: 6.048 - ETA: 2:10 - loss: 6.050 - ETA: 2:09 - loss: 6.049 - ETA: 2:07 - loss: 6.048 - ETA: 2:06 - loss: 6.046 - ETA: 2:05 - loss: 6.046 - ETA: 2:03 - loss: 6.047 - ETA: 2:02 - loss: 6.048 - ETA: 2:00 - loss: 6.048 - ETA: 1:59 - loss: 6.048 - ETA: 1:58 - loss: 6.048 - ETA: 1:56 - loss: 6.047 - ETA: 1:55 - loss: 6.047 - ETA: 1:53 - loss: 6.046 - ETA: 1:52 - loss: 6.045 - ETA: 1:50 - loss: 6.045 - ETA: 1:49 - loss: 6.044 - ETA: 1:48 - loss: 6.044 - ETA: 1:46 - loss: 6.044 - ETA: 1:45 - loss: 6.045 - ETA: 1:43 - loss: 6.046 - ETA: 1:42 - loss: 6.046 - ETA: 1:41 - loss: 6.045 - ETA: 1:39 - loss: 6.044 - ETA: 1:38 - loss: 6.044 - ETA: 1:36 - loss: 6.044 - ETA: 1:35 - loss: 6.044 - ETA: 1:34 - loss: 6.044 - ETA: 1:32 - loss: 6.044 - ETA: 1:31 - loss: 6.044 - ETA: 1:29 - loss: 6.043 - ETA: 1:28 - loss: 6.043 - ETA: 1:27 - loss: 6.043 - ETA: 1:25 - loss: 6.043 - ETA: 1:24 - loss: 6.043 - ETA: 1:22 - loss: 6.043 - ETA: 1:21 - loss: 6.044 - ETA: 1:20 - loss: 6.043 - ETA: 1:18 - loss: 6.043 - ETA: 1:17 - loss: 6.043 - ETA: 1:15 - loss: 6.044 - ETA: 1:14 - loss: 6.044 - ETA: 1:13 - loss: 6.044 - ETA: 1:11 - loss: 6.044 - ETA: 1:10 - loss: 6.045 - ETA: 1:08 - loss: 6.045 - ETA: 1:07 - loss: 6.044 - ETA: 1:06 - loss: 6.044 - ETA: 1:04 - loss: 6.044 - ETA: 1:03 - loss: 6.043 - ETA: 1:01 - loss: 6.043 - ETA: 1:00 - loss: 6.043 - ETA: 58s - loss: 6.042 - ETA: 57s - loss: 6.04 - ETA: 56s - loss: 6.04 - ETA: 54s - loss: 6.04 - ETA: 53s - loss: 6.04 - ETA: 51s - loss: 6.04 - ETA: 50s - loss: 6.04 - ETA: 49s - loss: 6.04 - ETA: 47s - loss: 6.04 - ETA: 46s - loss: 6.04 - ETA: 44s - loss: 6.04 - ETA: 43s - loss: 6.03 - ETA: 42s - loss: 6.03 - ETA: 40s - loss: 6.03 - ETA: 39s - loss: 6.03 - ETA: 37s - loss: 6.03 - ETA: 36s - loss: 6.03 - ETA: 35s - loss: 6.03 - ETA: 33s - loss: 6.03 - ETA: 32s - loss: 6.03 - ETA: 30s - loss: 6.03 - ETA: 29s - loss: 6.03 - ETA: 28s - loss: 6.03 - ETA: 26s - loss: 6.03 - ETA: 25s - loss: 6.03 - ETA: 23s - loss: 6.03 - ETA: 22s - loss: 6.03 - ETA: 21s - loss: 6.03 - ETA: 19s - loss: 6.03 - ETA: 18s - loss: 6.03 - ETA: 16s - loss: 6.02 - ETA: 15s - loss: 6.02 - ETA: 14s - loss: 6.02 - ETA: 12s - loss: 6.02 - ETA: 11s - loss: 6.02 - ETA: 9s - loss: 6.0281 - ETA: 8s - loss: 6.028 - ETA: 7s - loss: 6.027 - ETA: 5s - loss: 6.027 - ETA: 4s - loss: 6.026 - ETA: 2s - loss: 6.025 - ETA: 1s - loss: 6.024 - 281s 70ms/step - loss: 6.0248\n",
      "Epoch 4/10\n",
      "4000/4000 [==============================] - ETA: 4:34 - loss: 5.736 - ETA: 4:35 - loss: 5.729 - ETA: 4:33 - loss: 5.806 - ETA: 4:33 - loss: 5.850 - ETA: 4:33 - loss: 5.854 - ETA: 4:31 - loss: 5.866 - ETA: 4:30 - loss: 5.869 - ETA: 4:29 - loss: 5.862 - ETA: 4:29 - loss: 5.856 - ETA: 4:28 - loss: 5.834 - ETA: 4:26 - loss: 5.843 - ETA: 4:24 - loss: 5.846 - ETA: 4:22 - loss: 5.855 - ETA: 4:21 - loss: 5.860 - ETA: 4:20 - loss: 5.858 - ETA: 4:19 - loss: 5.863 - ETA: 4:18 - loss: 5.862 - ETA: 4:17 - loss: 5.862 - ETA: 4:16 - loss: 5.863 - ETA: 4:14 - loss: 5.865 - ETA: 4:13 - loss: 5.868 - ETA: 4:11 - loss: 5.867 - ETA: 4:10 - loss: 5.863 - ETA: 4:08 - loss: 5.869 - ETA: 4:07 - loss: 5.869 - ETA: 4:06 - loss: 5.869 - ETA: 4:04 - loss: 5.868 - ETA: 4:03 - loss: 5.872 - ETA: 4:01 - loss: 5.874 - ETA: 4:00 - loss: 5.879 - ETA: 3:59 - loss: 5.880 - ETA: 3:57 - loss: 5.881 - ETA: 3:56 - loss: 5.876 - ETA: 3:54 - loss: 5.880 - ETA: 3:53 - loss: 5.880 - ETA: 3:51 - loss: 5.876 - ETA: 3:50 - loss: 5.876 - ETA: 3:48 - loss: 5.875 - ETA: 3:47 - loss: 5.873 - ETA: 3:45 - loss: 5.875 - ETA: 3:44 - loss: 5.877 - ETA: 3:43 - loss: 5.877 - ETA: 3:41 - loss: 5.879 - ETA: 3:40 - loss: 5.880 - ETA: 3:38 - loss: 5.882 - ETA: 3:37 - loss: 5.880 - ETA: 3:35 - loss: 5.879 - ETA: 3:34 - loss: 5.877 - ETA: 3:33 - loss: 5.878 - ETA: 3:31 - loss: 5.875 - ETA: 3:30 - loss: 5.880 - ETA: 3:28 - loss: 5.878 - ETA: 3:27 - loss: 5.879 - ETA: 3:25 - loss: 5.876 - ETA: 3:24 - loss: 5.874 - ETA: 3:22 - loss: 5.874 - ETA: 3:21 - loss: 5.873 - ETA: 3:20 - loss: 5.872 - ETA: 3:18 - loss: 5.870 - ETA: 3:17 - loss: 5.872 - ETA: 3:15 - loss: 5.873 - ETA: 3:14 - loss: 5.869 - ETA: 3:12 - loss: 5.870 - ETA: 3:11 - loss: 5.870 - ETA: 3:09 - loss: 5.869 - ETA: 3:08 - loss: 5.865 - ETA: 3:07 - loss: 5.864 - ETA: 3:06 - loss: 5.863 - ETA: 3:05 - loss: 5.862 - ETA: 3:03 - loss: 5.864 - ETA: 3:02 - loss: 5.863 - ETA: 3:00 - loss: 5.865 - ETA: 2:59 - loss: 5.863 - ETA: 2:58 - loss: 5.861 - ETA: 2:56 - loss: 5.863 - ETA: 2:55 - loss: 5.863 - ETA: 2:53 - loss: 5.861 - ETA: 2:52 - loss: 5.861 - ETA: 2:51 - loss: 5.860 - ETA: 2:49 - loss: 5.859 - ETA: 2:48 - loss: 5.861 - ETA: 2:46 - loss: 5.864 - ETA: 2:45 - loss: 5.864 - ETA: 2:44 - loss: 5.863 - ETA: 2:42 - loss: 5.863 - ETA: 2:41 - loss: 5.862 - ETA: 2:39 - loss: 5.862 - ETA: 2:38 - loss: 5.861 - ETA: 2:37 - loss: 5.860 - ETA: 2:35 - loss: 5.860 - ETA: 2:34 - loss: 5.859 - ETA: 2:32 - loss: 5.859 - ETA: 2:31 - loss: 5.858 - ETA: 2:29 - loss: 5.858 - ETA: 2:28 - loss: 5.856 - ETA: 2:27 - loss: 5.855 - ETA: 2:25 - loss: 5.856 - ETA: 2:24 - loss: 5.855 - ETA: 2:22 - loss: 5.856 - ETA: 2:21 - loss: 5.856 - ETA: 2:19 - loss: 5.854 - ETA: 2:18 - loss: 5.853 - ETA: 2:17 - loss: 5.852 - ETA: 2:15 - loss: 5.853 - ETA: 2:14 - loss: 5.852 - ETA: 2:12 - loss: 5.853 - ETA: 2:11 - loss: 5.853 - ETA: 2:10 - loss: 5.853 - ETA: 2:08 - loss: 5.852 - ETA: 2:07 - loss: 5.852 - ETA: 2:05 - loss: 5.851 - ETA: 2:04 - loss: 5.851 - ETA: 2:02 - loss: 5.852 - ETA: 2:01 - loss: 5.852 - ETA: 2:00 - loss: 5.851 - ETA: 1:58 - loss: 5.850 - ETA: 1:57 - loss: 5.850 - ETA: 1:55 - loss: 5.849 - ETA: 1:54 - loss: 5.848 - ETA: 1:53 - loss: 5.847 - ETA: 1:51 - loss: 5.847 - ETA: 1:50 - loss: 5.847 - ETA: 1:48 - loss: 5.847 - ETA: 1:47 - loss: 5.847 - ETA: 1:45 - loss: 5.847 - ETA: 1:44 - loss: 5.847 - ETA: 1:43 - loss: 5.846 - ETA: 1:41 - loss: 5.846 - ETA: 1:40 - loss: 5.845 - ETA: 1:38 - loss: 5.845 - ETA: 1:37 - loss: 5.845 - ETA: 1:36 - loss: 5.844 - ETA: 1:34 - loss: 5.844 - ETA: 1:33 - loss: 5.843 - ETA: 1:31 - loss: 5.842 - ETA: 1:30 - loss: 5.842 - ETA: 1:29 - loss: 5.841 - ETA: 1:27 - loss: 5.841 - ETA: 1:26 - loss: 5.840 - ETA: 1:24 - loss: 5.839 - ETA: 1:23 - loss: 5.837 - ETA: 1:22 - loss: 5.837 - ETA: 1:20 - loss: 5.836 - ETA: 1:19 - loss: 5.836 - ETA: 1:17 - loss: 5.835 - ETA: 1:16 - loss: 5.834 - ETA: 1:14 - loss: 5.836 - ETA: 1:13 - loss: 5.834 - ETA: 1:12 - loss: 5.834 - ETA: 1:10 - loss: 5.834 - ETA: 1:09 - loss: 5.834 - ETA: 1:07 - loss: 5.833 - ETA: 1:06 - loss: 5.832 - ETA: 1:04 - loss: 5.832 - ETA: 1:03 - loss: 5.831 - ETA: 1:02 - loss: 5.832 - ETA: 1:00 - loss: 5.831 - ETA: 59s - loss: 5.831 - ETA: 57s - loss: 5.83 - ETA: 56s - loss: 5.83 - ETA: 55s - loss: 5.83 - ETA: 53s - loss: 5.83 - ETA: 52s - loss: 5.83 - ETA: 50s - loss: 5.83 - ETA: 49s - loss: 5.83 - ETA: 48s - loss: 5.83 - ETA: 46s - loss: 5.83 - ETA: 45s - loss: 5.83 - ETA: 43s - loss: 5.83 - ETA: 42s - loss: 5.82 - ETA: 40s - loss: 5.82 - ETA: 39s - loss: 5.82 - ETA: 38s - loss: 5.82 - ETA: 36s - loss: 5.82 - ETA: 35s - loss: 5.82 - ETA: 33s - loss: 5.82 - ETA: 32s - loss: 5.82 - ETA: 31s - loss: 5.82 - ETA: 29s - loss: 5.82 - ETA: 28s - loss: 5.82 - ETA: 26s - loss: 5.82 - ETA: 25s - loss: 5.82 - ETA: 24s - loss: 5.82 - ETA: 22s - loss: 5.82 - ETA: 21s - loss: 5.82 - ETA: 19s - loss: 5.82 - ETA: 18s - loss: 5.82 - ETA: 16s - loss: 5.82 - ETA: 15s - loss: 5.82 - ETA: 14s - loss: 5.82 - ETA: 12s - loss: 5.82 - ETA: 11s - loss: 5.82 - ETA: 9s - loss: 5.8233 - ETA: 8s - loss: 5.823 - ETA: 7s - loss: 5.822 - ETA: 5s - loss: 5.822 - ETA: 4s - loss: 5.821 - ETA: 2s - loss: 5.821 - ETA: 1s - loss: 5.821 - 282s 71ms/step - loss: 5.8217\n",
      "Epoch 5/10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4000/4000 [==============================] - ETA: 4:32 - loss: 5.700 - ETA: 4:34 - loss: 5.708 - ETA: 4:34 - loss: 5.707 - ETA: 4:34 - loss: 5.721 - ETA: 4:36 - loss: 5.681 - ETA: 4:36 - loss: 5.680 - ETA: 4:34 - loss: 5.690 - ETA: 4:32 - loss: 5.661 - ETA: 4:30 - loss: 5.663 - ETA: 4:28 - loss: 5.684 - ETA: 4:27 - loss: 5.673 - ETA: 4:26 - loss: 5.671 - ETA: 4:24 - loss: 5.673 - ETA: 4:23 - loss: 5.677 - ETA: 4:21 - loss: 5.670 - ETA: 4:19 - loss: 5.668 - ETA: 4:18 - loss: 5.665 - ETA: 4:17 - loss: 5.662 - ETA: 4:15 - loss: 5.662 - ETA: 4:14 - loss: 5.662 - ETA: 4:13 - loss: 5.663 - ETA: 4:11 - loss: 5.659 - ETA: 4:10 - loss: 5.663 - ETA: 4:09 - loss: 5.662 - ETA: 4:07 - loss: 5.660 - ETA: 4:05 - loss: 5.656 - ETA: 4:04 - loss: 5.652 - ETA: 4:03 - loss: 5.653 - ETA: 4:01 - loss: 5.651 - ETA: 3:59 - loss: 5.647 - ETA: 3:58 - loss: 5.646 - ETA: 3:56 - loss: 5.646 - ETA: 3:55 - loss: 5.641 - ETA: 3:54 - loss: 5.639 - ETA: 3:52 - loss: 5.642 - ETA: 3:51 - loss: 5.640 - ETA: 3:49 - loss: 5.643 - ETA: 3:48 - loss: 5.644 - ETA: 3:47 - loss: 5.643 - ETA: 3:46 - loss: 5.640 - ETA: 3:44 - loss: 5.639 - ETA: 3:43 - loss: 5.639 - ETA: 3:41 - loss: 5.641 - ETA: 3:40 - loss: 5.643 - ETA: 3:38 - loss: 5.641 - ETA: 3:37 - loss: 5.638 - ETA: 3:36 - loss: 5.639 - ETA: 3:34 - loss: 5.638 - ETA: 3:33 - loss: 5.635 - ETA: 3:31 - loss: 5.635 - ETA: 3:30 - loss: 5.634 - ETA: 3:29 - loss: 5.638 - ETA: 3:27 - loss: 5.635 - ETA: 3:26 - loss: 5.634 - ETA: 3:24 - loss: 5.635 - ETA: 3:23 - loss: 5.631 - ETA: 3:21 - loss: 5.637 - ETA: 3:20 - loss: 5.640 - ETA: 3:19 - loss: 5.638 - ETA: 3:17 - loss: 5.640 - ETA: 3:16 - loss: 5.639 - ETA: 3:14 - loss: 5.638 - ETA: 3:13 - loss: 5.640 - ETA: 3:12 - loss: 5.640 - ETA: 3:10 - loss: 5.638 - ETA: 3:09 - loss: 5.640 - ETA: 3:07 - loss: 5.639 - ETA: 3:06 - loss: 5.637 - ETA: 3:04 - loss: 5.637 - ETA: 3:03 - loss: 5.636 - ETA: 3:02 - loss: 5.632 - ETA: 3:00 - loss: 5.634 - ETA: 2:59 - loss: 5.635 - ETA: 2:57 - loss: 5.637 - ETA: 2:56 - loss: 5.638 - ETA: 2:55 - loss: 5.637 - ETA: 2:53 - loss: 5.636 - ETA: 2:52 - loss: 5.636 - ETA: 2:50 - loss: 5.636 - ETA: 2:49 - loss: 5.635 - ETA: 2:47 - loss: 5.636 - ETA: 2:46 - loss: 5.633 - ETA: 2:45 - loss: 5.632 - ETA: 2:43 - loss: 5.633 - ETA: 2:42 - loss: 5.633 - ETA: 2:40 - loss: 5.632 - ETA: 2:39 - loss: 5.631 - ETA: 2:38 - loss: 5.632 - ETA: 2:36 - loss: 5.634 - ETA: 2:35 - loss: 5.635 - ETA: 2:33 - loss: 5.635 - ETA: 2:32 - loss: 5.634 - ETA: 2:30 - loss: 5.634 - ETA: 2:29 - loss: 5.634 - ETA: 2:28 - loss: 5.634 - ETA: 2:26 - loss: 5.634 - ETA: 2:25 - loss: 5.634 - ETA: 2:23 - loss: 5.634 - ETA: 2:22 - loss: 5.635 - ETA: 2:21 - loss: 5.634 - ETA: 2:19 - loss: 5.634 - ETA: 2:18 - loss: 5.636 - ETA: 2:16 - loss: 5.636 - ETA: 2:15 - loss: 5.636 - ETA: 2:13 - loss: 5.636 - ETA: 2:12 - loss: 5.637 - ETA: 2:11 - loss: 5.637 - ETA: 2:09 - loss: 5.637 - ETA: 2:08 - loss: 5.636 - ETA: 2:06 - loss: 5.635 - ETA: 2:05 - loss: 5.636 - ETA: 2:04 - loss: 5.636 - ETA: 2:02 - loss: 5.637 - ETA: 2:01 - loss: 5.637 - ETA: 1:59 - loss: 5.636 - ETA: 1:58 - loss: 5.635 - ETA: 1:56 - loss: 5.634 - ETA: 1:55 - loss: 5.636 - ETA: 1:54 - loss: 5.635 - ETA: 1:52 - loss: 5.635 - ETA: 1:51 - loss: 5.634 - ETA: 1:49 - loss: 5.634 - ETA: 1:48 - loss: 5.634 - ETA: 1:47 - loss: 5.634 - ETA: 1:45 - loss: 5.633 - ETA: 1:44 - loss: 5.633 - ETA: 1:42 - loss: 5.632 - ETA: 1:41 - loss: 5.631 - ETA: 1:40 - loss: 5.632 - ETA: 1:38 - loss: 5.631 - ETA: 1:37 - loss: 5.630 - ETA: 1:35 - loss: 5.630 - ETA: 1:34 - loss: 5.628 - ETA: 1:33 - loss: 5.629 - ETA: 1:31 - loss: 5.629 - ETA: 1:30 - loss: 5.629 - ETA: 1:28 - loss: 5.628 - ETA: 1:27 - loss: 5.628 - ETA: 1:26 - loss: 5.628 - ETA: 1:24 - loss: 5.628 - ETA: 1:23 - loss: 5.627 - ETA: 1:21 - loss: 5.627 - ETA: 1:20 - loss: 5.627 - ETA: 1:18 - loss: 5.626 - ETA: 1:17 - loss: 5.626 - ETA: 1:16 - loss: 5.626 - ETA: 1:14 - loss: 5.626 - ETA: 1:13 - loss: 5.626 - ETA: 1:11 - loss: 5.626 - ETA: 1:10 - loss: 5.626 - ETA: 1:09 - loss: 5.626 - ETA: 1:07 - loss: 5.625 - ETA: 1:06 - loss: 5.624 - ETA: 1:04 - loss: 5.624 - ETA: 1:03 - loss: 5.624 - ETA: 1:01 - loss: 5.623 - ETA: 1:00 - loss: 5.624 - ETA: 59s - loss: 5.622 - ETA: 57s - loss: 5.62 - ETA: 56s - loss: 5.62 - ETA: 55s - loss: 5.61 - ETA: 53s - loss: 5.61 - ETA: 52s - loss: 5.61 - ETA: 50s - loss: 5.62 - ETA: 49s - loss: 5.62 - ETA: 48s - loss: 5.62 - ETA: 46s - loss: 5.61 - ETA: 45s - loss: 5.61 - ETA: 43s - loss: 5.61 - ETA: 42s - loss: 5.61 - ETA: 40s - loss: 5.61 - ETA: 39s - loss: 5.61 - ETA: 38s - loss: 5.61 - ETA: 36s - loss: 5.61 - ETA: 35s - loss: 5.61 - ETA: 33s - loss: 5.61 - ETA: 32s - loss: 5.61 - ETA: 31s - loss: 5.61 - ETA: 29s - loss: 5.61 - ETA: 28s - loss: 5.61 - ETA: 26s - loss: 5.61 - ETA: 25s - loss: 5.61 - ETA: 24s - loss: 5.61 - ETA: 22s - loss: 5.61 - ETA: 21s - loss: 5.61 - ETA: 19s - loss: 5.61 - ETA: 18s - loss: 5.61 - ETA: 16s - loss: 5.61 - ETA: 15s - loss: 5.60 - ETA: 14s - loss: 5.60 - ETA: 12s - loss: 5.60 - ETA: 11s - loss: 5.60 - ETA: 9s - loss: 5.6060 - ETA: 8s - loss: 5.605 - ETA: 7s - loss: 5.604 - ETA: 5s - loss: 5.603 - ETA: 4s - loss: 5.603 - ETA: 2s - loss: 5.602 - ETA: 1s - loss: 5.602 - 283s 71ms/step - loss: 5.6011\n",
      "Epoch 6/10\n",
      "4000/4000 [==============================] - ETA: 4:43 - loss: 5.208 - ETA: 4:36 - loss: 5.305 - ETA: 4:34 - loss: 5.345 - ETA: 4:34 - loss: 5.341 - ETA: 4:33 - loss: 5.344 - ETA: 4:31 - loss: 5.356 - ETA: 4:30 - loss: 5.360 - ETA: 4:29 - loss: 5.356 - ETA: 4:27 - loss: 5.347 - ETA: 4:26 - loss: 5.338 - ETA: 4:24 - loss: 5.343 - ETA: 4:23 - loss: 5.352 - ETA: 4:23 - loss: 5.359 - ETA: 4:22 - loss: 5.368 - ETA: 4:21 - loss: 5.368 - ETA: 4:19 - loss: 5.368 - ETA: 4:17 - loss: 5.369 - ETA: 4:16 - loss: 5.367 - ETA: 4:14 - loss: 5.372 - ETA: 4:13 - loss: 5.370 - ETA: 4:11 - loss: 5.366 - ETA: 4:10 - loss: 5.368 - ETA: 4:09 - loss: 5.371 - ETA: 4:07 - loss: 5.374 - ETA: 4:06 - loss: 5.378 - ETA: 4:05 - loss: 5.384 - ETA: 4:03 - loss: 5.386 - ETA: 4:02 - loss: 5.382 - ETA: 4:00 - loss: 5.383 - ETA: 3:59 - loss: 5.391 - ETA: 3:57 - loss: 5.390 - ETA: 3:56 - loss: 5.391 - ETA: 3:55 - loss: 5.390 - ETA: 3:54 - loss: 5.385 - ETA: 3:52 - loss: 5.386 - ETA: 3:51 - loss: 5.386 - ETA: 3:49 - loss: 5.384 - ETA: 3:48 - loss: 5.386 - ETA: 3:47 - loss: 5.386 - ETA: 3:45 - loss: 5.386 - ETA: 3:44 - loss: 5.386 - ETA: 3:42 - loss: 5.386 - ETA: 3:41 - loss: 5.382 - ETA: 3:40 - loss: 5.381 - ETA: 3:38 - loss: 5.380 - ETA: 3:37 - loss: 5.382 - ETA: 3:36 - loss: 5.383 - ETA: 3:34 - loss: 5.383 - ETA: 3:33 - loss: 5.380 - ETA: 3:31 - loss: 5.378 - ETA: 3:30 - loss: 5.375 - ETA: 3:29 - loss: 5.375 - ETA: 3:27 - loss: 5.377 - ETA: 3:26 - loss: 5.378 - ETA: 3:24 - loss: 5.376 - ETA: 3:23 - loss: 5.376 - ETA: 3:21 - loss: 5.376 - ETA: 3:20 - loss: 5.377 - ETA: 3:18 - loss: 5.378 - ETA: 3:17 - loss: 5.376 - ETA: 3:16 - loss: 5.379 - ETA: 3:14 - loss: 5.379 - ETA: 3:13 - loss: 5.377 - ETA: 3:11 - loss: 5.376 - ETA: 3:10 - loss: 5.378 - ETA: 3:08 - loss: 5.375 - ETA: 3:07 - loss: 5.374 - ETA: 3:05 - loss: 5.372 - ETA: 3:04 - loss: 5.372 - ETA: 3:03 - loss: 5.373 - ETA: 3:01 - loss: 5.373 - ETA: 3:00 - loss: 5.373 - ETA: 2:58 - loss: 5.372 - ETA: 2:57 - loss: 5.372 - ETA: 2:56 - loss: 5.373 - ETA: 2:54 - loss: 5.373 - ETA: 2:53 - loss: 5.373 - ETA: 2:51 - loss: 5.371 - ETA: 2:50 - loss: 5.369 - ETA: 2:49 - loss: 5.371 - ETA: 2:47 - loss: 5.371 - ETA: 2:46 - loss: 5.370 - ETA: 2:45 - loss: 5.371 - ETA: 2:43 - loss: 5.370 - ETA: 2:42 - loss: 5.370 - ETA: 2:40 - loss: 5.369 - ETA: 2:39 - loss: 5.368 - ETA: 2:37 - loss: 5.370 - ETA: 2:36 - loss: 5.368 - ETA: 2:35 - loss: 5.367 - ETA: 2:33 - loss: 5.367 - ETA: 2:32 - loss: 5.371 - ETA: 2:30 - loss: 5.370 - ETA: 2:29 - loss: 5.370 - ETA: 2:28 - loss: 5.368 - ETA: 2:26 - loss: 5.370 - ETA: 2:25 - loss: 5.371 - ETA: 2:23 - loss: 5.372 - ETA: 2:22 - loss: 5.371 - ETA: 2:21 - loss: 5.370 - ETA: 2:19 - loss: 5.370 - ETA: 2:18 - loss: 5.369 - ETA: 2:16 - loss: 5.369 - ETA: 2:15 - loss: 5.370 - ETA: 2:13 - loss: 5.371 - ETA: 2:12 - loss: 5.371 - ETA: 2:11 - loss: 5.370 - ETA: 2:09 - loss: 5.371 - ETA: 2:08 - loss: 5.371 - ETA: 2:06 - loss: 5.370 - ETA: 2:05 - loss: 5.369 - ETA: 2:04 - loss: 5.370 - ETA: 2:02 - loss: 5.371 - ETA: 2:01 - loss: 5.370 - ETA: 1:59 - loss: 5.371 - ETA: 1:58 - loss: 5.371 - ETA: 1:57 - loss: 5.372 - ETA: 1:55 - loss: 5.371 - ETA: 1:54 - loss: 5.372 - ETA: 1:52 - loss: 5.372 - ETA: 1:51 - loss: 5.370 - ETA: 1:49 - loss: 5.369 - ETA: 1:48 - loss: 5.369 - ETA: 1:47 - loss: 5.370 - ETA: 1:45 - loss: 5.369 - ETA: 1:44 - loss: 5.369 - ETA: 1:42 - loss: 5.368 - ETA: 1:41 - loss: 5.368 - ETA: 1:40 - loss: 5.368 - ETA: 1:38 - loss: 5.369 - ETA: 1:37 - loss: 5.367 - ETA: 1:35 - loss: 5.367 - ETA: 1:34 - loss: 5.366 - ETA: 1:33 - loss: 5.368 - ETA: 1:31 - loss: 5.366 - ETA: 1:30 - loss: 5.366 - ETA: 1:28 - loss: 5.365 - ETA: 1:27 - loss: 5.364 - ETA: 1:25 - loss: 5.363 - ETA: 1:24 - loss: 5.364 - ETA: 1:23 - loss: 5.365 - ETA: 1:21 - loss: 5.366 - ETA: 1:20 - loss: 5.367 - ETA: 1:18 - loss: 5.366 - ETA: 1:17 - loss: 5.367 - ETA: 1:16 - loss: 5.366 - ETA: 1:14 - loss: 5.366 - ETA: 1:13 - loss: 5.365 - ETA: 1:11 - loss: 5.366 - ETA: 1:10 - loss: 5.366 - ETA: 1:09 - loss: 5.365 - ETA: 1:07 - loss: 5.364 - ETA: 1:06 - loss: 5.364 - ETA: 1:04 - loss: 5.363 - ETA: 1:03 - loss: 5.362 - ETA: 1:02 - loss: 5.362 - ETA: 1:00 - loss: 5.362 - ETA: 59s - loss: 5.360 - ETA: 57s - loss: 5.36 - ETA: 56s - loss: 5.36 - ETA: 54s - loss: 5.36 - ETA: 53s - loss: 5.36 - ETA: 52s - loss: 5.36 - ETA: 50s - loss: 5.35 - ETA: 49s - loss: 5.35 - ETA: 47s - loss: 5.35 - ETA: 46s - loss: 5.35 - ETA: 45s - loss: 5.35 - ETA: 43s - loss: 5.35 - ETA: 42s - loss: 5.35 - ETA: 40s - loss: 5.35 - ETA: 39s - loss: 5.35 - ETA: 38s - loss: 5.35 - ETA: 36s - loss: 5.35 - ETA: 35s - loss: 5.35 - ETA: 33s - loss: 5.35 - ETA: 32s - loss: 5.35 - ETA: 30s - loss: 5.35 - ETA: 29s - loss: 5.35 - ETA: 28s - loss: 5.35 - ETA: 26s - loss: 5.35 - ETA: 25s - loss: 5.35 - ETA: 23s - loss: 5.35 - ETA: 22s - loss: 5.35 - ETA: 21s - loss: 5.35 - ETA: 19s - loss: 5.35 - ETA: 18s - loss: 5.35 - ETA: 16s - loss: 5.35 - ETA: 15s - loss: 5.35 - ETA: 14s - loss: 5.35 - ETA: 12s - loss: 5.35 - ETA: 11s - loss: 5.34 - ETA: 9s - loss: 5.3488 - ETA: 8s - loss: 5.347 - ETA: 7s - loss: 5.348 - ETA: 5s - loss: 5.348 - ETA: 4s - loss: 5.348 - ETA: 2s - loss: 5.349 - ETA: 1s - loss: 5.348 - 282s 70ms/step - loss: 5.3491\n",
      "Epoch 7/10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4000/4000 [==============================] - ETA: 4:38 - loss: 5.347 - ETA: 4:34 - loss: 5.259 - ETA: 4:33 - loss: 5.159 - ETA: 4:32 - loss: 5.155 - ETA: 4:32 - loss: 5.140 - ETA: 4:34 - loss: 5.138 - ETA: 4:33 - loss: 5.145 - ETA: 4:31 - loss: 5.147 - ETA: 4:29 - loss: 5.153 - ETA: 4:28 - loss: 5.160 - ETA: 4:26 - loss: 5.170 - ETA: 4:25 - loss: 5.178 - ETA: 4:23 - loss: 5.190 - ETA: 4:22 - loss: 5.182 - ETA: 4:20 - loss: 5.191 - ETA: 4:18 - loss: 5.178 - ETA: 4:17 - loss: 5.183 - ETA: 4:15 - loss: 5.185 - ETA: 4:14 - loss: 5.188 - ETA: 4:13 - loss: 5.189 - ETA: 4:11 - loss: 5.185 - ETA: 4:11 - loss: 5.184 - ETA: 4:09 - loss: 5.180 - ETA: 4:07 - loss: 5.183 - ETA: 4:06 - loss: 5.182 - ETA: 4:05 - loss: 5.174 - ETA: 4:03 - loss: 5.169 - ETA: 4:02 - loss: 5.175 - ETA: 4:00 - loss: 5.172 - ETA: 3:59 - loss: 5.173 - ETA: 3:58 - loss: 5.172 - ETA: 3:56 - loss: 5.172 - ETA: 3:55 - loss: 5.172 - ETA: 3:54 - loss: 5.173 - ETA: 3:52 - loss: 5.172 - ETA: 3:51 - loss: 5.171 - ETA: 3:49 - loss: 5.170 - ETA: 3:48 - loss: 5.166 - ETA: 3:47 - loss: 5.164 - ETA: 3:45 - loss: 5.166 - ETA: 3:44 - loss: 5.166 - ETA: 3:43 - loss: 5.166 - ETA: 3:41 - loss: 5.166 - ETA: 3:40 - loss: 5.165 - ETA: 3:38 - loss: 5.163 - ETA: 3:37 - loss: 5.162 - ETA: 3:35 - loss: 5.165 - ETA: 3:34 - loss: 5.165 - ETA: 3:32 - loss: 5.164 - ETA: 3:31 - loss: 5.165 - ETA: 3:29 - loss: 5.162 - ETA: 3:28 - loss: 5.162 - ETA: 3:27 - loss: 5.163 - ETA: 3:25 - loss: 5.160 - ETA: 3:24 - loss: 5.160 - ETA: 3:22 - loss: 5.158 - ETA: 3:21 - loss: 5.158 - ETA: 3:20 - loss: 5.160 - ETA: 3:18 - loss: 5.159 - ETA: 3:17 - loss: 5.160 - ETA: 3:15 - loss: 5.160 - ETA: 3:14 - loss: 5.158 - ETA: 3:12 - loss: 5.158 - ETA: 3:11 - loss: 5.158 - ETA: 3:10 - loss: 5.159 - ETA: 3:08 - loss: 5.163 - ETA: 3:07 - loss: 5.163 - ETA: 3:05 - loss: 5.162 - ETA: 3:04 - loss: 5.162 - ETA: 3:03 - loss: 5.161 - ETA: 3:01 - loss: 5.162 - ETA: 3:00 - loss: 5.160 - ETA: 2:58 - loss: 5.159 - ETA: 2:57 - loss: 5.159 - ETA: 2:56 - loss: 5.157 - ETA: 2:54 - loss: 5.156 - ETA: 2:53 - loss: 5.158 - ETA: 2:51 - loss: 5.159 - ETA: 2:50 - loss: 5.160 - ETA: 2:48 - loss: 5.160 - ETA: 2:47 - loss: 5.159 - ETA: 2:46 - loss: 5.158 - ETA: 2:44 - loss: 5.159 - ETA: 2:43 - loss: 5.159 - ETA: 2:41 - loss: 5.160 - ETA: 2:40 - loss: 5.161 - ETA: 2:39 - loss: 5.159 - ETA: 2:37 - loss: 5.158 - ETA: 2:36 - loss: 5.157 - ETA: 2:34 - loss: 5.156 - ETA: 2:33 - loss: 5.157 - ETA: 2:31 - loss: 5.155 - ETA: 2:30 - loss: 5.156 - ETA: 2:29 - loss: 5.154 - ETA: 2:27 - loss: 5.153 - ETA: 2:26 - loss: 5.152 - ETA: 2:25 - loss: 5.154 - ETA: 2:23 - loss: 5.154 - ETA: 2:22 - loss: 5.156 - ETA: 2:20 - loss: 5.157 - ETA: 2:19 - loss: 5.154 - ETA: 2:17 - loss: 5.154 - ETA: 2:16 - loss: 5.154 - ETA: 2:15 - loss: 5.153 - ETA: 2:13 - loss: 5.150 - ETA: 2:12 - loss: 5.151 - ETA: 2:10 - loss: 5.151 - ETA: 2:09 - loss: 5.152 - ETA: 2:08 - loss: 5.152 - ETA: 2:06 - loss: 5.150 - ETA: 2:05 - loss: 5.150 - ETA: 2:03 - loss: 5.150 - ETA: 2:02 - loss: 5.149 - ETA: 2:01 - loss: 5.148 - ETA: 1:59 - loss: 5.149 - ETA: 1:58 - loss: 5.150 - ETA: 1:56 - loss: 5.150 - ETA: 1:55 - loss: 5.150 - ETA: 1:54 - loss: 5.151 - ETA: 1:52 - loss: 5.151 - ETA: 1:51 - loss: 5.152 - ETA: 1:49 - loss: 5.152 - ETA: 1:48 - loss: 5.153 - ETA: 1:47 - loss: 5.153 - ETA: 1:45 - loss: 5.154 - ETA: 1:44 - loss: 5.153 - ETA: 1:43 - loss: 5.153 - ETA: 1:41 - loss: 5.153 - ETA: 1:40 - loss: 5.152 - ETA: 1:38 - loss: 5.152 - ETA: 1:37 - loss: 5.151 - ETA: 1:35 - loss: 5.153 - ETA: 1:34 - loss: 5.153 - ETA: 1:33 - loss: 5.153 - ETA: 1:31 - loss: 5.153 - ETA: 1:30 - loss: 5.153 - ETA: 1:28 - loss: 5.153 - ETA: 1:27 - loss: 5.152 - ETA: 1:26 - loss: 5.153 - ETA: 1:24 - loss: 5.152 - ETA: 1:23 - loss: 5.152 - ETA: 1:21 - loss: 5.150 - ETA: 1:20 - loss: 5.150 - ETA: 1:19 - loss: 5.150 - ETA: 1:17 - loss: 5.151 - ETA: 1:16 - loss: 5.151 - ETA: 1:14 - loss: 5.151 - ETA: 1:13 - loss: 5.151 - ETA: 1:11 - loss: 5.152 - ETA: 1:10 - loss: 5.152 - ETA: 1:09 - loss: 5.151 - ETA: 1:07 - loss: 5.151 - ETA: 1:06 - loss: 5.151 - ETA: 1:04 - loss: 5.151 - ETA: 1:03 - loss: 5.152 - ETA: 1:02 - loss: 5.152 - ETA: 1:00 - loss: 5.152 - ETA: 59s - loss: 5.152 - ETA: 57s - loss: 5.15 - ETA: 56s - loss: 5.15 - ETA: 55s - loss: 5.15 - ETA: 53s - loss: 5.15 - ETA: 52s - loss: 5.15 - ETA: 50s - loss: 5.15 - ETA: 49s - loss: 5.15 - ETA: 47s - loss: 5.15 - ETA: 46s - loss: 5.15 - ETA: 45s - loss: 5.15 - ETA: 43s - loss: 5.15 - ETA: 42s - loss: 5.15 - ETA: 40s - loss: 5.15 - ETA: 39s - loss: 5.15 - ETA: 38s - loss: 5.15 - ETA: 36s - loss: 5.15 - ETA: 35s - loss: 5.15 - ETA: 33s - loss: 5.15 - ETA: 32s - loss: 5.15 - ETA: 31s - loss: 5.15 - ETA: 29s - loss: 5.15 - ETA: 28s - loss: 5.15 - ETA: 26s - loss: 5.15 - ETA: 25s - loss: 5.15 - ETA: 23s - loss: 5.14 - ETA: 22s - loss: 5.14 - ETA: 21s - loss: 5.14 - ETA: 19s - loss: 5.14 - ETA: 18s - loss: 5.15 - ETA: 16s - loss: 5.14 - ETA: 15s - loss: 5.14 - ETA: 14s - loss: 5.14 - ETA: 12s - loss: 5.14 - ETA: 11s - loss: 5.14 - ETA: 9s - loss: 5.1476 - ETA: 8s - loss: 5.147 - ETA: 7s - loss: 5.147 - ETA: 5s - loss: 5.146 - ETA: 4s - loss: 5.147 - ETA: 2s - loss: 5.147 - ETA: 1s - loss: 5.146 - 282s 71ms/step - loss: 5.1463\n",
      "Epoch 8/10\n",
      "4000/4000 [==============================] - ETA: 4:34 - loss: 5.148 - ETA: 4:35 - loss: 5.019 - ETA: 4:34 - loss: 5.046 - ETA: 4:34 - loss: 5.006 - ETA: 4:33 - loss: 4.982 - ETA: 4:31 - loss: 4.986 - ETA: 4:31 - loss: 4.978 - ETA: 4:31 - loss: 4.960 - ETA: 4:30 - loss: 4.965 - ETA: 4:28 - loss: 4.969 - ETA: 4:27 - loss: 4.960 - ETA: 4:26 - loss: 4.969 - ETA: 4:24 - loss: 4.984 - ETA: 4:23 - loss: 4.992 - ETA: 4:21 - loss: 5.001 - ETA: 4:20 - loss: 4.998 - ETA: 4:18 - loss: 5.005 - ETA: 4:17 - loss: 5.003 - ETA: 4:16 - loss: 5.005 - ETA: 4:14 - loss: 5.000 - ETA: 4:13 - loss: 5.005 - ETA: 4:11 - loss: 4.997 - ETA: 4:10 - loss: 4.994 - ETA: 4:09 - loss: 4.990 - ETA: 4:07 - loss: 4.983 - ETA: 4:05 - loss: 4.987 - ETA: 4:04 - loss: 4.993 - ETA: 4:03 - loss: 4.987 - ETA: 4:01 - loss: 4.981 - ETA: 4:00 - loss: 4.978 - ETA: 3:59 - loss: 4.978 - ETA: 3:57 - loss: 4.979 - ETA: 3:56 - loss: 4.977 - ETA: 3:55 - loss: 4.976 - ETA: 3:53 - loss: 4.971 - ETA: 3:52 - loss: 4.963 - ETA: 3:50 - loss: 4.963 - ETA: 3:49 - loss: 4.960 - ETA: 3:47 - loss: 4.963 - ETA: 3:46 - loss: 4.955 - ETA: 3:45 - loss: 4.954 - ETA: 3:43 - loss: 4.955 - ETA: 3:42 - loss: 4.962 - ETA: 3:41 - loss: 4.957 - ETA: 3:40 - loss: 4.960 - ETA: 3:38 - loss: 4.962 - ETA: 3:37 - loss: 4.963 - ETA: 3:35 - loss: 4.962 - ETA: 3:34 - loss: 4.964 - ETA: 3:32 - loss: 4.962 - ETA: 3:31 - loss: 4.966 - ETA: 3:30 - loss: 4.966 - ETA: 3:28 - loss: 4.966 - ETA: 3:27 - loss: 4.966 - ETA: 3:25 - loss: 4.967 - ETA: 3:24 - loss: 4.966 - ETA: 3:23 - loss: 4.967 - ETA: 3:21 - loss: 4.968 - ETA: 3:20 - loss: 4.967 - ETA: 3:18 - loss: 4.966 - ETA: 3:17 - loss: 4.969 - ETA: 3:16 - loss: 4.968 - ETA: 3:14 - loss: 4.969 - ETA: 3:13 - loss: 4.969 - ETA: 3:11 - loss: 4.968 - ETA: 3:10 - loss: 4.969 - ETA: 3:08 - loss: 4.971 - ETA: 3:07 - loss: 4.969 - ETA: 3:06 - loss: 4.968 - ETA: 3:04 - loss: 4.968 - ETA: 3:03 - loss: 4.969 - ETA: 3:01 - loss: 4.968 - ETA: 3:00 - loss: 4.966 - ETA: 2:58 - loss: 4.967 - ETA: 2:57 - loss: 4.966 - ETA: 2:55 - loss: 4.966 - ETA: 2:54 - loss: 4.965 - ETA: 2:52 - loss: 4.964 - ETA: 2:51 - loss: 4.966 - ETA: 2:50 - loss: 4.967 - ETA: 2:48 - loss: 4.968 - ETA: 2:47 - loss: 4.971 - ETA: 2:45 - loss: 4.969 - ETA: 2:44 - loss: 4.969 - ETA: 2:42 - loss: 4.968 - ETA: 2:41 - loss: 4.970 - ETA: 2:40 - loss: 4.971 - ETA: 2:38 - loss: 4.970 - ETA: 2:37 - loss: 4.970 - ETA: 2:35 - loss: 4.969 - ETA: 2:34 - loss: 4.970 - ETA: 2:32 - loss: 4.971 - ETA: 2:31 - loss: 4.972 - ETA: 2:29 - loss: 4.971 - ETA: 2:28 - loss: 4.969 - ETA: 2:27 - loss: 4.969 - ETA: 2:25 - loss: 4.969 - ETA: 2:24 - loss: 4.968 - ETA: 2:22 - loss: 4.968 - ETA: 2:21 - loss: 4.967 - ETA: 2:20 - loss: 4.966 - ETA: 2:18 - loss: 4.966 - ETA: 2:17 - loss: 4.966 - ETA: 2:15 - loss: 4.965 - ETA: 2:14 - loss: 4.966 - ETA: 2:12 - loss: 4.968 - ETA: 2:11 - loss: 4.969 - ETA: 2:09 - loss: 4.970 - ETA: 2:08 - loss: 4.969 - ETA: 2:07 - loss: 4.967 - ETA: 2:05 - loss: 4.967 - ETA: 2:04 - loss: 4.968 - ETA: 2:02 - loss: 4.968 - ETA: 2:01 - loss: 4.968 - ETA: 2:00 - loss: 4.967 - ETA: 1:58 - loss: 4.967 - ETA: 1:57 - loss: 4.966 - ETA: 1:55 - loss: 4.967 - ETA: 1:54 - loss: 4.966 - ETA: 1:53 - loss: 4.966 - ETA: 1:51 - loss: 4.967 - ETA: 1:50 - loss: 4.967 - ETA: 1:48 - loss: 4.967 - ETA: 1:47 - loss: 4.968 - ETA: 1:45 - loss: 4.968 - ETA: 1:44 - loss: 4.968 - ETA: 1:43 - loss: 4.967 - ETA: 1:41 - loss: 4.967 - ETA: 1:40 - loss: 4.966 - ETA: 1:38 - loss: 4.965 - ETA: 1:37 - loss: 4.965 - ETA: 1:36 - loss: 4.965 - ETA: 1:34 - loss: 4.965 - ETA: 1:33 - loss: 4.965 - ETA: 1:31 - loss: 4.965 - ETA: 1:30 - loss: 4.965 - ETA: 1:28 - loss: 4.964 - ETA: 1:27 - loss: 4.964 - ETA: 1:26 - loss: 4.964 - ETA: 1:24 - loss: 4.965 - ETA: 1:23 - loss: 4.965 - ETA: 1:21 - loss: 4.964 - ETA: 1:20 - loss: 4.964 - ETA: 1:19 - loss: 4.964 - ETA: 1:17 - loss: 4.965 - ETA: 1:16 - loss: 4.964 - ETA: 1:14 - loss: 4.964 - ETA: 1:13 - loss: 4.965 - ETA: 1:12 - loss: 4.964 - ETA: 1:10 - loss: 4.964 - ETA: 1:09 - loss: 4.963 - ETA: 1:07 - loss: 4.963 - ETA: 1:06 - loss: 4.963 - ETA: 1:04 - loss: 4.963 - ETA: 1:03 - loss: 4.964 - ETA: 1:02 - loss: 4.963 - ETA: 1:00 - loss: 4.963 - ETA: 59s - loss: 4.962 - ETA: 57s - loss: 4.96 - ETA: 56s - loss: 4.96 - ETA: 55s - loss: 4.96 - ETA: 53s - loss: 4.96 - ETA: 52s - loss: 4.96 - ETA: 50s - loss: 4.96 - ETA: 49s - loss: 4.96 - ETA: 48s - loss: 4.96 - ETA: 46s - loss: 4.95 - ETA: 45s - loss: 4.96 - ETA: 43s - loss: 4.96 - ETA: 42s - loss: 4.95 - ETA: 40s - loss: 4.96 - ETA: 39s - loss: 4.96 - ETA: 38s - loss: 4.96 - ETA: 36s - loss: 4.95 - ETA: 35s - loss: 4.96 - ETA: 33s - loss: 4.95 - ETA: 32s - loss: 4.95 - ETA: 31s - loss: 4.95 - ETA: 29s - loss: 4.95 - ETA: 28s - loss: 4.96 - ETA: 26s - loss: 4.95 - ETA: 25s - loss: 4.96 - ETA: 24s - loss: 4.96 - ETA: 22s - loss: 4.96 - ETA: 21s - loss: 4.96 - ETA: 19s - loss: 4.96 - ETA: 18s - loss: 4.96 - ETA: 16s - loss: 4.96 - ETA: 15s - loss: 4.96 - ETA: 14s - loss: 4.96 - ETA: 12s - loss: 4.96 - ETA: 11s - loss: 4.96 - ETA: 9s - loss: 4.9614 - ETA: 8s - loss: 4.960 - ETA: 7s - loss: 4.960 - ETA: 5s - loss: 4.960 - ETA: 4s - loss: 4.961 - ETA: 2s - loss: 4.961 - ETA: 1s - loss: 4.961 - 282s 71ms/step - loss: 4.9615\n",
      "Epoch 9/10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4000/4000 [==============================] - ETA: 4:39 - loss: 4.682 - ETA: 4:38 - loss: 4.690 - ETA: 4:35 - loss: 4.687 - ETA: 4:33 - loss: 4.696 - ETA: 4:32 - loss: 4.727 - ETA: 4:31 - loss: 4.744 - ETA: 4:31 - loss: 4.735 - ETA: 4:29 - loss: 4.723 - ETA: 4:27 - loss: 4.725 - ETA: 4:25 - loss: 4.737 - ETA: 4:24 - loss: 4.731 - ETA: 4:22 - loss: 4.742 - ETA: 4:21 - loss: 4.735 - ETA: 4:20 - loss: 4.742 - ETA: 4:19 - loss: 4.738 - ETA: 4:17 - loss: 4.741 - ETA: 4:16 - loss: 4.747 - ETA: 4:14 - loss: 4.744 - ETA: 4:13 - loss: 4.746 - ETA: 4:12 - loss: 4.742 - ETA: 4:10 - loss: 4.744 - ETA: 4:09 - loss: 4.739 - ETA: 4:08 - loss: 4.734 - ETA: 4:07 - loss: 4.742 - ETA: 4:06 - loss: 4.742 - ETA: 4:05 - loss: 4.737 - ETA: 4:03 - loss: 4.743 - ETA: 4:02 - loss: 4.745 - ETA: 4:00 - loss: 4.750 - ETA: 3:59 - loss: 4.752 - ETA: 3:57 - loss: 4.753 - ETA: 3:56 - loss: 4.752 - ETA: 3:54 - loss: 4.752 - ETA: 3:53 - loss: 4.753 - ETA: 3:52 - loss: 4.751 - ETA: 3:50 - loss: 4.746 - ETA: 3:49 - loss: 4.746 - ETA: 3:47 - loss: 4.744 - ETA: 3:46 - loss: 4.748 - ETA: 3:44 - loss: 4.751 - ETA: 3:43 - loss: 4.751 - ETA: 3:42 - loss: 4.746 - ETA: 3:40 - loss: 4.745 - ETA: 3:39 - loss: 4.746 - ETA: 3:38 - loss: 4.749 - ETA: 3:36 - loss: 4.750 - ETA: 3:35 - loss: 4.752 - ETA: 3:33 - loss: 4.753 - ETA: 3:32 - loss: 4.756 - ETA: 3:31 - loss: 4.756 - ETA: 3:29 - loss: 4.756 - ETA: 3:28 - loss: 4.760 - ETA: 3:26 - loss: 4.764 - ETA: 3:25 - loss: 4.761 - ETA: 3:24 - loss: 4.761 - ETA: 3:22 - loss: 4.759 - ETA: 3:21 - loss: 4.759 - ETA: 3:19 - loss: 4.758 - ETA: 3:18 - loss: 4.760 - ETA: 3:16 - loss: 4.761 - ETA: 3:15 - loss: 4.763 - ETA: 3:14 - loss: 4.765 - ETA: 3:12 - loss: 4.765 - ETA: 3:11 - loss: 4.767 - ETA: 3:09 - loss: 4.769 - ETA: 3:08 - loss: 4.771 - ETA: 3:07 - loss: 4.771 - ETA: 3:05 - loss: 4.771 - ETA: 3:04 - loss: 4.771 - ETA: 3:03 - loss: 4.772 - ETA: 3:01 - loss: 4.769 - ETA: 3:00 - loss: 4.769 - ETA: 2:58 - loss: 4.769 - ETA: 2:57 - loss: 4.769 - ETA: 2:56 - loss: 4.768 - ETA: 2:54 - loss: 4.767 - ETA: 2:53 - loss: 4.768 - ETA: 2:52 - loss: 4.767 - ETA: 2:50 - loss: 4.767 - ETA: 2:49 - loss: 4.767 - ETA: 2:47 - loss: 4.766 - ETA: 2:46 - loss: 4.767 - ETA: 2:44 - loss: 4.768 - ETA: 2:43 - loss: 4.769 - ETA: 2:42 - loss: 4.770 - ETA: 2:40 - loss: 4.772 - ETA: 2:39 - loss: 4.772 - ETA: 2:38 - loss: 4.772 - ETA: 2:36 - loss: 4.772 - ETA: 2:35 - loss: 4.773 - ETA: 2:33 - loss: 4.774 - ETA: 2:32 - loss: 4.775 - ETA: 2:30 - loss: 4.774 - ETA: 2:29 - loss: 4.774 - ETA: 2:28 - loss: 4.774 - ETA: 2:26 - loss: 4.774 - ETA: 2:25 - loss: 4.774 - ETA: 2:24 - loss: 4.774 - ETA: 2:22 - loss: 4.775 - ETA: 2:21 - loss: 4.774 - ETA: 2:19 - loss: 4.775 - ETA: 2:18 - loss: 4.776 - ETA: 2:16 - loss: 4.777 - ETA: 2:15 - loss: 4.775 - ETA: 2:14 - loss: 4.775 - ETA: 2:12 - loss: 4.776 - ETA: 2:11 - loss: 4.777 - ETA: 2:09 - loss: 4.778 - ETA: 2:08 - loss: 4.779 - ETA: 2:06 - loss: 4.777 - ETA: 2:05 - loss: 4.778 - ETA: 2:04 - loss: 4.777 - ETA: 2:02 - loss: 4.778 - ETA: 2:01 - loss: 4.778 - ETA: 1:59 - loss: 4.780 - ETA: 1:58 - loss: 4.780 - ETA: 1:57 - loss: 4.780 - ETA: 1:55 - loss: 4.781 - ETA: 1:54 - loss: 4.780 - ETA: 1:52 - loss: 4.780 - ETA: 1:51 - loss: 4.780 - ETA: 1:50 - loss: 4.781 - ETA: 1:48 - loss: 4.781 - ETA: 1:47 - loss: 4.781 - ETA: 1:45 - loss: 4.780 - ETA: 1:44 - loss: 4.781 - ETA: 1:42 - loss: 4.780 - ETA: 1:41 - loss: 4.781 - ETA: 1:40 - loss: 4.780 - ETA: 1:38 - loss: 4.781 - ETA: 1:37 - loss: 4.783 - ETA: 1:35 - loss: 4.783 - ETA: 1:34 - loss: 4.785 - ETA: 1:33 - loss: 4.786 - ETA: 1:31 - loss: 4.786 - ETA: 1:30 - loss: 4.785 - ETA: 1:28 - loss: 4.785 - ETA: 1:27 - loss: 4.784 - ETA: 1:25 - loss: 4.785 - ETA: 1:24 - loss: 4.785 - ETA: 1:23 - loss: 4.785 - ETA: 1:21 - loss: 4.785 - ETA: 1:20 - loss: 4.784 - ETA: 1:18 - loss: 4.784 - ETA: 1:17 - loss: 4.785 - ETA: 1:16 - loss: 4.785 - ETA: 1:14 - loss: 4.785 - ETA: 1:13 - loss: 4.785 - ETA: 1:11 - loss: 4.786 - ETA: 1:10 - loss: 4.787 - ETA: 1:09 - loss: 4.789 - ETA: 1:07 - loss: 4.788 - ETA: 1:06 - loss: 4.789 - ETA: 1:04 - loss: 4.789 - ETA: 1:03 - loss: 4.788 - ETA: 1:02 - loss: 4.787 - ETA: 1:00 - loss: 4.789 - ETA: 59s - loss: 4.789 - ETA: 57s - loss: 4.79 - ETA: 56s - loss: 4.79 - ETA: 54s - loss: 4.79 - ETA: 53s - loss: 4.79 - ETA: 52s - loss: 4.79 - ETA: 50s - loss: 4.79 - ETA: 49s - loss: 4.79 - ETA: 47s - loss: 4.79 - ETA: 46s - loss: 4.79 - ETA: 45s - loss: 4.79 - ETA: 43s - loss: 4.79 - ETA: 42s - loss: 4.79 - ETA: 40s - loss: 4.79 - ETA: 39s - loss: 4.79 - ETA: 38s - loss: 4.79 - ETA: 36s - loss: 4.79 - ETA: 35s - loss: 4.79 - ETA: 33s - loss: 4.79 - ETA: 32s - loss: 4.79 - ETA: 31s - loss: 4.79 - ETA: 29s - loss: 4.79 - ETA: 28s - loss: 4.79 - ETA: 26s - loss: 4.79 - ETA: 25s - loss: 4.79 - ETA: 23s - loss: 4.79 - ETA: 22s - loss: 4.79 - ETA: 21s - loss: 4.79 - ETA: 19s - loss: 4.79 - ETA: 18s - loss: 4.79 - ETA: 16s - loss: 4.79 - ETA: 15s - loss: 4.79 - ETA: 14s - loss: 4.79 - ETA: 12s - loss: 4.79 - ETA: 11s - loss: 4.79 - ETA: 9s - loss: 4.7946 - ETA: 8s - loss: 4.794 - ETA: 7s - loss: 4.794 - ETA: 5s - loss: 4.794 - ETA: 4s - loss: 4.794 - ETA: 2s - loss: 4.794 - ETA: 1s - loss: 4.794 - 282s 70ms/step - loss: 4.7948\n",
      "Epoch 10/10\n",
      "4000/4000 [==============================] - ETA: 4:36 - loss: 4.684 - ETA: 4:35 - loss: 4.638 - ETA: 4:33 - loss: 4.612 - ETA: 4:32 - loss: 4.600 - ETA: 4:31 - loss: 4.576 - ETA: 4:30 - loss: 4.589 - ETA: 4:31 - loss: 4.563 - ETA: 4:31 - loss: 4.535 - ETA: 4:29 - loss: 4.549 - ETA: 4:27 - loss: 4.540 - ETA: 4:25 - loss: 4.540 - ETA: 4:24 - loss: 4.535 - ETA: 4:22 - loss: 4.529 - ETA: 4:20 - loss: 4.530 - ETA: 4:19 - loss: 4.531 - ETA: 4:18 - loss: 4.539 - ETA: 4:17 - loss: 4.551 - ETA: 4:15 - loss: 4.552 - ETA: 4:14 - loss: 4.552 - ETA: 4:12 - loss: 4.554 - ETA: 4:11 - loss: 4.556 - ETA: 4:10 - loss: 4.568 - ETA: 4:08 - loss: 4.570 - ETA: 4:07 - loss: 4.578 - ETA: 4:06 - loss: 4.576 - ETA: 4:05 - loss: 4.578 - ETA: 4:03 - loss: 4.581 - ETA: 4:02 - loss: 4.583 - ETA: 4:00 - loss: 4.584 - ETA: 3:59 - loss: 4.582 - ETA: 3:57 - loss: 4.585 - ETA: 3:56 - loss: 4.586 - ETA: 3:55 - loss: 4.591 - ETA: 3:54 - loss: 4.582 - ETA: 3:52 - loss: 4.581 - ETA: 3:51 - loss: 4.581 - ETA: 3:49 - loss: 4.582 - ETA: 3:48 - loss: 4.582 - ETA: 3:46 - loss: 4.588 - ETA: 3:45 - loss: 4.588 - ETA: 3:43 - loss: 4.587 - ETA: 3:42 - loss: 4.586 - ETA: 3:41 - loss: 4.587 - ETA: 3:39 - loss: 4.586 - ETA: 3:38 - loss: 4.584 - ETA: 3:36 - loss: 4.586 - ETA: 3:35 - loss: 4.585 - ETA: 3:33 - loss: 4.585 - ETA: 3:32 - loss: 4.588 - ETA: 3:31 - loss: 4.589 - ETA: 3:29 - loss: 4.591 - ETA: 3:28 - loss: 4.592 - ETA: 3:26 - loss: 4.593 - ETA: 3:25 - loss: 4.593 - ETA: 3:24 - loss: 4.594 - ETA: 3:22 - loss: 4.595 - ETA: 3:21 - loss: 4.593 - ETA: 3:20 - loss: 4.592 - ETA: 3:18 - loss: 4.594 - ETA: 3:17 - loss: 4.593 - ETA: 3:15 - loss: 4.597 - ETA: 3:14 - loss: 4.597 - ETA: 3:13 - loss: 4.596 - ETA: 3:11 - loss: 4.595 - ETA: 3:10 - loss: 4.597 - ETA: 3:08 - loss: 4.597 - ETA: 3:07 - loss: 4.600 - ETA: 3:05 - loss: 4.601 - ETA: 3:04 - loss: 4.601 - ETA: 3:03 - loss: 4.599 - ETA: 3:01 - loss: 4.599 - ETA: 3:00 - loss: 4.602 - ETA: 2:58 - loss: 4.601 - ETA: 2:57 - loss: 4.603 - ETA: 2:55 - loss: 4.603 - ETA: 2:54 - loss: 4.605 - ETA: 2:53 - loss: 4.608 - ETA: 2:51 - loss: 4.609 - ETA: 2:50 - loss: 4.612 - ETA: 2:48 - loss: 4.610 - ETA: 2:47 - loss: 4.610 - ETA: 2:45 - loss: 4.611 - ETA: 2:44 - loss: 4.612 - ETA: 2:43 - loss: 4.612 - ETA: 2:41 - loss: 4.612 - ETA: 2:40 - loss: 4.613 - ETA: 2:38 - loss: 4.611 - ETA: 2:37 - loss: 4.610 - ETA: 2:36 - loss: 4.610 - ETA: 2:34 - loss: 4.611 - ETA: 2:33 - loss: 4.610 - ETA: 2:31 - loss: 4.611 - ETA: 2:30 - loss: 4.611 - ETA: 2:29 - loss: 4.612 - ETA: 2:27 - loss: 4.612 - ETA: 2:26 - loss: 4.613 - ETA: 2:24 - loss: 4.612 - ETA: 2:23 - loss: 4.611 - ETA: 2:22 - loss: 4.612 - ETA: 2:20 - loss: 4.612 - ETA: 2:19 - loss: 4.613 - ETA: 2:17 - loss: 4.613 - ETA: 2:16 - loss: 4.612 - ETA: 2:15 - loss: 4.613 - ETA: 2:13 - loss: 4.613 - ETA: 2:12 - loss: 4.614 - ETA: 2:10 - loss: 4.615 - ETA: 2:09 - loss: 4.616 - ETA: 2:07 - loss: 4.617 - ETA: 2:06 - loss: 4.618 - ETA: 2:05 - loss: 4.618 - ETA: 2:03 - loss: 4.619 - ETA: 2:02 - loss: 4.618 - ETA: 2:00 - loss: 4.619 - ETA: 1:59 - loss: 4.619 - ETA: 1:58 - loss: 4.620 - ETA: 1:56 - loss: 4.619 - ETA: 1:55 - loss: 4.619 - ETA: 1:54 - loss: 4.619 - ETA: 1:52 - loss: 4.621 - ETA: 1:51 - loss: 4.620 - ETA: 1:49 - loss: 4.620 - ETA: 1:48 - loss: 4.622 - ETA: 1:47 - loss: 4.622 - ETA: 1:45 - loss: 4.622 - ETA: 1:44 - loss: 4.623 - ETA: 1:42 - loss: 4.622 - ETA: 1:41 - loss: 4.623 - ETA: 1:40 - loss: 4.624 - ETA: 1:38 - loss: 4.625 - ETA: 1:37 - loss: 4.624 - ETA: 1:35 - loss: 4.624 - ETA: 1:34 - loss: 4.625 - ETA: 1:32 - loss: 4.624 - ETA: 1:31 - loss: 4.624 - ETA: 1:30 - loss: 4.626 - ETA: 1:28 - loss: 4.626 - ETA: 1:27 - loss: 4.625 - ETA: 1:25 - loss: 4.625 - ETA: 1:24 - loss: 4.625 - ETA: 1:23 - loss: 4.626 - ETA: 1:21 - loss: 4.626 - ETA: 1:20 - loss: 4.626 - ETA: 1:18 - loss: 4.627 - ETA: 1:17 - loss: 4.628 - ETA: 1:16 - loss: 4.629 - ETA: 1:14 - loss: 4.629 - ETA: 1:13 - loss: 4.629 - ETA: 1:11 - loss: 4.629 - ETA: 1:10 - loss: 4.629 - ETA: 1:09 - loss: 4.630 - ETA: 1:07 - loss: 4.629 - ETA: 1:06 - loss: 4.631 - ETA: 1:04 - loss: 4.630 - ETA: 1:03 - loss: 4.629 - ETA: 1:01 - loss: 4.630 - ETA: 1:00 - loss: 4.631 - ETA: 59s - loss: 4.631 - ETA: 57s - loss: 4.63 - ETA: 56s - loss: 4.63 - ETA: 54s - loss: 4.63 - ETA: 53s - loss: 4.63 - ETA: 52s - loss: 4.63 - ETA: 50s - loss: 4.63 - ETA: 49s - loss: 4.63 - ETA: 47s - loss: 4.63 - ETA: 46s - loss: 4.63 - ETA: 45s - loss: 4.63 - ETA: 43s - loss: 4.63 - ETA: 42s - loss: 4.63 - ETA: 40s - loss: 4.63 - ETA: 39s - loss: 4.63 - ETA: 37s - loss: 4.63 - ETA: 36s - loss: 4.63 - ETA: 35s - loss: 4.63 - ETA: 33s - loss: 4.63 - ETA: 32s - loss: 4.63 - ETA: 30s - loss: 4.63 - ETA: 29s - loss: 4.63 - ETA: 28s - loss: 4.63 - ETA: 26s - loss: 4.63 - ETA: 25s - loss: 4.64 - ETA: 23s - loss: 4.64 - ETA: 22s - loss: 4.64 - ETA: 21s - loss: 4.64 - ETA: 19s - loss: 4.64 - ETA: 18s - loss: 4.64 - ETA: 16s - loss: 4.64 - ETA: 15s - loss: 4.64 - ETA: 14s - loss: 4.64 - ETA: 12s - loss: 4.64 - ETA: 11s - loss: 4.64 - ETA: 9s - loss: 4.6415 - ETA: 8s - loss: 4.641 - ETA: 7s - loss: 4.641 - ETA: 5s - loss: 4.641 - ETA: 4s - loss: 4.642 - ETA: 2s - loss: 4.643 - ETA: 1s - loss: 4.642 - 281s 70ms/step - loss: 4.6424\n"
     ]
    }
   ],
   "source": [
    "model.fit(x=train_padded_ids[:, :-1], \n",
    "          y=train_padded_ids[:, 1:, None], \n",
    "          epochs=10,\n",
    "          batch_size=20)\n",
    "\n",
    "model.save_weights('corpse_weights.h5')\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}