09.4_Engineering_rare_values.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Engineering Rare Labels / Rare Categories\n",
    "\n",
    "Rare values are labels/ categories within a categorical variable that are only present for a small percentage of the observations.\n",
    "\n",
    "There is no rule of thumb to determine how small is a small percentage, but typically, any value below 5% may cause over-fitting in trees.\n",
    "\n",
    "Rare labels may exist in variables that have intrinsically a huge number of labels, or they can be present in variables with few labels (e.g., 2-10). There is no rule of thumb to determine how many different labels is big (and therefore represent high cardinality) and it will depend as well on how many observations there are in the dataset. In a dataset with 1,000 observations, 100 labels may seem a lot, whereas in a dataset with 100,000 observations it may not be so high.\n",
    "\n",
    "In situations where rare labels are present in variables with only a few categories, the rare label may be adding some information. On the other hand, in variables with a high number of categories, likely there will be very many labels with a low frequency, which will quite likely add noise instead of information. \n",
    "\n",
    "Whether rare labels should be processed before training a machine learning algorithm will depend on the dataset and problem at hand. Ideally, if there are not too many variables, you could try and explore the variables and their categories one at a time, and determine whether the rare labels add noise or information.\n",
    "\n",
    "If, on the other hand, the dataset has very many categorical variables, and exploring one at a time is not an option you may choose to sacrifice the ideal / optimal performance for a higher delivery speed.\n",
    "\n",
    "### Engineering rare labels\n",
    "\n",
    "There are multiple ways of accounting for rare labels. Some of them handle rare labels at the same time of converting labels into numbers. I will explain those in the section \"Engineer labels of categorical variables\".\n",
    "\n",
    "In this section of the course, I will expand on how to handle rare labels by re-categorising the observation that show rare labels for a certain variable. These observations can be re-categorised by:\n",
    "\n",
    "- Replacing the rare label by most frequent label\n",
    "- Grouping the observations that show rare labels into a unique category (with a new label like 'Rare', or 'Other')\n",
    "\n",
    "In this and the comming lectures I will explain when it is convenient to use one or the other way of replacing rare values, and evaluate the consequences of replacing rare labels in variables with:\n",
    "\n",
    "- One predominant category\n",
    "- A small number of different categories\n",
    "- High cardinality\n",
    "\n",
    "Specifically, in this lecture I will demonstrate how to work with rare labels in variables with one predominant category using the Mercedes Benz dataset from Kaggle.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "===============================================================================\n",
    "\n",
    "## Real Life example: \n",
    "\n",
    "### The Mercedes-Benz Greener Manufacturing challenge in Kaggle\n",
    "\n",
    "Daimler’s Mercedes-Benz cars are leaders in the premium car industry. With a huge selection of car features and options, customers can choose the customized Mercedes-Benz of their dreams.\n",
    "\n",
    "To ensure the safety and reliability of each and every unique car configuration before they hit the road, Daimler’s engineers have developed a robust testing system. Testing time varies depending on the different car features and options. The task is to predict the time it takes for a car with certain features to pass the testing. Therefore it is a regression problem: we need to estimate a continuous variable.\n",
    "\n",
    "The dataset contains a set of car features, the variable names are masked, so it is impossible to find out what each one of them means. The variable to predict is _y_: time to pass the testing.\n",
    "\n",
    "============================================================================="
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "% matplotlib inline\n",
    "\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "#from sklearn.ensemble import GradientBoostingRegressor\n",
    "\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "pd.set_option('display.max_columns', None) # to display the total number columns present in the dataset\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Functions\n",
    "\n",
    "Below I will write a few functions to convert categories in categorical variables into numbers so we can use them in sklearn and then to quickly test these variables in a random forest."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def train_rf(X_train, y_train, X_test, y_test, columns):\n",
    "    # function to train the random forest\n",
    "    # and test it on train and test sets\n",
    "    \n",
    "    rf = RandomForestRegressor(n_estimators=800, random_state=39)\n",
    "    \n",
    "    if type(columns)==str: # if we train using only 1 variable (pass a string instead of list in the \"columns\" argument of the function)\n",
    "        rf.fit(X_train[columns].to_frame(), y_train.values)\n",
    "        pred_train = rf.predict(X_train[columns].to_frame())\n",
    "        pred_test = rf.predict(X_test[columns].to_frame())\n",
    "        \n",
    "    else: # if we train using multiple variables (pass a list in the argument \"columns\")\n",
    "        rf.fit(X_train[columns], y_train.values)\n",
    "        pred_train = rf.predict(X_train[columns])\n",
    "        pred_test = rf.predict(X_test[columns])\n",
    "        \n",
    "    print('Train set')\n",
    "    print('Random Forests mse: {}'.format(mean_squared_error(y_train, pred_train)))\n",
    "    print('Test set')\n",
    "    print('Random Forests mse: {}'.format(mean_squared_error(y_test, pred_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def labels_to_numbers(X_train, X_test, columns):\n",
    "    # function to encode labels into numbers\n",
    "    # each label will be assigned an ordinal number from 0 onwards\n",
    "    \n",
    "    for col in columns:\n",
    "        labels_dict = {k:i for i, k in enumerate(X_train[col].unique(), 0)}\n",
    "        X_train.loc[:, col] = X_train.loc[:, col].map(labels_dict )\n",
    "        X_test.loc[:, col] = X_test.loc[:, col].map(labels_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mercedes Benz dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>y</th>\n",
       "      <th>X1</th>\n",
       "      <th>X2</th>\n",
       "      <th>X3</th>\n",
       "      <th>X6</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>130.81</td>\n",
       "      <td>v</td>\n",
       "      <td>at</td>\n",
       "      <td>a</td>\n",
       "      <td>j</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>88.53</td>\n",
       "      <td>t</td>\n",
       "      <td>av</td>\n",
       "      <td>e</td>\n",
       "      <td>l</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>76.26</td>\n",
       "      <td>w</td>\n",
       "      <td>n</td>\n",
       "      <td>c</td>\n",
       "      <td>j</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>80.62</td>\n",
       "      <td>t</td>\n",
       "      <td>n</td>\n",
       "      <td>f</td>\n",
       "      <td>l</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>78.02</td>\n",
       "      <td>v</td>\n",
       "      <td>n</td>\n",
       "      <td>f</td>\n",
       "      <td>d</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        y X1  X2 X3 X6\n",
       "0  130.81  v  at  a  j\n",
       "1   88.53  t  av  e  l\n",
       "2   76.26  w   n  c  j\n",
       "3   80.62  t   n  f  l\n",
       "4   78.02  v   n  f  d"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's load the dataset with just the columns that we need for this exercise\n",
    "\n",
    "cols_to_use = ['y', 'X1', 'X2', 'X3', 'X6']\n",
    "data = pd.read_csv('mercedesbenz.csv', usecols=cols_to_use)\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "variable:  X1  number of labels:  27\n",
      "variable:  X2  number of labels:  44\n",
      "variable:  X3  number of labels:  7\n",
      "variable:  X6  number of labels:  12\n",
      "total cars:  4209\n"
     ]
    }
   ],
   "source": [
    "# let's check at the different number of labels within each variable\n",
    "\n",
    "cols_to_use = ['X1', 'X2', 'X3', 'X6']\n",
    "\n",
    "for col in cols_to_use:\n",
    "    print('variable: ', col, ' number of labels: ', len(data[col].unique()) )\n",
    "print('total cars: ', len(data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEMCAYAAADu7jDJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X2cXVV97/HP11TAIghoCLkQDbQpNKWY0khoiw9IUR4s\nAa9SUIFSNNKKQq1eU6+ordYiRe2lRWLUtKAoBZVLqqkUUxQRsAkP5bGpaQqS3JCkyJNQhcD3/rHX\nyGY4c2bPnNkzc5jv+/U6r7PX2mvt/duZnPM7e+0n2SYiImK0njPRAURERH9LIomIiJ4kkURERE+S\nSCIioidJJBER0ZMkkoiI6EkSSURE9CSJJCIiepJEEhERPfm5iQ5gPLzoRS/y7NmzJzqMiIi+csMN\nN/yX7enDtZsSiWT27NmsXr16osOIiOgrku5u0i5DWxER0ZMkkoiI6EkSSURE9KTVRCLpMElrJK2V\ntLjD/DdLukXSrZKulfTS4fpK2kXSlZJ+UN53bnMbIiKiu9YSiaRpwHnA4cBc4HhJcwc1+0/glbZ/\nFfgIsLRB38XASttzgJWlHBERE6TNPZIDgLW219l+DLgYWFhvYPta2/eX4vXAHg36LgQuKNMXAEe3\nuA0RETGMNhPJ7sA9tfL6UjeUU4B/bNB3hu2NZfpeYEanhUlaJGm1pNVbtmwZaewREdHQpDjYLulg\nqkTyvpH0c/Wc4I7PCra91PZ82/OnTx/2epqIiBilNi9I3ADMqpX3KHVPI2k/4HPA4bbva9B3k6SZ\ntjdKmglsHklQsxd/o2P9XWcdOZLFRERE0eYeySpgjqQ9JW0DHAcsrzeQ9GLga8AJtv+9Yd/lwEll\n+iTg8ha3ISIihtHaHontrZJOA64ApgHLbN8u6dQyfwnwQeCFwKclAWwtw1Ed+5ZFnwVcIukU4G7g\n2La2ISIihtfqvbZsrwBWDKpbUpt+K/DWpn1L/X3AIWMbaUREjNakONgeERH9K4kkIiJ6kkQSERE9\nSSKJiIieJJFERERPkkgiIqInSSQREdGTJJKIiOhJEklERPQkiSQiInqSRBIRET1JIomIiJ4kkURE\nRE+SSCIioidJJBER0ZMkkoiI6EkSSURE9KTVRCLpMElrJK2VtLjD/H0kXSfpp5LeU6vfW9LNtddD\nks4o8z4saUNt3hFtbkNERHTX2qN2JU0DzgMOBdYDqyQtt31HrdmPgHcBR9f72l4DzKstZwNwWa3J\np2yf01bsERHRXJt7JAcAa22vs/0YcDGwsN7A9mbbq4DHuyznEOA/bN/dXqgRETFabSaS3YF7auX1\npW6kjgO+PKjunZJukbRM0s6dOklaJGm1pNVbtmwZxWojIqKJSX2wXdI2wFHApbXq84G9qIa+NgKf\n6NTX9lLb823Pnz59euuxRkRMVW0mkg3ArFp5j1I3EocDN9reNFBhe5PtJ2w/CXyWaggtIiImSJuJ\nZBUwR9KeZc/iOGD5CJdxPIOGtSTNrBWPAW7rKcqIiOhJa2dt2d4q6TTgCmAasMz27ZJOLfOXSNoN\nWA3sCDxZTvGda/shSdtTnfH19kGLPlvSPMDAXR3mR0TEOGotkQDYXgGsGFS3pDZ9L9WQV6e+jwAv\n7FB/whiHGRERPZjUB9sjImLySyKJiIieJJFERERPkkgiIqInSSQREdGTJJKIiOhJEklERPQkiSQi\nInqSRBIRET1JIomIiJ4kkURERE+SSCIioidJJBER0ZMkkoiI6EkSSURE9CSJJCIiepJEEhERPWk1\nkUg6TNIaSWslLe4wfx9J10n6qaT3DJp3l6RbJd0saXWtfhdJV0r6QXnfuc1tiIiI7lpLJJKmAecB\nhwNzgeMlzR3U7EfAu4BzhljMwbbn2Z5fq1sMrLQ9B1hZyhERMUHa3CM5AFhre53tx4CLgYX1BrY3\n214FPD6C5S4ELijTFwBHj0WwERExOm0mkt2Be2rl9aWuKQPfknSDpEW1+hm2N5bpe4EZnTpLWiRp\ntaTVW7ZsGUncERExApP5YPtBtudRDY29Q9IrBjewbaqE8wy2l9qeb3v+9OnTWw41ImLqajORbABm\n1cp7lLpGbG8o75uBy6iGygA2SZoJUN43j0m0ERExKsMmEknbS3pOmf4lSUdJem6DZa8C5kjaU9I2\nwHHA8iZBlXXuMDANvAa4rcxeDpxUpk8CLm+yzIiIaMfPNWhzNfDycprtP1EliN8F3tytk+2tkk4D\nrgCmActs3y7p1DJ/iaTdgNXAjsCTks6gOsPrRcBlkgZi/JLtb5ZFnwVcIukU4G7g2JFscEREjK0m\niUS2Hy1f3J+2fbakm5ss3PYKYMWguiW16XuphrwGewh46RDLvA84pMn6IyKifU2OkUjSb1DtgXyj\n1E1rL6SIiOgnTRLJ6cCfAJeVoam9gKvaDSsiIvpF16GtcnX6UbaPGqizvY7qavSIiIjuicT2E5IO\nGq9gJqPZi78x5Ly7zjpyHCOJiJicmhxsv0nScuBS4JGBSttfay2qPjdU8kniiYhnoyaJZDvgPuDV\ntToDSSRjJIknIvrZsInE9snjEUhERPSnYROJpO2AU4Bfodo7AcD277cYV0RE9Ikmp/9+AdgNeC3w\nHaoLCB9uM6iIiOgfTRLJL9o+E3jE9gXAkcCCdsOKiIh+0SSRDDx06gFJ+wIvAHZtL6SIiOgnTc7a\nWlpu2PgBqjvvPh/4YKtRRURE32hy1tbnyuTVwF7thhMREf2myfNIPiZpp1p5Z0kfbTesiIjoF02O\nkRxu+4GBgu37gSPaCykiIvpJk0QyTdK2AwVJzwO27dI+IiKmkCYH2y8CVkr621I+GbigvZAiIqKf\nDLtHYvvjwEeBXy6vj9g+u8nCJR0maY2ktZIWd5i/j6TrJP1U0ntq9bMkXSXpDkm3Szq9Nu/DkjZI\nurm8MswWETGBmuyRUJ6X/s1hG9aUZ5mcBxwKrAdWSVpu+45asx9RPdvk6EHdtwJ/bPtGSTsAN0i6\nstb3U7bPGUk8ERHRjibHSEbrAGCt7XW2HwMuBhbWG9jebHsVT130OFC/0faNZfph4E5g9xZjjYiI\nUWozkewO3FMrr2cUyUDSbODXgO/Xqt8p6RZJy8rFkhERMUGGTCSSVpb3j49fOM+I4fnAV4EzbD9U\nqs+nujByHrAR+MQQfRdJWi1p9ZYtW8Yl3oiIqajbMZKZkn4TOErSxYDqMweGnrrYAMyqlfcodY1I\nei5VErmo/jRG25tqbT4LfL1Tf9tLgaUA8+fPd9P1RkTEyHRLJB8EzqRKAJ8cNM88/YmJnawC5kja\nkyqBHAe8qUlQkgR8HrjT9icHzZtpe2MpHgPc1mSZERHRjiETie2vAF+RdKbtj4x0wba3SjoNuAKY\nBiyzfbukU8v8JZJ2A1YDOwJPSjoDmAvsB5wA3Crp5rLI99teAZwtaR5VMrsLePtIY4uIiLHT5KaN\nH5F0FPCKUvVt2x2Hkzr0XQGsGFS3pDZ9L9Uez2DXMGgordbnhCbrjoiI8dHkpo1/AZwO3FFep0v6\nWNuBRUREf2hyQeKRwDzbTwJIugC4CXh/m4FFRER/aHodyU616Re0EUhERPSnJnskfwHcJOkqquMW\nrwCecd+siIiYmpocbP+ypG8DLytV7ysHySMiIhrftHEj1fPaIyIinqbNe21FRMQUkEQSERE9aZRI\nJB0k6eQyPb3c9iQiIqLRBYkfAt4H/Empei7wxTaDioiI/tFkj+QY4CjgEQDb/w/Yoc2gIiKifzRJ\nJI/ZNtVNEpG0fbshRUREP2mSSC6R9BlgJ0lvA74FfLbdsCIiol80uSDxHEmHAg8BewMftH1l65FF\nRERfaHpB4pVAkkdERDzDsIlE0sOU4yM1D1I9kOqPba9rI7CIiOgPTfZI/gpYD3yJ6qaNxwG/ANwI\nLANe1VZwEREx+TU52H6U7c/Yftj2Q7aXAq+1/ffAzi3HFxERk1yTRPKopGMlPae8jgV+UuYNHvJ6\nGkmHSVojaa2kZ9x6XtI+kq6T9FNJ72nSV9Iukq6U9IPynmQWETGBmiSSNwMnAJuBTWX6LZKeB5w2\nVCdJ04DzgMOBucDxkuYOavYj4F3AOSPouxhYaXsOsJI8GyUiYkI1Of13HfA7Q8y+pkvXA4C1Awfj\nJV0MLKR67vvAsjcDmyUdOYK+C3nquMwFwLepbuESEREToMlZW9sBpwC/Amw3UG/794fpujtwT628\nHljQMK5ufWeU56MA3AvMGCLuRcAigBe/+MUNVxsRESPVZGjrC8BuwGuB7wB7AA+3GVRT9Vu3dJi3\n1PZ82/OnT58+zpFFREwdTRLJL9o+E3jE9gXAkTTbs9gAzKqV9yh1TXTru0nSTIDyvrnhMiMiogVN\nriN5vLw/IGlfquGkXRv0WwXMKc8u2UB1/cmbGsbVre9y4CTgrPJ+ecNlPqvMXvyNjvV3nTX4cFNE\nRLuaJJKl5RTbD1B9iT8fOHO4Tra3SjoNuAKYBiyzfbukU8v8JZJ2o7pCfkfgSUlnAHNtP9Spb1n0\nWVQ3kjwFuBs4dgTbGxERY6xJIllp+37gamAvgKZPSLS9AlgxqG5JbfpeqmGrRn1L/X3AIU3WHxER\n7WtyjOSrHeq+MtaBREREfxpyj0TSPlSn/L5A0utrs3akdhpwRERMbd2GtvYGXgfsxNMvSHwYeFub\nQUVERP8YMpHYvhy4XNJv2L5uHGOKiIg+0uRg+1pJ7wdm19s3uLI9IiKmgCaJ5HLgu1TPan+i3XAi\nIqLfNEkkP287N0WMiIiOmpz++3VJR7QeSURE9KUmieR0qmTyE0kPSXpY0kNtBxYREf2hyfNIdhiP\nQCIioj8Nu0eiylsknVnKsyQd0H5oERHRD5oMbX0a+A2euvvuj6kegxsREdHorK0FtveXdBOA7fsl\nbdNyXBER0Sea7JE8Lmka5UmEkqYDT7YaVURE9I0mieRc4DJgV0l/DlwDfKzVqCIiom80OWvrIkk3\nUD0DRMDRtu9sPbKIiOgLwyYSSQcCt9s+r5R3lLTA9vdbjy4iIia9JkNb51OdqTXgx6VuWJIOk7RG\n0lpJizvMl6Rzy/xbJO1f6veWdHPt9VB5DC+SPixpQ21errqPiJhATc7akm0PFGw/KanJnsw0qtOE\nDwXWA6skLbd9R63Z4cCc8lpAlaAW2F4DzKstZwPVcZoBn7J9ToPYIyKiZU32SNZJepek55bX6cC6\nBv0OANbaXmf7MeBiYOGgNguBC125HthJ0sxBbQ4B/sP23Q3WGRER46xJIjkV+E2qvYL1VHsOixr0\n2x24p1ZeX+pG2uY44MuD6t5ZhsKWSdq508olLZK0WtLqLVu2NAg3IiJGo2siKcNKb7Z9nO1dbc+w\n/Sbbm8cjuHLh41HApbXq84G9qIa+NgKf6NTX9lLb823Pnz59euuxRkRMVV0Tie0ngONHuewNwKxa\neY9SN5I2hwM32t5Ui2mT7SdsPwl8lmoILSIiJkiToa3vSfobSS+XtP/Aq0G/VcAcSXuWPYvjgOWD\n2iwHTixnbx0IPGh7Y23+8Qwa1hp0DOUY4LYGsUREREuanLU1r7z/Wa3OwKu7dbK9VdJpwBXANGCZ\n7dslnVrmLwFWAEcAa4FHgZMH+kvanuqMr7cPWvTZkuaVGO7qMD8iIsZRkyvbDx7twm2voEoW9bol\ntWkD7xii7yPACzvUnzDaeCIiYuw1eR7JDEmfl/SPpTxX0inthxYREf2gyTGSv6ManvofpfzvwBlt\nBRQREf2lSSJ5ke1LKLeOt70VeKLVqCIiom80SSSPSHohTz2P5EDgwVajioiIvtHkrK13U52m+wuS\nvgdMB97QalQREdE3mpy1daOkVwJ7Uz2PZI3tx1uPLCIi+kKTu/huB/whcBDV8NZ3JS2x/ZO2g4uI\niMmvydDWhcDDwF+X8puALwBvbCuoiIjoH00Syb6259bKV0m6Y8jWERExpTQ5a+vGcqYWAJIWAKvb\nCykiIvpJkz2SXweulfTDUn4xsEbSrVR3OdmvtehizMxe/I2O9XeddeQ4RxIRzzZNEslhrUcRERF9\nq8npv3nEbUREDKnJMZKIiIghJZFERERPkkgiIqInSSQREdGTVhOJpMMkrZG0VtLiDvMl6dwy/5b6\ns+Al3SXpVkk3S1pdq99F0pWSflDed25zGyIiorvWEomkacB5wOHAXOB4SXMHNTscmFNei4DzB80/\n2PY82/NrdYuBlbbnACtLOSIiJkibeyQHAGttr7P9GHAxsHBQm4XAha5cD+wkaeYwy10IXFCmLwCO\nHsugIyJiZNpMJLsD99TK60td0zYGviXpBkmLam1m2N5Ypu8FZnRauaRFklZLWr1ly5bRbkNERAxj\nMh9sP8j2PKrhr3dIesXgBrZNeXJjh3lLbc+3PX/69OkthxoRMXU1uUXKaG0AZtXKe5S6Rm1sD7xv\nlnQZ1VDZ1cAmSTNtbyzDYJtbin9KG+reXJD7c0XE07W5R7IKmCNpT0nbAMdRPbK3bjlwYjl760Dg\nwZIgtpe0A4Ck7YHXALfV+pxUpk8CLm9xGyIiYhit7ZHY3irpNOAKYBqwzPbtkk4t85cAK4AjgLXA\no8DJpfsM4DJJAzF+yfY3y7yzgEsknQLcDRzb1jZERMTw2hzawvYKqmRRr1tSmzbwjg791gEvHWKZ\n9wGHjG2kERExWpP5YHtERPSBJJKIiOhJEklERPQkiSQiInqSRBIRET1JIomIiJ4kkURERE+SSCIi\noidJJBER0ZMkkoiI6EkSSURE9KTVe23F1DLUredz2/mIZ7fskURERE+SSCIioidJJBER0ZMkkoiI\n6EkOtseEycH5iGeHVvdIJB0maY2ktZIWd5gvSeeW+bdI2r/Uz5J0laQ7JN0u6fRanw9L2iDp5vI6\nos1tiIiI7lrbI5E0DTgPOBRYD6yStNz2HbVmhwNzymsBcH553wr8se0bJe0A3CDpylrfT9k+p63Y\nIyKiuTb3SA4A1tpeZ/sx4GJg4aA2C4ELXbke2EnSTNsbbd8IYPth4E5g9xZjjYiIUWozkewO3FMr\nr+eZyWDYNpJmA78GfL9W/c4yFLZM0s6dVi5pkaTVklZv2bJldFsQERHDmtRnbUl6PvBV4AzbD5Xq\n84G9gHnARuATnfraXmp7vu3506dPH5d4IyKmojYTyQZgVq28R6lr1EbSc6mSyEW2vzbQwPYm20/Y\nfhL4LNUQWkRETJA2E8kqYI6kPSVtAxwHLB/UZjlwYjl760DgQdsbJQn4PHCn7U/WO0iaWSseA9zW\n3iZERMRwWjtry/ZWSacBVwDTgGW2b5d0apm/BFgBHAGsBR4FTi7dfws4AbhV0s2l7v22VwBnS5oH\nGLgLeHtb2xAREcNr9YLE8sW/YlDdktq0gXd06HcNoCGWecIYhxkRET3Ile3RV3I1fMTkM6nP2oqI\niMkviSQiInqSRBIRET1JIomIiJ4kkURERE+SSCIioic5/Tee1XK6cET7skcSERE9yR5JRM1QezCQ\nvZiIoWSPJCIiepI9koge5ThMTHVJJBHjbDSJJ8kqJrMkkohnoZEmnhwbil7kGElERPQkeyQRMSpj\ntdeTIb3+l0QSEc8aGdKbGK0ObUk6TNIaSWslLe4wX5LOLfNvkbT/cH0l7SLpSkk/KO87t7kNERHR\nXWt7JJKmAecBhwLrgVWSltu+o9bscGBOeS0AzgcWDNN3MbDS9lklwSwG3tfWdkRE1GW47ZnaHNo6\nAFhrex2ApIuBhUA9kSwELizPbr9e0k6SZgKzu/RdCLyq9L8A+DZJJBExSU2FxKPqO7yFBUtvAA6z\n/dZSPgFYYPu0WpuvA2fZvqaUV1IlhdlD9ZX0gO2dSr2A+wfKg9a/CFhUinsDazqE+SLgv0a4aSPt\n03b7Z8s6JmNM47GOyRjTeKxjMsY0HuuYjDF16/MS29OH69zXB9ttW1LHTGh7KbC0W39Jq23PH8k6\nR9qn7fbPlnVMxpjGYx2TMabxWMdkjGk81jEZYxptn7o2D7ZvAGbVynuUuiZtuvXdVIa/KO+bxzDm\niIgYoTYTySpgjqQ9JW0DHAcsH9RmOXBiOXvrQOBB2xuH6bscOKlMnwRc3uI2RETEMFob2rK9VdJp\nwBXANGCZ7dslnVrmLwFWAEcAa4FHgZO79S2LPgu4RNIpwN3AsT2E2XXoa4z6tN3+2bKOyRjTeKxj\nMsY0HuuYjDGNxzomY0yj7fMzrR1sj4iIqSH32oqIiJ4kkURERE+SSKaAcjLDrOFbRkSMXBLJMCS9\nUdIOZfoDkr5WvyfYEH22k/Tu0varkv5I0nZjGNO7Je3etH25c8CKsVp/N5J2lnSApFcMvMZ4+S+V\ndFp5vbRB+y9KepukfRouf6WkIwbV9XQgssM6JOktkj5Yyi+WdMAYLv/jTep6XMc7m97nTtIXyvvp\nDdsPXKD8sKSHBr0elPSfkv5w9NE/bV2/3qHudcP0afXz3Y+m5MF2SfsCc4Gf/fFtXzhE21ts7yfp\nIOCjwF8CH7S9oMvyLwEeBr5Yqt4E7GT7jWMU/4eozlb7EfD3wKW2Nw3T5wLgb2yvGosYhljHW4HT\nqa77uRk4ELjO9qu7xHS67QdKeWfgE7Z/f4j2pwNvA75Wqo4Bltr+6y4xHQy8vLx+AbgJuNr2/xmi\n/TrgHuCfbf9pqbvR9pA/HgYSwmC2/2yI9ucDTwKvtv3LZbv/yfbLuqxjPvC/gZdQnW2pahXer0Pb\nZ8Q78P+4y/K3Bf4n1V0lfnY2Z5dt+CjVafk3AsuAKzzEl4mkO4DfBv6R6vZGqs+3/aOh4hpieS8E\nrrW99xhsx43AibZvK+XjgTPG8vMt6d0dqh8EbrB9c8P2P2P7k6Np26a+vrJ9NMqX8KuoEskKqhtH\nXgN0TCTAE+X9SKovrW+UD1E3+9qeWytfVT5Mg2N5GOj04Rv4ktix08LLF9yfStoP+F3gO5LW2/7t\nLjEtAN4s6W7gEbp/EV1j+6AO8XWNiyqJvAy43vbBZS/gY11i2m8giZTtul/Sr3VpfwrVrXIeKXF+\nHLgOGDKR2L5K0tUlroOBU4FfATomEuAB4BDgXEn/ALylSzwDHqlNbwe8DrizS/sFtveXdFOJ8f5y\nvVQ3FwHvBW6lSkLPIOkPgD8E9pJ0S23WDsD3hln+5ZQvN+Cnw7TF9gcknQm8huq0/b8pX7Cft/0f\ng5ovAVYCe5Xl1xOJS31jtu+T9Kqx2A7gDcBXJL2J6sfGiVTb1E2jz3fN/PL6h1J+HXALcKqkS22f\n3aH9y3jq2rnfAf4F+EGHZe9Q3vceok9HJRl/GPgtqr/BNcCf2b6vy3YMzfaUelF9EJ8D/GspzwCu\n7NL+68BngHXATsC2A3279PkicGCtvIDq5pRjvS27Ae+k+pK4ZZi2L+n0GuN4VpX3m4Fty/TtXdr/\nK7BzrbwLcOswf7vtauXturUvbVYC1wOfAl4P7DpM+5tq079X1rl+hP8O2wLf7jL/+1TXR91YytPr\n6x2izzUN1vsCql/iXx70d96lQd/bRvk3fynwV8C/Ud29+ybg7CHanj+W/9/GajuAX6K6Iew3gec1\naD+izzdwNfD8Wvn5wHeA5wF3DNF+h1p5B6q96G4xjagPcCVwJrBneX0A+NZo/92n3B4J8N+2n5S0\nVdKOVLdY6XYg+ljgMOAc2w+oui3Lezs1lHQrVXZ/LnCtpB+W8kuoPmhjoowPH0v1BXQp8DY//fb8\nz2D77rFafxfrJe0E/F/gSkn3U100OpRPANdJurSU3wj8eZf2fwt8X9JlpXw08PlhYroF+HVgX6pf\nqg9Ius72fw/RfsnAhO2/K3/TdwyzjsF+nmp4byjnApcBu0r6c6pfxR8YZpkfkvQ5qsT4s1/atr9W\nm36QahuPH2G8UP1//VXbtzZpXIYZT6S60d/ngPfaflzSc6h+Of+vwX1s/8Eo4hqpRttR+6wO2IUq\nuX9fEu4yDEj1/2ng8w3wYmDNwDI79N2Vp+8dPQ7MsP3fkjrtNc0AHquVHyt13Yy0z0zbH6mVPyrp\nd4dZx5CmYiJZXb7sPku1+/tj4NqhGtt+lKfG5HF1C5eNQzTvepBuDM2iGsd9xvjqRLJ9TJn8sKSr\nqH4hf7NL+wslrQYGjqG8vltCtP1JSd8GDipVJ9u+aZiY/ghA1QkTv0eVjHaj2mvo1P4zg8o3AB2P\n2QwY9KU0jSrBdxyTL8u8SNINVENoAo623W0oDKrho32ofqQMDG2Z2v/N0ajF/nPAyeUY0U/pMvRZ\n7EL193raD4XyI228PgedHAT8nqT/pPt29BLjYSNsfxFVghq4ndPvAF+StD1Pf6zGgAuBfxn0g+nv\nhlnHSPv8k6TjgEtK+Q1UdxIZlal6sH1nqodpzaXaBfxX21dPbFTRBlW32nk51a/Iu4DvAt+1/c9j\nuI6X1IpbgU22t47V8ss61rjDweUxWO5Lus0fpz3ZMTPU9kz0dpSTJX6rFL9ne/Uw7fen+n8L1RBV\n1x9MTfvUjnsK2J6njgFPA37soY9/dl/3VEskIz2zKPqbpPdQJY8bxvrLfTxJ+lvgL4cbwoxoStIu\nVD+o62evfmdUy5qCieRWnjqzaN7AmUW2Xz/BoUUMSdKdVKcvDzdkEzGsIX5QX2v7kNEsbyoeI/mJ\n7Z9IQtK2tv9N0pgPGUSMsZGOy0d0M9JT9buaiolkpGcWRUy4iR7jj2edMf1BPeUSyUjPLIqIeBYa\n0x/UU+4YSUREPEXSKyk/qG0/Nlz7jstIIomIiF7k7r8REdGTJJKIiOhJEknEOJA0S9VzNHYp5Z1L\nebakb0p6QNLXJzrOiNFIIokYB7bvobo77lml6iyqxxLcRfWMmxMmKLSIniWRRIyfTwEHSjqD6uaC\n5wDYXkn1oKSIvjTlriOJmCjlNuvvpbpu6TW2H5/omCLGQvZIIsbX4VSPIdh3ogOJGCtJJBHjRNI8\n4FCqG+T9UXlIWkTfSyKJGAeSRHWw/QzbP6Q6wH7OxEYVMTaSSCLGx9uAH9q+spQ/DfyypFdK+i7V\nI5MPkbRe0msnLMqIUcgtUiIioifZI4mIiJ4kkURERE+SSCIioidJJBER0ZMkkoiI6EkSSURE9CSJ\nJCIievLExqF6AAAABklEQVT/AWgUpKuHgExxAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xacbda13eb8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEQCAYAAACjnUNyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu8HVV99/HPlyBgEQUlKAUioHmkqQXEGLQPgmhRgpZ4\nwQIqPHIxpUrF2gtgRX2kVWO1T6VFYopYpCDeQFOIgCASFWmTAIIgaWPEkhRKtAiIFwh8nz9mbTI5\nnMs6J2fO2Um+79drv85es2bN/u05s+c3M2susk1ERMRItpjsACIiYuOQhBEREVWSMCIiokoSRkRE\nVEnCiIiIKkkYERFRJQkjIiKqJGFERESVJIyIiKiShBEREVW2nOwAxtOOO+7o3XfffbLDiIjYaCxb\ntuwntqfWjLtJJYzdd9+dpUuXTnYYEREbDUk/rh03h6QiIqJKpwlD0qGSlktaIem0YcZ7kaS1ko4Y\nbduIiJgYnSUMSVOAs4HZwAzgaEkzhhhvHnDVaNtGRMTE6XIPYxawwvZK2w8DFwNzBhnvj4EvA/eO\noW1EREyQLhPGLsBdrfKqMuxxknYBXgecM9q2ERExsSa70/vvgFNtPzbWCUiaK2mppKVr1qwZx9Ai\nIqKty9NqVwO7tcq7lmFtM4GLJQHsCBwmaW1lWwBsLwAWAMycOTPPm42I6EiXCWMJMF3SHjQr+6OA\nN7VHsL1H772kfwIus/0VSVuO1DYiIiZWZwnD9lpJJwNXAlOA82zfJumkUj9/tG1rPnf30y5fr3zn\nR149xm8QERFtnV7pbXsRsGjAsEEThe23jtQ2IiImz2R3ekdExEYiCSMiIqokYURERJUkjIiIqJKE\nERERVZIwIiKiShJGRERUScKIiIgqSRgREVElCSMiIqokYURERJUkjIiIqJKEERERVZIwIiKiShJG\nRERUScKIiIgqSRgREVGl04Qh6VBJyyWtkHTaIPVzJN0i6WZJSyUd0Kq7U9Ktvbou44yIiJF19ohW\nSVOAs4FDgFXAEkkLbd/eGu0aYKFtS9ob+AKwV6v+YNs/6SrGiIio1+Uexixghe2Vth8GLgbmtEew\n/XPbLsVtARMREX2py4SxC3BXq7yqDFuPpNdJugO4HDi+VWXgaknLJM3tMM6IiKgw6Z3eti+1vRfw\nWuDMVtUBtvcFZgPvkHTgYO0lzS39H0vXrFkzARFHRGyeukwYq4HdWuVdy7BB2V4M7Clpx1JeXf7e\nC1xKc4hrsHYLbM+0PXPq1KnjFXtERAzQZcJYAkyXtIekrYCjgIXtESQ9V5LK+/2ArYGfStpW0nZl\n+LbAK4HvdxhrRESMoLOzpGyvlXQycCUwBTjP9m2STir184E3AMdKegT4JXBkOWPqmcClJZdsCVxk\n+4quYo2IiJF1ljAAbC8CFg0YNr/1fh4wb5B2K4F9uowtIiJGZ9I7vSMiYuOQhBEREVWSMCIiokoS\nRkREVEnCiIiIKkkYERFRJQkjIiKqJGFERESVJIyIiKiShBEREVWSMCIiokoSRkREVEnCiIiIKkkY\nERFRJQkjIiKqJGFERESVJIyIiKiShBEREVU6TRiSDpW0XNIKSacNUj9H0i2Sbpa0VNIBtW0jImJi\ndZYwJE0BzgZmAzOAoyXNGDDaNcA+tvcFjgfOHUXbiIiYQF3uYcwCVtheafth4GJgTnsE2z+37VLc\nFnBt24iImFhdJoxdgLta5VVl2HokvU7SHcDlNHsZ1W1L+7nlcNbSNWvWjEvgERHxRJPe6W37Utt7\nAa8FzhxD+wW2Z9qeOXXq1PEPMCIigG4Txmpgt1Z51zJsULYXA3tK2nG0bSMionsjJgxJ20raorz/\nX5IOl/SkimkvAaZL2kPSVsBRwMIB036uJJX3+wFbAz+taRsRERNry4pxFgMvlbQDcBXNyvxI4M3D\nNbK9VtLJwJXAFOA827dJOqnUzwfeABwr6RHgl8CRpRN80LZj+oYRETEuahKGbP9C0gnAJ21/VNLN\nNRO3vQhYNGDY/Nb7ecC82rYRETF5avowJOklNHsUl5dhU7oLKSIi+lFNwjgFOB24tBxS2hO4ttuw\nIiKi3wx7SKpccX247cN7w2yvBN7ZdWAREdFfht3DsP0ocMBw40RExOahptP7JkkLgS8CD/UG2r6k\ns6giIqLv1CSMbWiujXh5a5iBJIyIiM3IiAnD9nETEUhERPS3EROGpG2AE4DfptnbAMD28UM2ioiI\nTU7NabUXAM8CXgVcR3Nfpwe7DCoiIvpPTcJ4ru0zgIdsnw+8Gti/27AiIqLf1CSMR8rfn0l6PvA0\nYKfuQoqIiH5Uc5bUgnLjwffS3DH2KcD7Oo0qIiL6Ts1ZUueWt4uBPbsNJyIi+lXN8zA+JGn7VnkH\nSX/VbVgREdFvavowZtv+Wa9g+z7gsO5CioiIflSTMKZI2rpXkPRkmifjRUTEZqSm0/tC4BpJnynl\n44DzuwspIiL6UU2n9zxJ3wN+rww60/aVNROXdCjwCZoHLp1r+yMD6t8MnAqI5mLAP7L9vVJ3Zxn2\nKLDW9syqbxQREZ2o2cPA9hXAFaOZcHmWxtnAIcAqYImkhbZvb432I+Ag2/dJmg0sYP2LAg+2/ZPR\nfG5ERHSjpg9jrGYBK2yvtP0wcDEwpz2C7etLJzrADTS3HYmIiD7UZcLYBbirVV5Vhg3lBOBrrbKB\nqyUtkzS3g/giImIUhkwYkq4pf+d1HYSkg2kSxqmtwQfY3heYDbxD0oFDtJ0raamkpWvWrOk61IiI\nzdZwexg7S/pd4HBJL5C0X/tVMe3VwG6t8q5l2Hok7Q2cC8yx/dPecNury997gUtpDnE9ge0Ftmfa\nnjl16tSKsCIiYiyG6/R+H3AGzYr+bwfUmfWfwDeYJcB0SXvQJIqjgDe1R5A0jebJfcfY/vfW8G2B\nLWw/WN6/EvjgyF8nIiK6MmTCsP0l4EuSzrB95mgnbHutpJOBK2lOqz3P9m2STir182mS0jOAT0qC\ndafPPhO4tAzbErionKkVERGTpOY6jDMlHQ70+hC+afuymonbXgQsGjBsfuv9icCJg7RbCexT8xkR\nETExam4++GHgFOD28jpF0oe6DiwiIvpLzYV7rwb2tf0YgKTzgZuA93QZWERE9Jfa6zC2b71/WheB\nREREf6vZw/gwcJOka2nu+XQgcFqnUUVERN+p6fT+nKRvAi8qg061fU+nUUVERN+pvfng3TTP846I\niM1Ul/eSioiITUgSRkREVKlKGJIOkHRceT+13O4jIiI2IzUX7r2f5i6yp5dBTwL+ucugIiKi/9Ts\nYbwOOBx4CMD2fwHbdRlURET0n5qE8bBt09yhtncn2YiI2MzUJIwvSPoUsL2ktwFXA//YbVgREdFv\nai7c+5ikQ4AHgOcB77P99c4ji4iIvlJ74d7XgSSJiIjN2IgJQ9KDlP6LlvuBpcCflmdXRETEJq5m\nD+PvgFXARTQ3HzwKeA5wI3Ae8LKugouIiP5R0+l9uO1P2X7Q9gO2FwCvsv15YIfhGko6VNJySSsk\nPeEOt5LeLOkWSbdKul7SPrVtIyJiYtUkjF9I+gNJW5TXHwC/KnUDD1U9TtIU4GxgNjADOFrSjAGj\n/Qg4yPbvAGcCC0bRNiIiJlBNwngzcAxwL/Df5f1bJD0ZOHmYdrOAFbZX2n4YuBiY0x7B9vW27yvF\nG4Bda9tGRMTEqjmtdiXw+0NUf3uYprsAd7XKq4D9hxn/BOBrY2wbEREdqzlLahualflvA9v0hts+\nfryCkHRw+YwDxtB2LjAXYNq0aWi8goqIiPXUHJK6AHgW8CrgOprDRg9WtFsN7NYq71qGrUfS3sC5\nwBzbPx1NWwDbC2zPtD1z6tSpFWFFRMRY1CSM59o+A3jI9vnAq6k7PLQEmC5pD0lb0ZyOu95T+yRN\nAy4BjrH976NpGxERE6vmOoxHyt+fSXo+cA+w00iNbK+VdDJwJTAFOM/2bZJOKvXzgfcBzwA+KQlg\nbdlbGLTtKL9bRESMo5qEsUDSDsB7abbynwKcUTNx24uARQOGzW+9PxE4sbZtRERMnpqEcU059XUx\nsCdAnrgXEbH5qenD+PIgw7403oFERER/G3IPQ9JeNKfSPk3S61tVT6V1em1ERGwehjsk9TzgNcD2\nrH/h3oPA27oMKiIi+s+QCcP2V4GvSnqJ7e9OYEwREdGHajq9V0h6D7B7e/zxvNI7IiL6X03C+Crw\nLZpneT/abTgREdGvahLGb9g+tfNIIiKir9WcVnuZpMM6jyQiIvpaTcI4hSZp/ErSA5IelPRA14FF\nRER/qXkexnYTEUhERPS3Efcw1HiLpDNKeTdJs7oPLSIi+knNIalPAi8B3lTKP6d53nZERGxGas6S\n2t/2fpJuArB9X3lGRUREbEZq9jAekTQFMICkqcBjnUYVERF9pyZhnAVcCuwk6a+BbwMf6jSqiIjo\nOzVnSV0oaRnwCkDAa23/oPPIIiKir9ScJfViYLXts23/A7BaUs0zvZF0qKTlklZIOm2Q+r0kfVfS\nryX92YC6OyXdKulmSUtrv1BERHSjptP7HGC/Vvnngwx7gtLvcTZwCLAKWCJpoe3bW6P9D/BO4LVD\nTOZg2z+piDEiIjpW04ch2+4VbD9GXaKZBaywvdL2w8DFwJz2CLbvtb0EeGQUMUdExCSoSRgrJb1T\n0pPK6xRgZUW7XYC7WuVVZVgtA1dLWiZp7ijaRUREB2oSxknA7wKraVb6+wMTsQI/wPa+wGzgHZIO\nHGwkSXMlLZW0dM2aNRMQVkTE5mnYhFH6Id5s+yjbO9l+pu032b63Ytqrgd1a5V3LsCq2V5e/99Kc\n1jvo7UhsL7A90/bMqVOn1k4+IiJGadiEYftR4OgxTnsJMF3SHuXK8KOAhTUNJW0rabvee+CVwPfH\nGEdERIyDms7r70j6B+DzwEO9gbZvHK6R7bWSTgauBKYA59m+TdJJpX6+pGcBS4GnAo9JehcwA9gR\nuFRSL8aLbF8x6m8XERHjpiZh7Fv+frA1zMDLR2poexGwaMCw+a3399AcqhroAWCfitgiImKC1Fzp\nffBEBBIREf2t5krvZ0r6tKSvlfIMSSd0H1pERPSTmtNq/4mmH+I3S/nfgXd1FVBERPSnmoSxo+0v\nUG5pbnst8GinUUVERN+pSRgPSXoG656H8WLg/k6jioiIvlNzltS7aa6feI6k7wBTgSM6jSoiIvpO\nzVlSN0o6CHgezfMwltvOzQIjIjYzIyYMSdsAbwcOoDks9S1J823/quvgIiKif9Qckvos8CDw96X8\nJuAC4I1dBRUREf2nJmE83/aMVvlaSbcPOXZERGySas6SurGcGQVAeTxrHpkaEbGZqdnDeCFwvaT/\nLOVpwHJJtwK2vXdn0UVERN+oSRiHdh7FBNr9tMvXK9/5kVdPUiQRERuXmtNqfzwRgURERH+r6cOI\niIhIwoiIiDpJGBERUSUJIyIiqnSaMCQdKmm5pBWSThukfi9J35X0a0l/Npq2ERExsTpLGJKmAGcD\ns4EZwNGSZgwY7X+AdwIfG0PbiIiYQF3uYcwCVtheafth4GJgTnsE2/faXgIMvPvtiG0jImJidZkw\ndgHuapVXlWFdt42IiA7UXOnd1yTNBeYCTJs2DW3g9HIleETE4Lrcw1gN7NYq71qGjWtb2wtsz7Q9\nc+rUqWMKNCIiRtZlwlgCTJe0h6StgKNoHvXadduIiOhAZ4ekbK+VdDJwJTAFOM/2bZJOKvXzJT2L\n5lbpTwUek/QuYIbtBwZr21WsERExsk77MGwvAhYNGDa/9f4emsNNVW0jImLy5ErviIiostGfJTWR\ncgZVRGzOsocRERFVkjAiIqJKEkZERFRJwoiIiCpJGBERUSUJIyIiqiRhRERElSSMiIiokoQRERFV\nkjAiIqJKEkZERFRJwoiIiCq5+eA4ys0JI2JTlj2MiIiokoQRERFVOk0Ykg6VtFzSCkmnDVIvSWeV\n+lsk7dequ1PSrZJulrS0yzgjImJknfVhSJoCnA0cAqwClkhaaPv21mizgenltT9wTvnbc7Dtn3QV\nY0RE1OtyD2MWsML2StsPAxcDcwaMMwf4rBs3ANtL2rnDmCIiYoy6TBi7AHe1yqvKsNpxDFwtaZmk\nuZ1FGRERVfr5tNoDbK+WtBPwdUl32F48cKSSTOYCTJs2DU10lBERm4ku9zBWA7u1yruWYVXj2O79\nvRe4lOYQ1xPYXmB7pu2ZU6dOHafQIyJioC4TxhJguqQ9JG0FHAUsHDDOQuDYcrbUi4H7bd8taVtJ\n2wFI2hZ4JfD9DmONiIgRdHZIyvZaSScDVwJTgPNs3ybppFI/H1gEHAasAH4BHFeaPxO4VFIvxots\nX9FVrBERMbJO+zBsL6JJCu1h81vvDbxjkHYrgX26jC0iIkYnV3pHRESVJIyIiKjSz6fVbnLad7PN\nnWwjYmOTPYyIiKiSPYw+kWdpRES/yx5GRERUScKIiIgqSRgREVElCSMiIqqk03sjkU7xiJhsSRib\ngCSTiJgISRibgSSUiBgP6cOIiIgq2cOI3LIkIqokYcSwcjgrInqSMGKDJKFEbD7ShxEREVU63cOQ\ndCjwCZpHtJ5r+yMD6lXqD6N5ROtbbd9Y0zb6X/Y+IjYtnSUMSVOAs4FDgFXAEkkLbd/eGm02ML28\n9gfOAfavbBsbuZESynCd8UlGEROvyz2MWcCK8nxuJF0MzAHaK/05wGfLs71vkLS9pJ2B3SvaRgxp\nuIQymkQ1Un0SWWxOukwYuwB3tcqraPYiRhpnl8q2ERudiUpkXU57IuOK/qJm476DCUtHAIfaPrGU\njwH2t31ya5zLgI/Y/nYpXwOcSrOHMWzb1jTmAnNL8XnA8vJ+R+Anw4S4IfWT1XZzjKvLaSeuxLWx\nTns843q27anDjLuO7U5ewEuAK1vl04HTB4zzKeDoVnk5sHNN24rPX9pV/WS13Rzj2hy/c+LaNOLa\nWL/zcK8uT6tdAkyXtIekrYCjgIUDxlkIHKvGi4H7bd9d2TYiIiZQZ30YttdKOhm4kubU2PNs3ybp\npFI/H1hEc0rtCprTao8brm1XsUZExMg6vQ7D9iKapNAeNr/13sA7atuO0oIO6yerbZfT7te4upx2\n4pq4tl1Ou1/j6nLaXcY1pM46vSMiYtOSW4NERESVJIyOlQ793SY7joiIDZWE0bHSTzPmvhhJb5S0\nXXn/XkmXSNpP0gVl2CnDtJWkt0h6XylPkzRrrLEMMv0dJM2SdGDv1aqbN8j481rvXzhI/WsqP3cb\nSe8u8+LLkv5E0jaVba+RdNiAYWM6njueJP2xpB3G2HbGIMNetsFBbcQ2ZBmZTJL2kXRyee3TGr6F\npN8dZPzeNWwPSnpgwOt+ST+S9PZxi29z6sOQ9BvAnwLTbL9N0nRgX9tflPT0wdrY/p/Sdibwl8Cz\naU4WUFPtvSXNtv21AZ91Uq+DX9L5wD/YXjJEXFsDb6C5YPHxExFsf1DSLeUzDgD+Cvgb4H3AdsDv\nAV8DXlbiWS9uSecAjwEvt/1bZYV0le0XVc6vjzPEGWqSTgROAXYFbgZeDHzX9stL/Y229xvQ5hbb\ne/fqgWNtf7+UjwbeZXvEK/olfQF4EPjnMuhNwPa231jqDwd6yes62//SaruS5i4C37D9f4eKdcDn\nvXu4eGz/bUXMg03jfmCZ7Zsl/RXN6eM3AufRXIdU9eOU9H3gAuCjwDbl70zbLyn13wauA74FfMf2\ngwPaD7f8XWD7mAHjX0Dzvx9SWf7GPN8knQVcbPv64aYxTPthl5ExTvMa26+QNM/2qRswnecDM2j+\nVwDY/mzZ+HsbcEkZ/Dpgge2/L+1usv2CUX7WM4DrgRuAU2z/rAzfAfi47eNHNb1NKWGUGf4ZmgXl\nXOAFwGm2ryr1nweW0ayonl8SyN22nybpR0B7ZvQSwp6l7XLgz4FbaVbC0IzwY0nXA++1/Y0y7l8A\nB9ueXcp3AM8Ffgw81Jp2b+V5BWXlATzamvbHewuJpA8Dt9q+SNJN5Xv+EbAnsJr1E4Zt79lbEbYX\nNEnfs71PeX/WCLP0FppTnbcsn/c52/eXtrcCLwJusL2vpL2ADwFfB95e4vpha1rb0ays3lLa7wl8\nieaH/FLgWOA1wOW2D5D04BD/j6dKut32elvVvWFlPs0CLixVRwNLbL+njHdjqT8L2A14C3BtL2EM\nMU/2oZnHlO/cuybo94F/a32nrwNvHPCjvNj2qyRdBMwEesnrNWX+7g580fZHJQl4ZZnnM4EvAJ+2\n/cPyw/8A8L/LfPk28EHbP5W0LTAPeGGZzxcC82w/VuLYo8zjl9Ik9l8D37L9J6V+uOVvvWQqacsS\n95NLHL3lrve/evx3U77zE+YX8B+l/EmaFeTurJ+ojpf0f4Ajae7ecGmZj0tbcUwdqm2pH24Z+Qzr\nL1sD274GOJMBG4c0tyg6Efg0zXKrAZO4brDpsv6y+36aDbwZNEceZgPftn2EpFuAl9h+qMSxLc1G\nWG898THgu8AltRsTpd3OwKKByWYsCWhTe4DS8bY/IelVwA7AMTRbXleV+ufYPrJszWL7F5LuLO/3\nKHsZ02ll/pY1toe6ePBw4DJJfw4cCuxFc7PEnleNEPeutg8dom61pE/R3Ll3Xtka3ML2WcBZks6x\n/UdDtH1EzZ1/DY//yB5r1W9Ds+B+vpTfSHODx+8C2D4fOFfS82hWYrdI+g7wj8CvbP9KEpK2tn1H\na7yvAR8GTmt91oO9vbUy7ZWSjgK+Avwn8ErbvwQOKPXbDTO/bpT0Yts3lO+1P9BbmbyaZq+xt7I8\nH7gJeE+pl+21wNslvZVmxds+FLQNzf/vi6X8BmAl8AxgP+C3elvokj4AtG+GtGMvWZTvcJ+knUpx\nV2A/2z8vbd9f2h5Is6L+qG1Luge4B1hb4vpSSUQvABaXeADeTPN/+z3gEeCXNCvxbYAf9b5/ieNH\nkn4FPFxeBwO/1Yr7CcufpNNL0n6ypAdaVY/QbPWe3hp3qN9N7zuvN79aCfZ6mr2eq2klqhLz+cD5\nZdpvoFn2p9meXkb56lBti+GWkcta421DsyX/X61hfwe8nmYD7fEVs5rbHZ1RvtfAvSOPsMz2HEGz\nAXKT7eMkPZN1e0Ea8F0eZf2k9IfAu4G15f/5eCIa7gNt363mkNYOtu8r3+XpjGX97zFcHt6vL+CW\n8vcTwOvK+5ta9dfT/KhuLOXn0GwhQrPlcCtwH3AtzQ/wmlbbV9DstRxNszC9Hnh9q34nmi2vz1D2\n3EYR9wLgd4ao+43yWdNLeWealWvNdN9Ms3W3CvhrmluvvLFVfwOwZav8JJo9hvY0ptAkv6/QrNhO\npdlKXgVsT7PVu5jmB7yoIqZby3zqve4pcd3S+/9VtP0BTeK7E/hReX97bxkAnt5q8/T2dIE/HDDN\nF9IcdmvPkymt8pY0CXQKzZb51q26rYHlrfIymsOdvfLurWXtDuBJA9re0VtGaQ7xLKO5WPWNvXFp\n+hl/CHx/sPlR/n4P+GD5/+1c/hdfbI33Q+Bfy2fsR7PBUbv8fZgmce0PHFReB7bqh/zdlP/rcPPr\n5orlZRbwcZqLe/+ltu2AZeTO8v4HvWVowLhbANe3ytcOnEcDxj9jwDw5sD1PRoirt75ZBjyVZqXf\nWw7eXf6XHyivm2kO07bbP33A/+Kgys89tiyDZ5bXHcAxNW3br01tD2OZpKuAPYDT1XQWt7eo3w9c\nAewm6UKa3fu3lrpTWHeI5eDWIZae42i2PJ/UmuZbJP2SdbusW9EcinmDJDxC5m85AHhrOSz2a9Zt\nOext+xesO6aJm1un3F0zUdsXSlpGk+wEvNb2D1qj7ECz0Pa2/J9Ca2tb0v+jOYxwDfAh2/9WquZJ\nWu5ma/oDkq4FnkYzb0dS1bG9AW0/DNxUYhLNj/nxPR3bn2qPbHsZ0D6OuwPNfLi/lLelSUCPSvop\n8G+SLi11rwX+qdX2L4FvS7qufPZLWXdjzAuBf5X01VL+feCictjhdpoVwett/3hAfI+VQyRvK3tk\nXyhVR9AkF4ATvO5wzd3AHDU37Ow5i2YZO5pmT+U6SYtpNgJMkxSPU9O/s97yR7N3tZgBfVXAy8u0\nh/vdfHaE+XWZpMPcXKS7Hkkfpdny/yFwMXCmW3tvw7UthtpjH8x0mg2+nr8AFpX/4697A72uz+Ue\nnjhPrqf5nY1kqaTtafbSlwE/L22x/beSvknZywaOs31Tr6EG7zes+lw3fSRLWfd/e73H8HyhTa0P\nYwtgX5qV+tY0d2TcxaXTqIyzE82xz5tp9jbutb1Y0hLbL5J0M82dcX8t6Tbbv13aLbf9vGE++wm7\n5bavq4z72YMNH7jyGG+SjqPZkmmvXD/g5nBAr/5qmq3W9vdaLOlpLv0Z/UbSb9Js+d5Es8L/L9uL\nK9ueALwX+Cbr5smHgM/RzKvP0SQCgMXtH3RpvxNNkriJ1vJV6mbSbKRA05+zlErl8NC2rDtkMYWm\nPwwqDkuUaTyFZsPnz2hWOnsON76b/rlB+6psv75Mc6TfzX4MMb9a3+nXNIe62sf6/5BmQ2lPmt9y\nz+Ws6zsZtG3FfGj3jxn4b5q+zktK/VU0K/KB/ZW9kySGnScVn78DzbpiBk2f0/dqls8N/dzxsKnt\nYRzPIGfuAL2zDAY9s4cm664qmf8rwNcl3UfTSd1zvaQZg2XlDcn80H1iGOZzPyPpSpq+nh/Q9D20\nj+VuSdMx94T51cfJYrj/8Yhsf1rS11g3T64CVrnpiPzzMtqNY/nskiCqk8SAuLYb60aJmrPdXkqz\ngr2e5iy7b1Uud0P1VfUM+7tx88jlQefXUN+peAz4Bk88C+/xfoIR+hyHNMTntrecf9P284eZxEjz\nZEgbuHyO+XPHzWiPYfXzi2aLYBvK8U2aQ0iX1Na3xjuIpiN7q9awH9B0GPaOtz9+LLR2uv32YvDj\nz98Y7fzqp9eGxjzSPOnys8cQ1zWVbY+g2Uvcn3K8nfpj7pdS2Vc12O9mrN+p4re8IfNjpOX+owzT\nTziaeTKey8iGfO54vSbsgybkyzSnT0KTubcu72+rrR9h2s8e7LWh053k+TXSj3Kj+14bGvMG/qA7\nm18bGNdgtHBOAAACWklEQVTbhltBjiKGUSWEDflOFb/lDZkfIy33D9Ls4fwSeKCUHxiPeTJey8h4\n/y9qX5vaIamRDiuNVD8kD7/7PubpTrINOtzQpzY05g3Z7e9yfm1IXO9k+BM6qriyT24UhvtOI83L\nDZkfw7b18IfK1jOGeTIuy0gH/4sqm1Snd5ukgyhn7th+eLT1XX1uPylnrxwHvIvmGOp9NKdzHjbI\nuBvN9+oZS8yjmSfj/dldxTVSx/Rkqf1Og83LDZwfw7Ydqk/SdlWfZK2N8je1qSaMGJ2NceHtWr/O\nk9HGNV5JsEsbMq/Hu20/nI3Ur5IwIjYj/ZoE+0m/7pH1g02tDyMihjFZx743Mhtj392EyB5GRMQQ\nske2viSMiIiokgcoRURElSSMiIiokoQRMY4k7abmsZhPL+UdSnlfSd+VdJukWyQdOdmxRoxW+jAi\nxpmaJy4+1/ZcNQ+/uhP4Ms3dVP9Dzd10l9E8jOlnw0wqoq8kYUSMM0lPokkI59Hcx2lf248MGOd7\nwBG2/2OQSUT0pVyHETHObD+i5nG9V9Dc9XRgsphF87CtHw7WPqJfpQ8johuzaZ5+t95zFSTtTPOc\n+ePceu52xMYgCSNinEnaFziE5qZ1f1KSBJKeSvPEuL+0fcMkhhgxJkkYEeNIkoBzgHfZ/k/gb4CP\nSdqK5gE4n7X9pcmMMWKs0ukdMY4kzQVeYfvIUp4CLKF5Otp7gdtao7/V9s0TH2XE2CRhRERElRyS\nioiIKkkYERFRJQkjIiKqJGFERESVJIyIiKiShBEREVWSMCIiokoSRkREVPn/jNRluvTZ570AAAAA\nSUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xacbe00b7f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEGCAYAAACKB4k+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEpdJREFUeJzt3XuwHnV9x/H3xyjEegFaglpCDAjFpoxaGgEtxSsVREGt\ntnhtvTRDO9RYewE7Qm2tF1q1DhWJqaXirdQbhUoULYNS7wSwWtQ4mXgBtBUtSkSRRL7949mzPsbk\nnD2H7LN5wvs1c+Y8v909ez4whE/28ttNVSFJEsBdhg4gSdp1WAqSpJalIElqWQqSpJalIElqWQqS\npJalIElqWQqSpJalIElq3XXoAPO177771vLly4eOIUlT5aqrrvp2VS2Za7upK4Xly5ezfv36oWNI\n0lRJ8rUu23n6SJLUshQkSS1LQZLUshQkSS1LQZLUshQkSS1LQZLUshQkSa2pm7w2X8tPv6TX/X/1\n1Sf0un9JmiSPFCRJLUtBktSyFCRJLUtBktSyFCRJLUtBktSyFCRJLUtBktSyFCRJLUtBktSyFCRJ\nLUtBktSyFCRJLUtBktSyFCRJLUtBktSyFCRJLUtBktSyFCRJLUtBktSyFCRJLUtBktSyFCRJLUtB\nktTqtRSSHJdkQ5KNSU6fZbuHJtma5Kl95pEkza63UkiyCDgHOB5YATw9yYodbHcW8KG+skiSuunz\nSOEIYGNVbaqq24ALgJO2s90fAe8FvtVjFklSB32Wwv7AdWPj65tlrST7A08Gzp1tR0lWJVmfZP2N\nN96404NKkkaGvtD8euC0qrp9to2qam1VrayqlUuWLJlQNEm687lrj/u+AThgbLy0WTZuJXBBEoB9\ngccn2VpV/9ZjLknSDvRZClcChyQ5kFEZnAw8Y3yDqjpw5nOStwDvtxAkaTi9lUJVbU1yKnApsAg4\nr6quTXJKs35NX79bkrQwfR4pUFXrgHXbLNtuGVTV7/WZRZI0t6EvNEuSdiGWgiSpZSlIklqWgiSp\nZSlIklqWgiSpZSlIklqWgiSpZSlIklqWgiSpZSlIklqWgiSpZSlIklqWgiSpZSlIklqWgiSpZSlI\nklqWgiSpZSlIklpzlkKSeyS5S/P5l5KcmORu/UeTJE1alyOFK4DFSfYHPgQ8G3hLn6EkScPoUgqp\nqh8ATwHeWFVPA36l31iSpCF0KoUkDwOeCVzSLFvUXyRJ0lC6lMJq4CXAhVV1bZKDgMv7jSVJGsJd\nZ1uZZBFwYlWdOLOsqjYBL+w7mCRp8mY9UqiqHwNHTyiLJGlgsx4pNK5JcjHwbuCWmYVV9b7eUkmS\nBtGlFBYD3wEePbasAEtBknYzc5ZCVT13EkEkScObsxSSLAaez2huwuKZ5VX1vB5zSZIG0OWW1LcB\n9wUeB3wUWAps7jOUJGkYXUrh4Ko6A7ilqs4HTgCO7DeWJGkIXUphS/P9u0kOA/YC9usvkiRpKF3u\nPlqbZB/gpcDFwD2BM3tNJUkaRJe7j97cfLwCOKjfOJKkIXV5n8Irk+w9Nt4nyd902XmS45JsSLIx\nyenbWX9Sks8l+WyS9UmcPS1JA+pyTeH4qvruzKCqbgIeP9cPNc9NOgc4HlgBPD3Jim02uwx4cFU9\nBHge8GYkSYPpUgqLkuw5M0hyd2DPWbafcQSwsao2VdVtwAXASeMbVNX3q6qa4T0YzZSWJA2ky4Xm\ndwCXJfnnZvxc4PwOP7c/cN3Y+Hq2cytrkicDr2J0R9MJHfYrSepJlwvNZyX5L+CxzaKXV9WlOytA\nVV0IXJjkGODlY7+nlWQVsApg2bJlO+tXS5K20eVIgar6IPDBee77BuCAsfHSZtmOfscVSQ5Ksm9V\nfXubdWuBtQArV670FJMk9aTLNYWFuhI4JMmBSfYATmY0z6GV5OAkaT4fzuhaxXd6zCRJmkWnI4WF\nqKqtSU4FLmX0Tufzmtd5ntKsXwP8FvCcJFuAHwK/M3bhWZI0YTsshSSXVdVjkpxVVactZOdVtQ5Y\nt82yNWOfzwLOWsi+JUk732xHCvdL8nDgxCQXABlfWVVX95pMkjRxs5XCmcAZjC4Qv26bdcVPv4lN\nkrQb2GEpVNV7gPckOaOqXj7BTJKkgXSZp/DyJCcCxzSLPlJV7+83liRpCF0eiPcqYDXwheZrdZJX\n9h1MkjR5XW5JPQF4SFXdDpDkfOAa4C/6DCZJmryuk9f2Hvu8Vx9BJEnD63Kk8CrgmiSXM7ot9Rjg\nZ96NIEmafl0uNP9Lko8AD20WnVZV/9NrKknSILo+EO+bbPPcIknS7qfPB+JJkqaMpSBJanUqhSRH\nJ3lu83lJkgP7jSVJGkKXyWt/CZwGvKRZdDfg7X2GkiQNo8uRwpOBE4FbAKrqG8C9+gwlSRpGl7uP\nbquqSlIASe7RcyaNWX76Jb3u/6uvPqHX/UuaLl2OFN6V5E3A3kl+H/gP4B/7jSVJGkKXyWuvSXIs\ncDNwKHBmVX2492SSpInrOnntw4BFIEm7uTlLIclmRm9aG/c9YD3wJ1W1qY9gkqTJ63Kk8HrgeuCd\njB6IdzLwAOBq4DzgkX2FkyRNVpcLzSdW1ZuqanNV3VxVa4HHVdW/Avv0nE+SNEFdSuEHSX47yV2a\nr98Gbm3WbXtaSZI0xbqUwjOBZwPfAv63+fysJHcHTu0xmyRpwrrckroJeOIOVn9s58aRJA2py91H\ni4HnA78CLJ5ZXlXP6zGXJGkAXU4fvQ24L/A44KPAUmBzn6EkScPoUgoHV9UZwC1VdT5wAnBkv7Ek\nSUPoUgpbmu/fTXIYsBewX3+RJElD6TJ5bW2SfYCXMnpP8z2BM3pNJUkaRJdSuKyqbgKuAA4C8M1r\nkrR76nL66L3bWfaenR1EkjS8HR4pJHkgo9tQ90rylLFV92bs1lRJ0u5jttNHhwJPAPbmpyevbQZ+\nv89QkqRh7LAUquoi4KIkD6uqT04wkyRpIF0uNG9M8hfA8vHtndEsSbufLheaL2I0N+E/gEvGvuaU\n5LgkG5JsTHL6dtY/M8nnknw+ySeSPHg+4SVJO1eXI4Wfq6rT5rvjJIuAc4BjGb2k58okF1fVF8Y2\n+wrwiKq6KcnxwFqcLS1Jg+lypPD+JI9fwL6PADZW1aaqug24ADhpfIOq+kQzBwLgU4yeqyRJGkiX\nUljNqBhuTXJzks1Jbu7wc/sD142Nr2+W7cjzgQ902K8kqSdd3qdwr75DJHkUo1I4egfrVwGrAJYt\nW9Z3HEm605rzSCEjz0pyRjM+IMkRHfZ9A3DA2Hhps2zb/T8IeDNwUlV9Z3s7qqq1VbWyqlYuWbKk\nw6+WJC1El9NHbwQeBjyjGX+f0QXkuVwJHJLkwCR7ACczeqBeK8ky4H3As6vqy51TS5J60eXuoyOr\n6vAk1wA0dwrtMdcPVdXWJKcClwKLgPOq6tokpzTr1wBnAr8AvDEJwNaqWrnAfxZJ0h3UpRS2NLeX\nFkCSJcDtXXZeVeuAddssWzP2+QXACzqnlST1qsvpo7OBC4H9krwC+Bjwyl5TSZIG0eXuo3ckuQp4\nDBDgSVX1xd6TSZImbs5SSHIUcG1VndOM753kyKr6dO/pJEkT1eX00bmM7jia8f1mmSRpN9OlFFJV\nNTOoqtvpdoFakjRlupTCpiQvTHK35ms1sKnvYJKkyetSCqcAD2c0G/l6Rk8xXdVnKEnSMGY9DdTM\nT3hmVZ08oTySpAHNeqRQVT8Gnj6hLJKkgXW5YPzxJG8A/hW4ZWZhVV3dWypJ0iC6lMJDmu9/Pbas\ngEfv/DiSpCF1mdH8qEkEkSQNr8v7FO6T5J+SfKAZr0jy/P6jSZImrcstqW9h9PjrX2zGXwZe1Fcg\nSdJwupTCvlX1LprHZVfVVuDHvaaSJA2iSynckuQX+Mn7FI4CvtdrKknSILrcffRiRq/RfECSjwNL\ngKf2mkqSNIgudx9dneQRwKGM3qewoaq29J5MkjRxXd6nsBj4Q+BoRqeQ/jPJmqq6te9wkqTJ6nL6\n6K3AZuAfmvEzgLcBT+srlCRpGF1K4bCqWjE2vjzJF/oKJEkaTpe7j65u7jgCIMmRwPr+IkmShtLl\nSOHXgE8k+XozXgZsSPJ5oKrqQb2lkyRNVJdSOK73FJKkXUKXW1K/NokgkqThdbmmIEm6k7AUJEkt\nS0GS1LIUJEktS0GS1LIUJEktS0GS1LIUJEktS0GS1LIUJEktS0GS1Oq1FJIcl2RDko1JTt/O+gcm\n+WSSHyX50z6zSJLm1uUpqQuSZBFwDnAscD1wZZKLq2r8BT3/B7wQeFJfOSRJ3fVWCsARwMaq2gSQ\n5ALgJKAthar6FvCtJCf0mEMDWn76Jb3u/6uv9j8daWfq8/TR/sB1Y+Prm2WSpF3UVFxoTrIqyfok\n62+88cah40jSbqvPUrgBOGBsvLRZNm9VtbaqVlbVyiVLluyUcJKkn9VnKVwJHJLkwCR7ACcDF/f4\n+yRJd1BvF5qramuSU4FLgUXAeVV1bZJTmvVrktwXWA/cG7g9yYuAFVV1c1+5JEk71ufdR1TVOmDd\nNsvWjH3+H0anlSRJu4CpuNAsSZoMS0GS1LIUJEktS0GS1LIUJEktS0GS1LIUJEktS0GS1LIUJEmt\nXmc0S9PO90HozsYjBUlSy1KQJLUsBUlSy1KQJLW80CztprxIroXwSEGS1LIUJEktS0GS1LIUJEkt\nS0GS1LIUJEktS0GS1HKegqRdkvMshuGRgiSpZSlIklqWgiSpZSlIklqWgiSpZSlIklqWgiSpZSlI\nklqWgiSpZSlIklo+5kKSejCtj+nwSEGS1LIUJEmtXkshyXFJNiTZmOT07axPkrOb9Z9LcnifeSRJ\ns+utFJIsAs4BjgdWAE9PsmKbzY4HDmm+VgHn9pVHkjS3Po8UjgA2VtWmqroNuAA4aZttTgLeWiOf\nAvZOcr8eM0mSZpGq6mfHyVOB46rqBc342cCRVXXq2DbvB15dVR9rxpcBp1XV+m32tYrRkQTAocCG\nXkKP7At8u8f99838w5rm/NOcHcw/l/tX1ZK5NpqKW1Krai2wdhK/K8n6qlo5id/VB/MPa5rzT3N2\nMP/O0ufpoxuAA8bGS5tl891GkjQhfZbClcAhSQ5MsgdwMnDxNttcDDynuQvpKOB7VfXNHjNJkmbR\n2+mjqtqa5FTgUmARcF5VXZvklGb9GmAd8HhgI/AD4Ll95ZmHiZym6pH5hzXN+ac5O5h/p+jtQrMk\nafo4o1mS1LIUJEktS0GS1LIUplSStzXfVw+d5Y5Ksk+SI5IcM/M1dCZNh+bOxWclObMZL0tyxNC5\nukqyOMmLk7wvyXuT/HGSxYNm8kIzJDkfWF1V323G+wCvrarnDZtsx5J8AXgs8AHgkUDG11fV/w0Q\na96SvABYzWiOymeBo4BPVtWjBw02hyQvnm19Vb1uUlnuiB38c3wPuKqqPjvpPPOV5FzgduDRVfXL\nzZ/dD1XVQweO1kmSdwGbgbc3i54B7F1VTxsq01TMaJ6AB80UAkBV3ZTkV4cM1MEa4DLgIOAqfroU\nqlk+DVYDDwU+VVWPSvJA4JUDZ+riXs33Qxnln5mD80TgM4MkWpiVzde/N+MnAJ8DTkny7qr628GS\ndXNkVR2e5Bpo/+zuMXSoeTisqsYfFHp58xe+wVgKI3dJsk9V3QSQ5OfZxf/dVNXZwNlJzq2qPxg6\nzx1wa1XdmoQke1bVl5IcOnSouVTVXwEkuQI4vKo2N+OXAf2+cmvnWsoo//cBkvwlo/zHMPrLxq5e\nCluaJzIXQJIljI4cpsXVSY5qHghKkiOB9XP8TK926f/xTdBrgU8meXczfhrwigHzdDblhQBwfZK9\ngX8DPpzkJuBrA2eaj/sAt42Nb2uWTYv9gB+NjbcA96mqHyb50Q5+ZldyNnAhsF+SVwBPBV46bKS5\nJfk8oyK7G/CJJF9vxvcHvjRkNksBqKq3JlkPzJzHfkpVDXoId2dRVU9uPr4syeXAXsAHB4w0X28F\nPpPkwmb8JOAtw8WZt3cAn05yUTN+IvDOJPcAdvk/A1X1jiRXAY9hdAr1SVX1xYFjdfGEoQPsiBea\npTuoeWPgbzTDK6rqmiHzzFeSlcCvN8OPb/voet25WAqSpJbzFCRJLUtBktSyFKR5SnJAkq80ty7P\nzMj+SpJHJLk6yWeTtI+Jl6aJ1xSkBUjy58DBVbUqyZuArzK6tTlV9aMk9wT+G3h4VX1jwKjSvHhL\nqrQwfw9cleRFwNHAqVW1ZWz9nngkrilkKUgLUFVbkvwZozkVvzlTCEkOYDQj+GDgzzxK0LTxbzLS\nwh0PfBM4bGZBVV1XVQ9iVAq/m2SaZjdLloK0EEkeAhzL6Kmuf5zkfuPrmyOE/+Ynk9qkqWApSPOU\nJMC5wIuq6uvA3wGvSbI0yd2bbfZhdK1hw3BJpfnz7iNpnpKsAh5TVb/TjBcBVwIXAb/F6MFmAd5Q\nVWsHCyotgKUgSWp5+kiS1LIUJEktS0GS1LIUJEktS0GS1LIUJEktS0GS1Pp/K9jYINwrka8AAAAA\nSUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xacbf2152e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEGCAYAAAB2EqL0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFZdJREFUeJzt3Xu03WV95/H3hwDSIgJKQIcQA5XipBQcGgl2qIoWBLFg\nO2oRxDUIZjEtS9TpDGkrdEZHpTO2a4YpEjMOLdo6eCmMqUQQqJYqMpNwGSgIrqyIkowOoFwilELg\nO3/s39HNaS7PCft3zk7O+7XWWXs/z+/37Od7TrLO5/zuqSokSdqanWa6AEnS9sHAkCQ1MTAkSU0M\nDElSEwNDktTEwJAkNTEwJElNDAxJUhMDQ5LUZOeZLmCU9tlnn1qwYMFMlyFJ242bb775waqa27Lu\nDhUYCxYsYPXq1TNdhiRtN5J8t3Vdd0lJkpoYGJKkJgaGJKlJr4GR5Pgk9yRZk2TpJpafluT2JHck\nuTHJ4UPL7u36b0vigQlJmmG9HfROMge4GDgWWAesSrKiqu4aWu07wGuq6qEkJwDLgcVDy4+pqgf7\nqlGS1K7PLYwjgTVVtbaqngQuB04eXqGqbqyqh7rmTcC8HuuRJD0HfQbG/sB9Q+11Xd/mnAl8eahd\nwHVJbk6yZHODkixJsjrJ6gceeOA5FSxJ2ryxuA4jyTEMAuPooe6jq2p9kn2Ba5PcXVU3TB5bVcsZ\n7Mpi0aJFPm9WknrSZ2CsBw4Yas/r+p4lyWHAJ4ETquqHE/1Vtb57vT/JlQx2cf2jwGi1YOlV2zTu\n3gtP3NYpJWmH0mdgrAIOTnIgg6A4BTh1eIUk84ErgNOr6ttD/bsDO1XVhu79ccAHe6x1pAwnSTui\n3gKjqjYmOQe4BpgDXFpVdyY5u1u+DLgAeBHw8SQAG6tqEbAfcGXXtzPwmaq6uq9aJUlb1+sxjKpa\nCayc1Lds6P1ZwFmbGLcWOHxyvyRp5niltySpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoY\nGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoY\nGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoY\nGJKkJgaGJKlJr4GR5Pgk9yRZk2TpJpafluT2JHckuTHJ4a1jJUnTq7fASDIHuBg4AVgIvD3Jwkmr\nfQd4TVX9IvAhYPkUxkqSplGfWxhHAmuqam1VPQlcDpw8vEJV3VhVD3XNm4B5rWMlSdOrz8DYH7hv\nqL2u69ucM4EvT3VskiVJVidZ/cADDzyHciVJWzIWB72THMMgMM6b6tiqWl5Vi6pq0dy5c0dfnCQJ\ngJ17/Oz1wAFD7Xld37MkOQz4JHBCVf1wKmMlSdOnzy2MVcDBSQ5MsitwCrBieIUk84ErgNOr6ttT\nGStJml69bWFU1cYk5wDXAHOAS6vqziRnd8uXARcALwI+ngRgY7d7aZNj+6pVkrR1fe6SoqpWAisn\n9S0ben8WcFbrWEnSzBmLg96SpPFnYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJlsNjCS7J9mp\ne//zSU5Kskv/pUmSxknLFsYNwG5J9ge+ApwO/FmfRUmSxk9LYKSqHgd+A/h4Vb0V+IV+y5IkjZum\nwEjyKuA04Kqub05/JUmSxlFLYJwL/C5wZXfzwIOAr/ZbliRp3Gzx5oPds7VPqqqTJvqqai3wnr4L\nkySNly1uYVTV08DR01SLJGmMtdze/NYkK4DPA49NdFbVFb1VJUkaOy2BsRvwQ+B1Q33F4El5kqRZ\nYquBUVVnTEchkqTxttXASLIbcCaDay92m+ivqnf1WJckacy0nFb7aeDFwBuAvwHmARv6LEqSNH5a\nAuNlVXU+8FhVXQacCCzutyxJ0rhpCYynuteHkxwK7Ans219JkqRx1HKW1PIkewMfAFYAzwcu6LUq\nSdLYaTlL6pPd2xuAg/otR5I0rlqeh/GRJHsNtfdO8h/6LUuSNG5ajmGcUFUPTzSq6iHgjf2VJEka\nRy2BMSfJ8yYaSX4GeN4W1pck7YBaDnr/BXB9kj/t2mcAl/VXkiRpHLUc9P7DJP8H+NWu60NVdU2/\nZUmSxk3LFgZVdTVwdc+1SJLGWMsxDEmSDAxJUpvNBkaS67vXP5y+ciRJ42pLxzBekuSXgZOSXA5k\neGFV3dJrZZKksbKlwLgAOJ/B7cz/eNKy4tlP4NukJMcD/wWYA3yyqi6ctPzlwJ8CRwC/X1UfG1p2\nL4PbqD8NbKyqRVubT5LUn80GRlV9AfhCkvOr6kNT/eAkc4CLgWOBdcCqJCuq6q6h1X4EvAd482Y+\n5piqenCqc0uSRq/lOowPJTkJeHXX9bWq+lLDZx8JrKmqtQDdbq2TgZ8ERlXdD9yf5MQpVy5JmlYt\nNx/8KHAug1/0dwHnJvlIw2fvD9w31F7X9bUq4LokNydZsoX6liRZnWT1Aw88MIWPlyRNRcuFeycC\nr6iqZwCSXAbcCvxen4UBR1fV+iT7Atcmubuqbpi8UlUtB5YDLFq0qHquaSwtWHrVNo2790I37CS1\na70OY6+h93s2jlkPHDDUntf1Namq9d3r/cCVDHZxSZJmSMsWxkeBW5N8lcGpta8GljaMWwUcnORA\nBkFxCnBqS1FJdgd2qqoN3fvjgA+2jJUk9aPloPf/SPI14JVd13lV9YOGcRuTnANcw+C02kur6s4k\nZ3fLlyV5MbAaeAHwTJL3AguBfYArk0zU+JnuflaSpBnSevPB7zN4nveUVNVKYOWkvmVD73/AYFfV\nZI8Ch091PklSf7yXlCSpiYEhSWrSFBhJjk5yRvd+bncgW5I0i7RcuPcHwHnA73ZduwB/3mdRkqTx\n07KF8evAScBjAFX1f4E9+ixKkjR+WgLjyaoqBrfqmLhGQpI0y7QExueSfALYK8m7geuA/9ZvWZKk\ncdNy4d7HkhzL4NqIQ4ALqura3iuTJI2V1gv3rgUMCUmaxbYaGEk20B2/GPIIg1t6/OuJ511IknZs\nLVsY/5nBsyw+w+Dmg6cAPwfcAlwKvLav4iRJ46PloPdJVfWJqtpQVY92z594Q1V9Fti75/okSWOi\nJTAeT/K2JDt1X28DnuiWzcoHFknSbNQSGKcBpwP3A/+ve/+OJD8DnNNjbZKkMdJyWu1a4Nc2s/jr\noy1HkjSuWs6S2g04E/gFYLeJ/qp6V491SZLGTMsuqU8DLwbeAPwNgwcebeizKEnS+GkJjJdV1fnA\nY1V1GXAisLjfsiRJ46YlMJ7qXh9OciiwJ7BvfyVJksZRy4V7y5PsDXyAwXO9nw+c32tVkqSx0xIY\n11fVQ8ANwEEAPnFPkmafll1Sf7mJvi+MuhBJ0njb7BZGkpczOJV2zyS/MbToBQydXqvZZ8HSq7Zp\n3L0XnjjiSiRNpy3tkjoEeBOwF8++cG8D8O4+i5IkjZ/NBkZVfRH4YpJXVdU3p7Em6VncopHGQ8tB\n7zVJfg9YMLy+V3pL0uzSEhhfBP6WwbO8n+63HEnSuGoJjJ+tqvN6r0SSNNZaTqv9UpI39l6JJGms\ntQTGuQxC44kkjybZkOTRvguTJI2Xludh7DEdhUiSxttWtzAy8I4k53ftA5Ic2X9pkqRx0rJL6uPA\nq4BTu/aPgYtbPjzJ8UnuSbImydJNLH95km8m+YckvzOVsZKk6dUSGIur6reBJwC6GxHuurVBSeYw\nCJYTgIXA25MsnLTaj4D3AB/bhrGSpGnU9DyM7hd4ASSZCzzTMO5IYE1Vra2qJ4HLgZOHV6iq+6tq\nFT995kbzWEnS9GoJjIuAK4F9k3wY+DrwkYZx+wP3DbXXdX0tnstYSVIPWs6S+oskNwOvBwK8uaq+\n1XtljZIsAZYAzJ8/f4arkaQdV8tZUkcB66vq4qr6E2B9kpZneq8HDhhqz+v6WjSPrarlVbWoqhbN\nnTu38eMlSVPVskvqEgZnRk34cde3NauAg5McmGRX4BQGj3ht8VzGSpJ60HIvqVRVTTSq6pkkLbuy\nNiY5B7gGmANcWlV3Jjm7W74syYuB1QweyvRMkvcCC6vq0U2NnfJ3J0kamZbAWJvkPfx0q+K3gLUt\nH15VK4GVk/qWDb3/AYPdTU1jJUkzp2WX1NnALzM4hrAOWEx3kFmSNHtscQuju/7itKo6ZZrqkSSN\nqS1uYVTV08Dbp6kWSdIYazmG8Y0kfwJ8FnhsorOqbumtKknS2GkJjFd0rx8c6ivgdaMvR5I0rlpO\njz1mOgqRJI23liu990vy35N8uWsvTHJm/6VJksZJy2m1f8bgArp/0rW/Dby3r4IkSeOpJTD2qarP\n0d3SvKo2Ak/3WpUkaey0BMZjSV7ET5+HcRTwSK9VSZLGTstZUu9ncOO/n0vyDWAu8JZeq5IkjZ2W\ns6RuSfIa4BAGz8O4p6omPyFPkrSD22pgJNmNwQ0Hj2awW+pvkyyrqif6Lk6SND5adkl9CtgA/Neu\nfSrwaeCtfRUlSRo/LYFxaFUtHGp/NcldfRUkSRpPLWdJ3dKdGQVA93jW1f2VJEkaRy1bGL8E3Jjk\ne117PnBPkjuAqqrDeqtOkjQ2WgLj+N6rkCSNvZbTar87HYVIksZbyzEMSZIMDElSGwNDktTEwJAk\nNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTQwMSVITA0OS1MTAkCQ1MTAkSU0MDElSk14DI8nxSe5J\nsibJ0k0sT5KLuuW3JzliaNm9Se5IclsSn/AnSTOs5QFK2yTJHOBi4FhgHbAqyYqqGn4e+AnAwd3X\nYuCS7nXCMVX1YF81SpLa9bmFcSSwpqrWVtWTwOXAyZPWORn4VA3cBOyV5CU91iRJ2kZ9Bsb+wH1D\n7XVdX+s6BVyX5OYkS3qrUpLUpLddUiNwdFWtT7IvcG2Su6vqhskrdWGyBGD+/PnTXaN2QAuWXrVN\n4+698MQRVyKNlz63MNYDBwy153V9TetU1cTr/cCVDHZx/SNVtbyqFlXVorlz546odEnSZH0Gxirg\n4CQHJtkVOAVYMWmdFcA7u7OljgIeqarvJ9k9yR4ASXYHjgP+rsdaJUlb0dsuqaramOQc4BpgDnBp\nVd2Z5Oxu+TJgJfBGYA3wOHBGN3w/4MokEzV+pqqu7qtWSdLW9XoMo6pWMgiF4b5lQ+8L+O1NjFsL\nHN5nbdK48JiJthde6S1JamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBI\nkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBI\nkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpyc4zXYCk6bVg6VXbNO7eC08ccSXa3hgYknpl\nQO043CUlSWriFoakHYpbNP0xMCTpOdiWgNpew6nXXVJJjk9yT5I1SZZuYnmSXNQtvz3JEa1jJUnT\nq7ctjCRzgIuBY4F1wKokK6rqrqHVTgAO7r4WA5cAixvHStKsMtO72/rcwjgSWFNVa6vqSeBy4ORJ\n65wMfKoGbgL2SvKSxrGSpGmUqurng5O3AMdX1Vld+3RgcVWdM7TOl4ALq+rrXft64DxgwdbGDn3G\nEmBJ1zwEuGcbyt0HeHAbxm2L6ZzL+ZzP+WbPfNs610uram7Litv9Qe+qWg4sfy6fkWR1VS0aUUlj\nM5fzOZ/zzZ75pmOuPgNjPXDAUHte19eyzi4NYyVJ06jPYxirgIOTHJhkV+AUYMWkdVYA7+zOljoK\neKSqvt84VpI0jXrbwqiqjUnOAa4B5gCXVtWdSc7uli8DVgJvBNYAjwNnbGlsX7XyHHdpjfFczud8\nzjd75ut9rt4OekuSdizeS0qS1MTAkCQ1MTAkSU0MjJ4kmbgYcUOSRyd9PZLkO0l+a6brVJskeyc5\nMsmrJ75muqZRSfJLm+h700zU0ockuyV5f5Irkvxlkvcl2W3Ec3y6ez13lJ/bMG+SvCPJBV17fpIj\ne5tvNh70TvL+TXQ/AtxcVbdNUw0vAm6sqkNG+Jmb+r5+oqr+eFRzzSZJzgLOZXA90G3AUcA3q+p1\nPc33POBfMLjjwU/OZKyqD/Y03y3AO6vq77r224H3VtXiHua6DDi3qh7u2nsDf1RV7xr1XENzfg7Y\nAPx513UqsFdVvXWEc9wF/CrwZeC1QIaXV9WPRjXXpHkvAZ4BXldV/7T7eX6lql7Zx3zb/ZXe22hR\n9/VXXftNwO3A2Uk+X1X/se8CquqHSV474o/do3s9BHglP7125deA/z3iuUjy9ao6OskGYPgvjwBV\nVS8Y8XyT5+l1viHnMvh53lRVxyR5OfCRnuYC+CLdHzDAP/Q4z4S3AF9IcirwK8A7geN6muuwibAA\nqKqHkvyznuaacGhVLRxqf7X7BT9Ky4DrgYMY/LsNB0Z1/X1YXFVHJLkVfvLz3LWnuWZtYMwDjqiq\nHwMk+QPgKuDVDP6xew8MgO4ixVF+3r8HSHIDg+9vQ9f+dwy+v5GqqqO71z22tu6I5puWeTbhiap6\nIglJnldVdycZ2ZbhJsyrquN7/Pxnqaq1SU4B/ifwPeC4qvr7nqbbKcneVfUQQJIX0v/voVuSHNXd\n4JQki4HVo5ygqi4CLkpySVX9q1F+9lY81d3duwCSzGWwxdGL2RoY+/Lsv9yeAvarqr9PMh1/0fVt\nP+DJofaTXZ+2zbokezH4hXptkoeA7/Y4341JfrGq7uhxDpLcwbO32F7I4ELZ/5WEqjqsh2n/CPhm\nks937bcCH+5hnuHvbxcGP9Pvde2XAnf3Mec0hwXARcCVwL5JPsxga/EDfU02W49hnA/8OoNNfxjs\nslnB4D/z8qo6baZqG4Ukvw+8jcF/JIA3A5+tqo/OXFU7hiSvAfYEru5uvT/Kz574Bbczg2fErGXw\nh83ELreR/gJP8tItLa+qXkIxyUJg4vjPX/f1nJuZ+v6mW7eL9PUM/p9cX1Xf6m2u2RgYAEkWAf+8\na36jqka6iTrTMnh64a90zRuq6taZrEdbN1t+wWn7NWsDQ5I0NV6HIUlqYmBIkpoYGNIIJTmgu4r/\nhV177669oLsK9ytJvpXkriQLZrZaaWo8hiGNWJJ/C7ysqpYk+QRwb1V9NMnXgA9X1bVJng88U1WP\nz2ix0hQYGNKIJdmFwQWglwLvBl7B4DTZ5RMXO0rbo9l64Z7Um6p6Ksm/Aa5mcNX0U0l+Hng4yRXA\ngcB1wNKqenoma5WmwmMYUj9OAL4PHNq1d2ZwXczvMLgv1UHAv5yRyqRtZGBII5bkFcCxDO5q+74k\nLwHWAbdV1dqq2sjgNiNHzGCZ0pQZGNIIJQlwCYPbg38P+E/Ax4BVwF7dzeFgcGuMXm6JIfXFg97S\nCCVZAry+qn6za89hEBbvA3ZlcL+yMDgovmTU96OS+mRgSJKauEtKktTEwJAkNTEwJElNDAxJUhMD\nQ5LUxMCQJDUxMCRJTf4/igyqVoMP7FYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xacbf2d56a0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# let's plot how frequently each label appear in the dataset, for each of the categorical variables\n",
    "\n",
    "total_cars = len(data)\n",
    "for col in cols_to_use:\n",
    "    temp_df = pd.Series(data[col].value_counts() / total_cars)\n",
    "    fig = temp_df.sort_values(ascending=False).plot.bar()\n",
    "    fig.set_xlabel(col)\n",
    "    fig.set_ylabel('percentage of cars')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that for each of the above variables, some labels appear for more than 10% of the observations in the dataset, but very many appear in less than 10% or even 5% of the observations. These are infrequent labels (Rare Values), and could cause over-fitting.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((2946, 4), (1263, 4))"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's separate into training and testing set\n",
    "X_train, X_test, y_train, y_test = train_test_split(data[cols_to_use].fillna(0), data.y,\n",
    "                                                    test_size=0.3,\n",
    "                                                    random_state=0)\n",
    "X_train.shape, X_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X1</th>\n",
       "      <th>X2</th>\n",
       "      <th>X3</th>\n",
       "      <th>X6</th>\n",
       "      <th>X1_rare_imp</th>\n",
       "      <th>X1_freq_imp</th>\n",
       "      <th>X2_rare_imp</th>\n",
       "      <th>X2_freq_imp</th>\n",
       "      <th>X3_rare_imp</th>\n",
       "      <th>X3_freq_imp</th>\n",
       "      <th>X6_rare_imp</th>\n",
       "      <th>X6_freq_imp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3059</th>\n",
       "      <td>aa</td>\n",
       "      <td>ai</td>\n",
       "      <td>c</td>\n",
       "      <td>g</td>\n",
       "      <td>aa</td>\n",
       "      <td>aa</td>\n",
       "      <td>rare</td>\n",
       "      <td>as</td>\n",
       "      <td>c</td>\n",
       "      <td>c</td>\n",
       "      <td>g</td>\n",
       "      <td>g</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3014</th>\n",
       "      <td>b</td>\n",
       "      <td>m</td>\n",
       "      <td>c</td>\n",
       "      <td>i</td>\n",
       "      <td>b</td>\n",
       "      <td>b</td>\n",
       "      <td>rare</td>\n",
       "      <td>as</td>\n",
       "      <td>c</td>\n",
       "      <td>c</td>\n",
       "      <td>i</td>\n",
       "      <td>i</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3368</th>\n",
       "      <td>o</td>\n",
       "      <td>f</td>\n",
       "      <td>f</td>\n",
       "      <td>l</td>\n",
       "      <td>rare</td>\n",
       "      <td>aa</td>\n",
       "      <td>rare</td>\n",
       "      <td>as</td>\n",
       "      <td>f</td>\n",
       "      <td>f</td>\n",
       "      <td>l</td>\n",
       "      <td>l</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2772</th>\n",
       "      <td>aa</td>\n",
       "      <td>as</td>\n",
       "      <td>d</td>\n",
       "      <td>j</td>\n",
       "      <td>aa</td>\n",
       "      <td>aa</td>\n",
       "      <td>as</td>\n",
       "      <td>as</td>\n",
       "      <td>rare</td>\n",
       "      <td>c</td>\n",
       "      <td>j</td>\n",
       "      <td>j</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3383</th>\n",
       "      <td>v</td>\n",
       "      <td>e</td>\n",
       "      <td>c</td>\n",
       "      <td>g</td>\n",
       "      <td>rare</td>\n",
       "      <td>aa</td>\n",
       "      <td>rare</td>\n",
       "      <td>as</td>\n",
       "      <td>c</td>\n",
       "      <td>c</td>\n",
       "      <td>g</td>\n",
       "      <td>g</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      X1  X2 X3 X6 X1_rare_imp X1_freq_imp X2_rare_imp X2_freq_imp  \\\n",
       "3059  aa  ai  c  g          aa          aa        rare          as   \n",
       "3014   b   m  c  i           b           b        rare          as   \n",
       "3368   o   f  f  l        rare          aa        rare          as   \n",
       "2772  aa  as  d  j          aa          aa          as          as   \n",
       "3383   v   e  c  g        rare          aa        rare          as   \n",
       "\n",
       "     X3_rare_imp X3_freq_imp X6_rare_imp X6_freq_imp  \n",
       "3059           c           c           g           g  \n",
       "3014           c           c           i           i  \n",
       "3368           f           f           l           l  \n",
       "2772        rare           c           j           j  \n",
       "3383           c           c           g           g  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's automate the replacement of infrequent categories for the label 'rare' in all\n",
    "# categorical variables as we have been doing so far (i.e., by rare or frequent label imputation)\n",
    "\n",
    "for col in cols_to_use:\n",
    "    # find non-rare categories:those are the ones we will keep \n",
    "    total_cars = len(X_train)\n",
    "    temp_df = pd.Series(X_train[col].value_counts() / total_cars)\n",
    "    categories_to_keep = temp_df[temp_df>=0.1].index # non-rare labels\n",
    "    \n",
    "    # group observations with infrequent labels under the most frequent label \n",
    "    # find frequent label\n",
    "    frequent_cat = X_train.groupby(col)[col].count().sort_values().tail(1).index.values[0]\n",
    "       \n",
    "    # group observations with infrequent labels under new label \"Rare\"\n",
    "    # train set\n",
    "    X_train[col+'_rare_imp'] = np.where(X_train[col].isin(categories_to_keep), X_train[col], 'rare')\n",
    "    X_test[col+'_rare_imp'] = np.where(X_test[col].isin(categories_to_keep), X_test[col], 'rare')\n",
    "       \n",
    "    # replace in train set\n",
    "    X_train[col+'_freq_imp'] = np.where(X_train[col].isin(categories_to_keep), X_train[col], frequent_cat)\n",
    "    X_test[col+'_freq_imp'] = np.where(X_test[col].isin(categories_to_keep), X_test[col], frequent_cat)\n",
    "\n",
    "    \n",
    "X_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# let's make lists with the variables after the different imputed methods\n",
    "# or comparatively the original variable\n",
    "# so we can train random forests using these lists\n",
    "\n",
    "cols_to_use = ['X1', 'X2','X3', 'X6']\n",
    "cols_freq = [x+'_freq_imp' for x in cols_to_use]\n",
    "cols_rare = [x+'_rare_imp' for x in cols_to_use]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['ar', 'j']"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[x for x in X_test.X2.unique() if x not in X_train.X2.unique()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are 2 categories in the test set not present in the training set, so this will not allow us to build a random forest, somehow, we need to replace this by something else. Therefore, we see quite clearly, that even if rare label does not boost the performance of the algorithm, it is acually required to account for labels present in the test set that are not present in the training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'as'"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frequent_cat = X_train.groupby('X2')['X2'].count().sort_values().tail(1).index.values[0]\n",
    "frequent_cat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# here I will replace those 2 by the most frequent category\n",
    "#so I can build the trees and be able to compare them with the other\n",
    "# engineered variables\n",
    "\n",
    "X_test['X2'] = np.where(X_test['X2'].isin(['ar', 'j']), frequent_cat, X_test['X2'])\n",
    "[x for x in X_test.X2.unique() if x not in X_train.X2.unique()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Perfect, it worked, now I can continue with the random forest comparison"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train set\n",
      "Random Forests mse: 73.52552796670184\n",
      "Test set\n",
      "Random Forests mse: 147.79694559295893\n"
     ]
    }
   ],
   "source": [
    "# I will build RF using the (almost) original variables\n",
    "labels_to_numbers(X_train, X_test, cols_to_use)\n",
    "train_rf(X_train, y_train, X_test, y_test, cols_to_use)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train set\n",
      "Random Forests mse: 120.70163006877415\n",
      "Test set\n",
      "Random Forests mse: 156.60887148893417\n"
     ]
    }
   ],
   "source": [
    "# or those in which the infrequent labels were grouped under the 'rare' label\n",
    "labels_to_numbers(X_train, X_test, cols_rare)\n",
    "train_rf(X_train, y_train, X_test, y_test, cols_rare)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train set\n",
      "Random Forests mse: 136.65168543109723\n",
      "Test set\n",
      "Random Forests mse: 165.94902177678532\n"
     ]
    }
   ],
   "source": [
    "# or those in which the infrequent labels were grouped under the most frequent label\n",
    "labels_to_numbers(X_train, X_test, cols_freq)\n",
    "train_rf(X_train, y_train, X_test, y_test, cols_freq)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Re-grouping the infrequent categories under the new category 'Rare' outperforms replacing the rare labels by the most frequent one (mse test 156 vs 165)  or using the original variables.\n",
    "\n",
    "### Conclusion\n",
    "\n",
    "Engineering of rare labels causes in many cases an increased performance of tree based machine learning methods. In addition, to get the most value from the data science / data analysis /  machine learning project, it is a good idea, whenever possible, to understand the distribution of observations among the different labels, the cardinality of the variables, and their relationship to the target.\n",
    "\n",
    "**That is all for this demonstration. I hope you enjoyed the notebook, and see you in the next one.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": "block",
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}