16.2_Classification_I.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Classification\n",
    "\n",
    "In this lecture, I will bring together various techniques for feature engineering that we have covered in this course to tackle a classification problem. This would give you an idea of the end-to-end pipeline to build machine learning algorithms for classification.\n",
    "\n",
    "============================================================================\n",
    "\n",
    "## Real Life example: \n",
    "\n",
    "### Predicting Survival on the Titanic: understanding society behaviour and beliefs\n",
    "\n",
    "Perhaps one of the most infamous shipwrecks in history, the Titanic sank after colliding with an iceberg, killing 1502 out of 2224 people on board. Interestingly, by analysing the probability of survival based on few attributes like gender, age, and social status, we can make very accurate predictions on which passengers would survive. Some groups of people were more likely to survive than others, such as women, children, and the upper-class. Therefore, we can learn about the society priorities and privileges at the time.\n",
    "\n",
    "====================================================================================================\n",
    "\n",
    "To download the Titanic data, go ahead to this website:\n",
    "https://www.kaggle.com/c/titanic/data\n",
    "\n",
    "Click on the link 'train.csv', and then click the 'download' blue button towards the right of the screen, to download the dataset. Save it in a folder of your choice.\n",
    "\n",
    "**Note that you need to be logged in to Kaggle in order to download the datasets**.\n",
    "\n",
    "If you save it in the same directory from which you are running this notebook, and you rename the file to 'titanic.csv' then you can load it the same way I will load it below.\n",
    "\n",
    "===================================================================================================="
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Titanic dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# to handle datasets\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "# for text / string processing\n",
    "import re\n",
    "\n",
    "# for plotting\n",
    "import matplotlib.pyplot as plt\n",
    "% matplotlib inline\n",
    "\n",
    "# to divide train and test set\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# feature scaling\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "\n",
    "# for tree binarisation\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "\n",
    "# to build the models\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "import xgboost as xgb\n",
    "\n",
    "# to evaluate the models\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn import metrics\n",
    "\n",
    "pd.pandas.set_option('display.max_columns', None)\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "3            4         1       1   \n",
       "4            5         0       3   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                           Allen, Mr. William Henry    male  35.0      0   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  \n",
       "0      0         A/5 21171   7.2500   NaN        S  \n",
       "1      0          PC 17599  71.2833   C85        C  \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      0            113803  53.1000  C123        S  \n",
       "4      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# load dataset\n",
    "data = pd.read_csv('titanic.csv')\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>892</td>\n",
       "      <td>3</td>\n",
       "      <td>Kelly, Mr. James</td>\n",
       "      <td>male</td>\n",
       "      <td>34.5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>330911</td>\n",
       "      <td>7.8292</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>893</td>\n",
       "      <td>3</td>\n",
       "      <td>Wilkes, Mrs. James (Ellen Needs)</td>\n",
       "      <td>female</td>\n",
       "      <td>47.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>363272</td>\n",
       "      <td>7.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>894</td>\n",
       "      <td>2</td>\n",
       "      <td>Myles, Mr. Thomas Francis</td>\n",
       "      <td>male</td>\n",
       "      <td>62.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>240276</td>\n",
       "      <td>9.6875</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>895</td>\n",
       "      <td>3</td>\n",
       "      <td>Wirz, Mr. Albert</td>\n",
       "      <td>male</td>\n",
       "      <td>27.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>315154</td>\n",
       "      <td>8.6625</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>896</td>\n",
       "      <td>3</td>\n",
       "      <td>Hirvonen, Mrs. Alexander (Helga E Lindqvist)</td>\n",
       "      <td>female</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3101298</td>\n",
       "      <td>12.2875</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Pclass                                          Name     Sex  \\\n",
       "0          892       3                              Kelly, Mr. James    male   \n",
       "1          893       3              Wilkes, Mrs. James (Ellen Needs)  female   \n",
       "2          894       2                     Myles, Mr. Thomas Francis    male   \n",
       "3          895       3                              Wirz, Mr. Albert    male   \n",
       "4          896       3  Hirvonen, Mrs. Alexander (Helga E Lindqvist)  female   \n",
       "\n",
       "    Age  SibSp  Parch   Ticket     Fare Cabin Embarked  \n",
       "0  34.5      0      0   330911   7.8292   NaN        Q  \n",
       "1  47.0      1      0   363272   7.0000   NaN        S  \n",
       "2  62.0      0      0   240276   9.6875   NaN        Q  \n",
       "3  27.0      0      0   315154   8.6625   NaN        S  \n",
       "4  22.0      1      1  3101298  12.2875   NaN        S  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load the dataset for submission (the one on which our model will be evaluated by Kaggle)\n",
    "# it contains exactly the same variables, but not the target\n",
    "\n",
    "submission = pd.read_csv('titanic_submission.csv')\n",
    "submission.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Types of variables (section 2)\n",
    "\n",
    "Let's go ahead and find out what types of variables there are in this dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PassengerId      int64\n",
       "Survived         int64\n",
       "Pclass           int64\n",
       "Name            object\n",
       "Sex             object\n",
       "Age            float64\n",
       "SibSp            int64\n",
       "Parch            int64\n",
       "Ticket          object\n",
       "Fare           float64\n",
       "Cabin           object\n",
       "Embarked        object\n",
       "dtype: object"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's inspect the type of variables in pandas\n",
    "data.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are a mixture of categorical and numerical variables. Numerical are those of type int and float. Categorical those of type object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of PassengerId labels:  891\n",
      "Number  of passengers on the Titanic:  891\n"
     ]
    }
   ],
   "source": [
    "print('Number of PassengerId labels: ', len(data.PassengerId.unique()))\n",
    "print('Number  of passengers on the Titanic: ', len(data))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PassengerId is a unique identifier for each of the passengers. Thus this is not a variable that we can use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 5 categorical variables\n"
     ]
    }
   ],
   "source": [
    "# find categorical variables\n",
    "categorical = [var for var in data.columns if data[var].dtype=='O']\n",
    "print('There are {} categorical variables'.format(len(categorical)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 7 numerical variables\n"
     ]
    }
   ],
   "source": [
    "# find numerical variables\n",
    "numerical = [var for var in data.columns if data[var].dtype!='O']\n",
    "print('There are {} numerical variables'.format(len(numerical)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>113803</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>373450</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                Name     Sex  \\\n",
       "0                            Braund, Mr. Owen Harris    male   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female   \n",
       "2                             Heikkinen, Miss. Laina  female   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female   \n",
       "4                           Allen, Mr. William Henry    male   \n",
       "\n",
       "             Ticket Cabin Embarked  \n",
       "0         A/5 21171   NaN        S  \n",
       "1          PC 17599   C85        C  \n",
       "2  STON/O2. 3101282   NaN        S  \n",
       "3            113803  C123        S  \n",
       "4            373450   NaN        S  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# view of categorical variables\n",
    "data[categorical].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 2 potential mixed type of variables: Cabin and Ticket\n",
    "\n",
    "Cabin and Ticket contain both numbers and letters. We could extract the numerical part and then the non-numerical part and generate 2 variables out of them, to see if that adds value to our predictive models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71.2833</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.9250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass   Age  SibSp  Parch     Fare\n",
       "0            1         0       3  22.0      1      0   7.2500\n",
       "1            2         1       1  38.0      1      0  71.2833\n",
       "2            3         1       3  26.0      0      0   7.9250\n",
       "3            4         1       1  35.0      1      0  53.1000\n",
       "4            5         0       3  35.0      0      0   8.0500"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# view of numerical variables\n",
    "data[numerical].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 3 Discrete variables: Pclass, SibSp and Parch\n",
    "- 2 continuous variables: Fare and Age\n",
    "- 1 Id variable: PassengerId (it is a label for each of the passengers)\n",
    "- 1 binary: Survived (target variable)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pclass  values:  [3 1 2]\n",
      "SibSp  values:  [1 0 3 4 2 5 8]\n",
      "Parch  values:  [0 1 2 5 3 4 6]\n"
     ]
    }
   ],
   "source": [
    "# let's visualise the values of the discrete variables\n",
    "for var in ['Pclass',  'SibSp', 'Parch']:\n",
    "    print(var, ' values: ', data[var].unique())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Types of variables, summary:\n",
    "\n",
    "- 5 categorical variables: from them 2 could be treated as mixed type of variables (numbers and strings)\n",
    "- 7 numerical variables: 3 discrete, 2 continuous, 1 Id, and 1 binary target"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Types of problems within the variables (section 3)\n",
    "\n",
    "#### Missing values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PassengerId    0.000000\n",
       "Survived       0.000000\n",
       "Pclass         0.000000\n",
       "Name           0.000000\n",
       "Sex            0.000000\n",
       "Age            0.198653\n",
       "SibSp          0.000000\n",
       "Parch          0.000000\n",
       "Ticket         0.000000\n",
       "Fare           0.000000\n",
       "Cabin          0.771044\n",
       "Embarked       0.002245\n",
       "dtype: float64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's visualise the percentage of missing values\n",
    "data.isnull().mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Three of the variables contain missing data, Age (~20%), Cabin (~77%) and Embarked (< 1%)\n",
    "\n",
    "#### Outliers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Pclass', 'Age', 'SibSp', 'Parch', 'Fare']"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numerical = [var for var in numerical if var not in['Survived', 'PassengerId']]\n",
    "numerical"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x8b75e20898>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3gAAAFpCAYAAADOe4pZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X+U3XWd5/nnO5VYAYNAUKvTkjbMHLTzg2g3Jd209G4V\nAYofbUK3itSubmapk2yC1tgT9kyC2R1P75y0ibuHMz01Utmki+nMdHcE7E5XFAViJndcOg60iEKS\nkoEV6MAGaEFsgxpJzXv/qG9iJaYgBO79fu+t5+OcnPp+P/fe+r4855aX1/1+v59PZCaSJEmSpOY3\npewAkiRJkqQ3hwVPkiRJklqEBU+SJEmSWoQFT5IkSZJahAVPkiRJklqEBU+SJEmSWoQFT5IkSZJa\nhAVPkiRJklqEBU+SJEmSWoQFT5IkSZJaxNSyA5yMt7/97TlnzpyyY0gN9fLLL/PWt7617BhSQz34\n4IM/yMx3lJ2jWfj5qMnKz0hNRif7GdkUBW/OnDl861vfKjuG1FC1Wo2urq6yY0gNFRFPlZ2hmfj5\nqMnKz0hNRif7GeklmpIkSZLUIix4kiRJktQiLHiSJEmS1CIseJIkSZLUIix4kiRJktQiLHiSJEmS\n1CIseJIkSZLUIix4kiRJktQiLHiSJEmS1CLqWvAi4l9ExN6I2BMRWyNiekTMjIgdEfFY8fPsemaQ\nms3WrVtZsGABixYtYsGCBWzdurXsSJIkVUJPTw9Tpkyhu7ubKVOm0NPTU3YkqXKm1usXR8S7gH8O\nzMvMn0bEHcD1wDxgZ2auj4g1wBpgdb1ySM1k69atrF27lqGhIUZHR2lra6Ovrw+A3t7ektNJklSe\nnp4e7r33XlauXMnVV1/NV7/6VQYHB+np6eGee+4pO55UGfW+RHMqcFpETAVOB/4/YAmwpXh8C3Bt\nnTNITWPdunUMDQ3R3d3N1KlT6e7uZmhoiHXr1pUdTZKkUu3YsYOVK1dy6623MmPGDG699VZWrlzJ\njh07yo4mVUrdzuBl5jMR8X8Bfw/8FLg3M++NiI7MPFA87Vmg40Svj4jlwHKAjo4OarVavaJKlTEy\nMsLo6Ci1Wo2DBw9Sq9UYHR1lZGTEvwFJ0qSWmXzuc587Zuxzn/scg4ODJSWSqqmel2iezdjZuvOA\nl4A7I+Lj45+TmRkReaLXZ+YmYBNAZ2dndnV11SuqVBlz586lra2Nrq4uarUaXV1d7Nq1i7lz5+Lf\ngCRpMosIbr75Zm699dajYzfffDMRUWIqqXrqVvCAy4AnMvMfACLir4HfAZ6LiFmZeSAiZgHP1zGD\n1FTWrl1LX1/f0Xvwdu3aRV9fn5doSpImvcsvv/zo2bqrr76aG2+8kcHBQa644oqSk0nVUs+C9/fA\nb0fE6YxdorkI+BbwMrAUWF/8HK5jBqmpHJlIpb+/n5GREebOncu6deucYEWSNOndc8899PT0sHHj\nRgYHB4kIrrjiCidYkY5Tz3vw7o+ILwHfBg4DDzF2yeUM4I6I6AOeAq6rVwapGfX29tLb23v0Ek1J\nzSsingR+DIwChzOzMyJmArcDc4Angesy84fF828G+orn//PM9L9cpXGOlDk/I6WJ1fMMHpn5WeCz\nxw0fYuxsniRJk0F3Zv5g3P4aTrBcUETMY2w5ofnArwJfj4j3ZOZo4yNLkppVvZdJkCRJx5pouaAl\nwBcz81BmPgE8DlxUQj5JUhOz4EmSVD/J2Jm4B4vlfwAmWi7oXcD+ca99uhiTJOmk1fUSTUmSJrlL\ninVh3wnsiIjvjX/w1ZYLmojrxEocXStW0i+z4EmSVCeZ+Uzx8/mI2MbYJZcTLRf0DDB73MvPLcaO\n/52uE6tJz0lWpIl5iaYkSXUQEW+NiDOObANXAHuA7YwtEwTHLhe0Hbg+Itoj4jzgfOCBxqaWJDU7\nz+BJklQfHcC2iICxz9u/zMy7I+LvOMFyQZm5NyLuAPYxtrzQJ51BU5L0elnwJEmqg8z8PvC+E4y/\nwATLBWXmOmBdnaNJklqYl2hKkiRJUouw4EmSJElSi7DgSZIkSVKLsOBJkiRJUouw4EmSJElSi7Dg\nSZIkSVKLsOBJkiRJUouw4EkVs3XrVhYsWMCiRYtYsGABW7duLTuSJEmSmoQLnUsVsnXrVtauXcvQ\n0BCjo6O0tbXR19cHQG9vb8npJEmSVHWewZMqZN26dQwNDdHd3c3UqVPp7u5maGiIdevWlR1NkiRJ\nTcCCJ1XIyMgIl1xyyTFjl1xyCSMjIyUlkiRJUjOx4EkVMnfuXO67775jxu677z7mzp1bUiJJkiQ1\nEwueVCFr166lr6+PXbt2cfjwYXbt2kVfXx9r164tO5okSZKagJOsSBVyZCKV/v5+RkZGmDt3LuvW\nrXOCFUmSJJ0UC55UMb29vfT29lKr1ejq6io7jiRJkpqIl2hKkiRJUouw4EmSJElSi7DgSZIkSVKL\nsOBJFdPf38/06dPp7u5m+vTp9Pf3lx1JkiRJTcJJVqQK6e/vZ+PGjWzYsIF58+axb98+Vq9eDcDA\nwEDJ6SRJklR1nsGTKmTz5s187GMf47bbbuOaa67htttu42Mf+xibN28uO5okSZKaQN3O4EXEe4Hb\nxw39E+BfAf+hGJ8DPAlcl5k/rFcOqZkcOnSIv/3bv+W2225jdHSUtrY2brjhBg4dOlR2NEmSJDWB\nuhW8zHwUeD9ARLQBzwDbgDXAzsxcHxFriv3V9cohNZOIYMaMGVx11VUcOnSI9vZ2zj//fCKi7GiS\nJElqAo26RHMR8P9m5lPAEmBLMb4FuLZBGaTKy0z27NlDT08P27Zto6enhz179pCZZUeTJElSE2jU\nJCvXA1uL7Y7MPFBsPwt0NCiDVHkRwbx587jnnnvYvn077e3tzJ8/n3379pUdTZIkSU2g7gUvIt4C\nLAZuPv6xzMyIOOGpiYhYDiwH6OjooFar1TOmVAmZyQ9+8AM+97nPcd555/HEE0+wYcMGMtO/AUmS\nJL2mRpzBuwr4dmY+V+w/FxGzMvNARMwCnj/RizJzE7AJoLOzM7u6uhoQVSpXe3s7l112GUNDQ4yM\njDB37lwuu+wyvvSlL+HfgCRJkl5LI+7B6+UXl2cCbAeWFttLgeEGZJCawrJly7j99tu54YYbuOuu\nu7jhhhu4/fbbWbZsWdnRJEmS1ATqegYvIt4KXA78L+OG1wN3REQf8BRwXT0zSM3kyGLmn/nMZ47O\norlixQoXOZckSdJJqWvBy8yXgXOOG3uBsVk1JZ3AwMAAAwMD1Go1L8uUJEnS69KoZRIkSZIkSXVm\nwZMkSZKkFmHBkyRJkqQWYcGTJEmSpBZhwZMkSZKkFmHBkyRJkqQWYcGTJEmSpBZhwZMqZuvWrSxY\nsIBFixaxYMECtm7dWnYkSZIkNYm6LnQu6fXZunUra9euZWhoiNHRUdra2ujr6wOgt7e35HSSJEmq\nOs/gSRWybt06hoaG6O7uZurUqXR3dzM0NMS6devKjiZJkqQm4Bk8qUJGRkb44z/+YxYtWkRmEhEs\nWrSIkZGRsqNJkiSpCXgGT6qQ0047ja9//eusWLGCL3/5y6xYsYKvf/3rnHbaaWVHkyRJUhOw4EkV\n8vLLLzNjxgw++tGPMn36dD760Y8yY8YMXn755bKjSZIkqQlY8KSKueWWW+jv76enp4f+/n5uueWW\nsiNJkiSpSVjwpAqJCB566CH27NnDzp072bNnDw899BARUXY0SZIkNQEnWZEq5PLLL2dwcBCAq6++\nmhtvvJHBwUGuuOKKkpNJkiSpGVjwpAq555576OnpYePGjQwODhIRXHHFFdxzzz1lR5MkSVITsOBJ\ndfZGLq/MTO69997X/Tsy85SPKUmSpOblPXhSnWXmKf179+qvnPJrJUmSNDlZ8CRJqqOIaIuIhyLi\nK8X+zIjYERGPFT/PHvfcmyPi8Yh4NCJ6ykstSWpWFjxJkurr08DIuP01wM7MPB/YWewTEfOA64H5\nwJXArRHR1uCskqQmZ8GTJKlOIuJc4BrgT8cNLwG2FNtbgGvHjX8xMw9l5hPA48BFjcoqSWoNTrIi\nSVL9/BvgXwJnjBvryMwDxfazQEex/S7gv4x73tPF2DEiYjmwHKCjo4NarfYmR5aq7+DBg773pQlY\n8CRJqoOI+D3g+cx8MCK6TvSczMyIeF0zI2XmJmATQGdnZ3Z1nfBXSy2tVqvhe186MQueJEn18UFg\ncURcDUwH3hYRfw48FxGzMvNARMwCni+e/wwwe9zrzy3GJEk6ad6DJ0lSHWTmzZl5bmbOYWzylP+U\nmR8HtgNLi6ctBYaL7e3A9RHRHhHnAecDDzQ4tiSpyXkGT5KkxloP3BERfcBTwHUAmbk3Iu4A9gGH\ngU9m5mh5MSVJzciCJ0lSnWVmDagV2y8AiyZ43jpgXcOCSZJaTl0v0YyIsyLiSxHxvYgYiYiLX22B\nV0mSJEnSqav3PXh/Atydmb8OvI+xhV5PuMCrJEmSJOmNqVvBi4gzgf8OGALIzJ9n5ktMvMCrJEmS\nJOkNqOcZvPOAfwD+fUQ8FBF/GhFvZeIFXiVJkiRJb0A9J1mZCvwm0J+Z90fEn3Dc5ZivtsBrRCwH\nlgN0dHRQq9XqGFWqJt/3kiRJej3qWfCeBp7OzPuL/S8xVvAmWuD1GJm5CdgE0NnZmV1dXXWMKlXQ\n3Xfh+16SJEmvR90u0czMZ4H9EfHeYmgRY2v7TLTAqyRJkiTpDaj3Onj9wF9ExFuA7wP/M2Ol8pcW\neJUkSZIkvTF1LXiZ+R2g8wQPnXCBV0mSJEnSqav3OniSJEmSpAax4EmSJElSi7DgSZIkSVKLsOBJ\nkiRJUouw4EmSJElSi7DgSZIkSVKLsOBJkiRJUouw4EmSJElSi7DgSZIkSVKLsOBJkiRJUouw4EmS\nJElSi7DgSZIkSVKLsOBJkiRJUouw4EmSJElSi7DgSZIkSVKLsOBJkiRJUouw4EmSJElSi7DgSZIk\nSVKLsOBJkiRJUouw4EmSJElSi7DgSZIkSVKLsOBJkiRJUouw4EmSJElSi7DgSZIkSVKLsOBJkiRJ\nUouw4EmSJElSi7DgSZIkSVKLsOBJkiRJUouYWs9fHhFPAj8GRoHDmdkZETOB24E5wJPAdZn5w3rm\nkCRJkqTJoBFn8Loz8/2Z2VnsrwF2Zub5wM5iX5IkSZL0BpVxieYSYEuxvQW4toQMkiRJktRy6l3w\nEvh6RDwYEcuLsY7MPFBsPwt01DmDJEmSJE0Kdb0HD7gkM5+JiHcCOyLie+MfzMyMiDzRC4tCuByg\no6ODWq1W56hS9fi+lyRJ0utR14KXmc8UP5+PiG3ARcBzETErMw9ExCzg+QleuwnYBNDZ2ZldXV31\njCpVz9134ftekiRJr0fdLtGMiLdGxBlHtoErgD3AdmBp8bSlwHC9MkiSJEnSZFLPM3gdwLaIOHKc\nv8zMuyPi74A7IqIPeAq4ro4ZJEmSJGnSqFvBy8zvA+87wfgLwKJ6HVeSJEmSJqsylkmQJEmSJNWB\nBU+SJEmSWoQFT5KkOoiI6RHxQER8NyL2RsQfFeMzI2JHRDxW/Dx73GtujojHI+LRiOgpL70kqVlZ\n8CRJqo9DwKWZ+T7g/cCVEfHbwBpgZ2aeD+ws9omIecD1wHzgSuDWiGgrJbkkqWlZ8CRJqoMcc7DY\nnVb8S2AJsKUY3wJcW2wvAb6YmYcy8wngccbWj5Uk6aRZ8CRJqpOIaIuI7wDPAzsy836gIzMPFE95\nlrFlhQDeBewf9/KnizFJkk5aPdfBkyRpUsvMUeD9EXEWY2vDLjju8YyIfD2/MyKWA8sBOjo6qNVq\nb1ZcqWkcPHjQ9740AQueJEl1lpkvRcQuxu6tey4iZmXmgYiYxdjZPYBngNnjXnZuMXb879oEbALo\n7OzMrq6uumaXqqhWq+F7XzoxL9GUJKkOIuIdxZk7IuI04HLge8B2YGnxtKXAcLG9Hbg+Itoj4jzg\nfOCBxqaWJDU7z+BJklQfs4AtxUyYU4A7MvMrEfFN4I6I6AOeAq4DyMy9EXEHsA84DHyyuMRTkqST\nZsGTJKkOMvNh4DdOMP4CsGiC16wD1tU5miSphXmJpiRJkiS1CAueJEmSJLUIC54kSZIktQgLniRJ\nkiS1CAueJEmSJLUIC54kSZIktQgLniRJkiS1CAueJEmSJLUIC54kSZIktQgLniRJkiS1CAueJEmS\nJLUIC54kSZIktQgLniRJryHGfDwi/lWx/2sRcVHZuSRJOt5rFryI6IiIoYj4WrE/LyL66h9NkqTK\nuBW4GOgt9n8MfKG8OJIkndjJnMH7M+Ae4FeL/f8K/GG9AkmSVEG/lZmfBH4GkJk/BN5SbiRp8jnn\nnHOICLq7u4kIzjnnnLIjSZVzMgXv7Zl5B/DfADLzMDBa11SSJFXLKxHRBiRARLyD4nNRUmOcc845\nvPjii8yfP5+tW7cyf/58XnzxRUuedJyTKXgvR8Q5/OJD7beBH9U1lSRJ1fJvgW3AOyNiHXAf8Mfl\nRpImlyPlbs+ePfzKr/wKe/bsOVryJP3C1JN4zipgO/BPI+JvgXcAH6lrKkmSKiQz/yIiHgQWAQFc\nm5kjJceSJp2vfvWrv7T/7ne/u6Q0UjW9ZsHLzG9HxH8PvJexD7VHM/OVkz1AcUnLt4BnMvP3ImIm\ncDswB3gSuK64l0GSpMopPsf2ZuavA98rO480mV199dXs2bPnmH1JxzqZWTT/AFjMWMF7D/ChiFgU\nEe88yWN8Ghj/LecaYGdmng/sLPYlSaqkzBwFHo2IXys7izSZzZw5k71797JgwQKeffZZFixYwN69\ne5k5c2bZ0aRKOZlLNPsYmxp6V7HfBTwInBcR/0dm/seJXhgR5wLXAOsYu9QTYEnxOwC2ADVg9evM\nLUlSI50N7I2IB4CXjwxm5uLyIkmTywsvvMA555zD3r176e0dW7Fk5syZvPDCCyUnk6rlZAreVGBu\nZj4HY+viAf8B+C3gG8CEBQ/4N8C/BM4YN9aRmQeK7WeBjtcbWpKkBvvfyw4giaNlrlar0dXVVW4Y\nqaJOpuDNPlLuCs8XYy9GxIT34kXE7wHPZ+aDEdF1oudkZkZETvD65cBygI6ODmq12klElVqL73up\nGjLzP5edQZKkk3EyBa8WEV8B7iz2P1yMvRV46VVe90FgcURcDUwH3hYRfw48FxGzMvNARMxirDD+\nkszcBGwC6OzsTL+l0aRz911+OylVRLFE0AAwl7EFztuAlzPzbaUGkyaZ/v5+Nm/ezKFDh2hvb2fZ\nsmUMDAyUHUuqlJMpeJ8E/gC4pNj/FmOXWb4MdE/0osy8GbgZoDiD979m5scj4v8ElgLri5/Dp5xe\nkqTG+HfA9Yx92dkJ/E+MTTwmqUH6+/vZuHEjGzZsYN68eezbt4/Vq8emcbDkSb/wmrNoZmYC3wcO\nA7/PWKl7I2v/rAcuj4jHgMuKfUmSKi0zHwfaMnM0M/89cGXZmaTJZPPmzWzYsIFVq1Yxffp0Vq1a\nxYYNG9i8eXPZ0aRKmfAMXkS8B+gt/v2AsbXrIjMnPGs3kcysMTZbJpn5AmMLxUqS1Cx+EhFvAb4T\nEZ8HDnASX5JKevMcOnSIFStWHDO2YsUKbrrpppISSdX0ah9O3wMuBX4vMy/JzAFgtDGxJEmqlE8w\n9pn5KcaWSZjN2D3pkhqkvb2djRs3HjO2ceNG2tvbS0okVdOr3YP3B4zdb7ArIu4GvghEQ1JJklQB\nEfFrmfn3mflUMfQz4I/KzCRNVsuWLTt6z928efO45ZZbWL169S+d1ZMmuwkLXmb+DfA3xWyZS4A/\nBN4ZEYPAtsy8t0EZpUp43x/dy49+OuHKIHUxZ81dDT3emadN47ufvaKhx5Qq7m+A3wSIiL/KTM/a\nSSU5MpHKZz7zmaOzaK5YscIJVqTjvOYsmsVsmX8J/GVEnA18FFgNWPA0qfzop6/w5PprGna8MhZx\nbXShlJrA+CtX/klpKSQBYyVvYGDAhc6lV/G6bhDPzB9m5qbMdJIUSdJkkBNsS5JUSSezDp4kSZPV\n+yLiHxk7k3dasU2xny50LkmqGgueJEkTyMy2sjNIkvR6uIaPJEmSmsLChQuJCLq7u4kIFi5cWHYk\nqXIseJIkSaq8hQsX8sgjj7B48WK2bdvG4sWLeeSRRyx50nEseJIkSaq8I+VueHiYs846i+Hh4aMl\nT9IvWPAkSZLUFIaGhl51X5KTrEiSJKlJXHHFFfz85z9nZGSEuXPn8pa3vKXsSFLleAZPkiRJlTd7\n9mweeughzjzzTG6//XbOPPNMHnroIWbPnl12NKlSPIMnSZKkynvb297Geeedx+7du9m9ezcA5513\nHqeffnrJyaRqseBJkiSp8kZGRvjZz37GtGnTqNVqdHV18corrzB9+vSyo0mV4iWakiRJqry5c+dy\n3333HTN23333MXfu3JISSdVkwZMkSVLlrV27lr6+Pnbt2sXhw4fZtWsXfX19rF27tuxoUqVY8CRJ\nklR5vb29HD58mEsvvZTLL7+cSy+9lMOHD9Pb21t2NKlSLHiSJEmqvIULF7J//34WL17Mtm3bWLx4\nMfv372fhwoVlR5MqxYInSZKkynvkkUdYvHgxw8PDnHXWWQwPD7N48WIeeeSRsqNJlWLBkyRJUlMY\nGhp61X1JFjxJkiQ1ib6+vlfdl2TBkySpLiJidkTsioh9EbE3Ij5djM+MiB0R8Vjx8+xxr7k5Ih6P\niEcjoqe89FL1XHDBBWzfvp0lS5bw0ksvsWTJErZv384FF1xQdjSpUlzoXJKk+jgM3JSZ346IM4AH\nI2IH8M+AnZm5PiLWAGuA1RExD7gemA/8KvD1iHhPZo6WlF+qlIcffpiFCxeyfft2tm/fDoyVvocf\nfrjkZFK1eAZPkqQ6yMwDmfntYvvHwAjwLmAJsKV42hbg2mJ7CfDFzDyUmU8AjwMXNTa1VG0PP/ww\nmcmuXbvITMuddAIWPEmS6iwi5gC/AdwPdGTmgeKhZ4GOYvtdwP5xL3u6GJMk6aR5iaYkSXUUETOA\nvwL+MDP/MSKOPpaZGRH5On/fcmA5QEdHB7Va7U1MKzWHgwcP+t6XJmDBkySpTiJiGmPl7i8y86+L\n4eciYlZmHoiIWcDzxfgzwOxxLz+3GDtGZm4CNgF0dnZmV1dXveJLlVWr1fC9L51Y3S7RjIjpEfFA\nRHy3mD3sj4rxCWcPkySpVcTYqbohYCQzbxn30HZgabG9FBgeN359RLRHxHnA+cADjcorSWoN9bwH\n7xBwaWa+D3g/cGVE/DZjs4XtzMzzgZ3FviRJreaDwCeASyPiO8W/q4H1wOUR8RhwWbFPZu4F7gD2\nAXcDn3QGTUnS61W3SzQzM4GDxe604l8yNktYVzG+BagBq+uVQ5KkMmTmfUBM8PCiCV6zDlhXt1CS\npJZX11k0I6ItIr7D2P0FOzLz1WYPkyRJkiS9AXWdZKW4tOT9EXEWsC0iFhz3+ISzhzlLmKqoke/D\nsmYI829NkiSpeTVkFs3MfCkidgFXMvHsYce/xlnCVC1339XQGbtKmSGswf8bJUmS9Oaq5yya7yjO\n3BERpwGXA99j4tnDJEmSJElvQD3P4M0CtkREG2NF8o7M/EpEfBO4IyL6gKeA6+qYQZIkSZImjXrO\novkw8BsnGH+BCWYPkyRJkiSdurrOoilJkiRJahwLniRJkiS1iIbMoim1gjPmruGCLWsae9AtjT3c\nGXMBrmnsQSVJkvSmseBJJ+nHI+t5cn3jyk8ZyyTMWXNXQ48nSZKkN5eXaEqSJElSi7DgSZIkSVKL\nsOBJkiRJUouw4EmSJKkp9Pf3M336dLq7u5k+fTr9/f1lR5Iqx0lWJEmSVHn9/f1s3LiRDRs2MG/e\nPPbt28fq1asBGBgYKDmdVB2ewZMkSVLlbd68mQ0bNrBq1SqmT5/OqlWr2LBhA5s3by47mlQpFjxJ\nkiRV3qFDh1ixYsUxYytWrODQoUMlJZKqyYInSZKkymtvb2fjxo3HjG3cuJH29vaSEknV5D14kiRJ\nqrxly5Ydvedu3rx53HLLLaxevfqXzupJk50FT5IkSZU3MDDA8PAwN91009Gx2bNnO8GKdBwv0ZQk\nSVLl9fT0sH///mPG9u/fT09PT0mJpGqy4EmSJKny7r33XgAWL17Mtm3bWLx48THjksZY8CRJktQU\nrrrqKoaHhznrrLMYHh7mqquuKjuSVDnegydJkqSm8P3vf5/p06dz6NAh2tvbmTNnTtmRpMrxDJ4k\nSZKawqOPPsqFF17InXfeyYUXXsijjz5adiSpcjyDJ0mSpKaxe/dudu/eXXYMqbI8gydJkqSmMH/+\n/Ffdl2TBkyRJUhNob2/nhhtuIDPZtWsXmckNN9xAe3t72dGkSvESTUmSJFXesmXLWL16NQDz5s3j\nlltuYfXq1axYsaLkZFK1WPAkSZJUeQMDAwwPD3PTTTcdHZs9ezYDAwMlppKqx0s0JUmSVHk9PT3s\n37+fKVPG/vN1ypQp7N+/n56enpKTSdViwZMkSVLl3XvvvQBExDE/j4xLGmPBkyRJUlOYMmUKn//8\n5/na177G5z//+aNn8yT9gn8VkiRJagof+MAHWLVqFdOnT2fVqlV84AMfKDuSVDl1K3gRMTsidkXE\nvojYGxGfLsZnRsSOiHis+Hl2vTJIkiSpddx///3ceOONHDx4kBtvvJH777+/7EhS5dRzFs3DwE2Z\n+e2IOAN4MCJ2AP8M2JmZ6yNiDbAGWF3HHJIkSWpybW1tjI6OMjg4yODg4DHjkn6hbmfwMvNAZn67\n2P4xMAK8C1gCbCmetgW4tl4ZJEmS1BpWrlxJRBwtdG1tbUQEK1euLDmZVC0NWQcvIuYAvwHcD3Rk\n5oHioWeBjkZkkCRJUvM6st7d5s2bGR0dZerUqSxbtsx18KTj1L3gRcQM4K+AP8zMfzwypS1AZmZE\n5ASvWw4sB+jo6KBWq9U7qvSa5qy5q7EHvLuxx3vrNPxbkyRV1sDAAAMDA9RqNbq6usqOI1VSXQte\nRExjrNz9RWb+dTH8XETMyswDETELeP5Er83MTcAmgM7OzvSPWGV7squxx5uz5i6eXH9NYw8qSVKF\nLVy4kEf+/rQBAAALKklEQVQeeeTo/gUXXMDDDz9cYiKpeuo5i2YAQ8BIZt4y7qHtwNJieykwXK8M\nkiRJag1Hyt3ixYvZtm0bixcv5pFHHmHhwoVlR5MqpZ7r4H0Q+ARwaUR8p/h3NbAeuDwiHgMuK/Yl\nSZKkCR0pd8PDw5x11lkMDw8fLXmSfqFul2hm5n1ATPDwonodV5IkSa3ppZdeYsqUKWQmEcHv/u7v\nlh1Jqpx6nsGTJEmS3jTf+MY3WLFiBV/+8pdZsWIF3/jGN8qOJFVOQ5ZJkCRJkt4Mxy90LulYnsGT\nJEmSpBZhwZMkSVLTmDp16jE/JR3LgidJkqSmcdFFF3HnnXdy0UUXlR1FqiS/+pAkSVJTOP300/nm\nN7/J7t27iQhOP/10fvKTn5QdS6oUz+BJklQHEXFbRDwfEXvGjc2MiB0R8Vjx8+xxj90cEY9HxKMR\n0VNOaqnafvKTn/ChD32Ibdu28aEPfchyJ52ABU+SpPr4M+DK48bWADsz83xgZ7FPRMwDrgfmF6+5\nNSLaGhdVqr62trE/ie3bt/P7v//7bN++/ZhxSWMseJIk1UFmfgN48bjhJcCWYnsLcO248S9m5qHM\nfAJ4HPAGI2mclStXEhFHC11bWxsRwcqVK0tOJlWL9+BJktQ4HZl5oNh+Fugott8F/Jdxz3u6GJNU\nGBgYAGDz5s2Mjo4ydepUli1bdnRc0hgLniRJJcjMjIh8va+LiOXAcoCOjg5qtdqbHU2qrA9/+MN8\n+MMf5uDBg8yYMQPAvwHpOBY8SZIa57mImJWZByJiFvB8Mf4MMHvc884txn5JZm4CNgF0dnZmV1dX\nHeNK1dLf38/mzZs5dOgQ7e3tnsGTTsB78CRJapztwNJieykwPG78+ohoj4jzgPOBB0rIJ1VWf38/\nX/jCFzh8+DAAhw8f5gtf+AL9/f0lJ5OqxYInSVIdRMRW4JvAeyPi6YjoA9YDl0fEY8BlxT6ZuRe4\nA9gH3A18MjNHy0kuVdPg4CCZyejo2J/G6Ogomcng4GDJyaRqseBJklQHmdmbmbMyc1pmnpuZQ5n5\nQmYuyszzM/OyzHxx3PPXZeY/zcz3ZubXyswuVdGRYjd//ny2bt3K/PnzjxmXNMZ78CRJktQUpk2b\nxr59++jt7SUimDZtGq+88krZsaRK8QyeJEmSmsIrr7zCxRdfzJ133snFF19suZNOwDN4kiRJahq7\nd+9m9+7dZceQKsszeJIkSZLUIix4kiRJktQiLHiSJEmS1CIseJIkSZLUIix4kiRJktQiLHiSJEmS\n1CIseJIkSZLUIix4kiRJktQiLHiSJEmS1CIseJIkSZLUIupW8CLitoh4PiL2jBubGRE7IuKx4ufZ\n9Tq+JEmSWktEvOq+pPqewfsz4MrjxtYAOzPzfGBnsS9JkiS9psxk5cqVfPnLX2blypVkZtmRpMqZ\nWq9fnJnfiIg5xw0vAbqK7S1ADVhdrwySJEmqplM9+zY4OMjg4OAp/y5LoVpdo+/B68jMA8X2s0BH\ng48vSZKkCsjM1/XvU5/61Al/z6c+9anX9XukVle3M3ivJTMzIib8K4uI5cBygI6ODmq1WqOiSZXh\n+16SpDEDAwMAbN68mUOHDtHe3s6yZcuOjksa0+iC91xEzMrMAxExC3h+oidm5iZgE0BnZ2d2dXU1\nKKJUEXffhe97SZJ+YWBggIGBAeasuYsn119Tdhypkhp9ieZ2YGmxvRQYbvDxJUmSJKll1XOZhK3A\nN4H3RsTTEdEHrAcuj4jHgMuKfUmSJEnSm6Ces2j2TvDQonodU5IkSZIms0ZfoilJkiRJqhMLniRJ\nkiS1CAueJEmSJLUIC54kSZIktQgLniRJkiS1CAueJEmSJLUIC54kSZIktQgLniRJkiS1CAueJEmS\nJLUIC54kSZIktQgLniRJkiS1CAueJEmSJLUIC54kSZIktYipZQeQJElS83rfH93Lj376SsOPO2fN\nXQ071pmnTeO7n72iYceT3ggLniRJkk7Zj376Ck+uv6ahx6zVanR1dTXseI0sk9Ib5SWakiRJktQi\nLHiSJEmS1CIseJIkSZLUIix4kiRJktQiLHiSJEmS1CIseJIkSZLUIix4kiRJktQiLHiSJEmS1CJc\n6FySJEmn7Iy5a7hgy5rGH3hL4w51xlyAxi7mLp0qC54kSZJO2Y9H1vPk+saWn1qtRldXV8OON2fN\nXQ07lvRGeYmmJEmSJLUIz+BJkiTpDSnlDNfdjTvmmadNa9ixpDfKgidJUoVExJXAnwBtwJ9m5vqS\nI0mvqtGXZ8JYoSzjuFIz8BJNSZIqIiLagC8AVwHzgN6ImFduKklSMyml4EXElRHxaEQ8HhElTLsk\nSVIlXQQ8npnfz8yfA18ElpScSZLURBpe8Px2UpKkCb0L2D9u/+liTJKkk1LGPXhHv50EiIgj307u\nKyGLVHcRceqv3XBqr8vMUz6mpGqLiOXAcoCOjg5qtVq5gaRT1N3d/YZef6qfkbt27XpDx5WqroyC\nd6JvJ3+rhBxSQ5xq2Wr0Gj+SKuEZYPa4/XOLsaMycxOwCaCzszP9/wk1qzfyZaSfkdLEKjuLpt9Q\narI7ePCg73tp8vk74PyIOI+xYnc98D+UG0mS1EzKKHiv+e0k+A2l5LeT0uSTmYcj4lPAPYwtk3Bb\nZu4tOZYkqYmUUfD8dlKSpAlk5leBr5adQ5LUnBpe8Px2UpIkSZLqo5R78Px2UpIkSZLefKUsdC5J\nkiRJevNZ8CRJkiSpRVjwJEmSJKlFWPAkSZIkqUVY8CRJkiSpRVjwJEmSJKlFWPAkSZIkqUVY8CRJ\nkiSpRURmlp3hNUXEPwBPlZ1DarC3Az8oO4TUYO/OzHeUHaJZ+PmoSczPSE1GJ/UZ2RQFT5qMIuJb\nmdlZdg5JkqrGz0hpYl6iKUmSJEktwoInSZIkSS3CgidV16ayA0iSVFF+RkoT8B48SZIkSWoRnsGT\nJEmSpBZhwZNKFBHXRkRGxK+XnUWSpLJFxGhEfGfcvzllZ5KajZdoSiWKiNuBXwX+U2Z+tuw8kiSV\nKSIOZuaMU3jd1Mw8XI9MUrPxDJ5UkoiYAVwC9AHXF2NTIuLWiPheROyIiK9GxEeKxy6MiP8cEQ9G\nxD0RMavE+JIkNUREzImI/ycivl38+51ivKsY3w7sK8Y+HhEPFGf//u+IaCs1vFQCC55UniXA3Zn5\nX4EXIuJC4A+AOcA84BPAxQARMQ0YAD6SmRcCtwHryggtSVIdnTbu8sxtxdjzwOWZ+ZvAx4B/O+75\nvwl8OjPfExFzi8c/mJnvB0aB/7GR4aUqmFp2AGkS6wX+pNj+YrE/FbgzM/8b8GxE7Coefy+wANgR\nEQBtwIHGxpUkqe5+WpSz8aYB/y4ijpS294x77IHMfKLYXgRcCPxd8Vl5GmPlUJpULHhSCSJiJnAp\ncEFEJGOFLYFtE70E2JuZFzcooiRJVfEvgOeA9zF29dnPxj328rjtALZk5s0NzCZVjpdoSuX4CPAf\nM/PdmTknM2cDTwAvAh8u7sXrALqK5z8KvCMijl6yGRHzywguSVKDnQkcKK5u+QRjX4qeyE7gIxHx\nThj7MjUi3t2gjFJlWPCkcvTyy2fr/gr4FeBpxm4W/3Pg28CPMvPnjJXCDRHxXeA7wO80Lq4kSaW5\nFVhafP79OseetTsqM/cB/xtwb0Q8DOwAnJBMk47LJEgVExEzMvNgRJwDPMDYzeLPlp1LkiRJ1ec9\neFL1fCUizgLeAvxry50kSZJOlmfwJEmSJKlFeA+eJEmSJLUIC54kSZIktQgLniRJkiS1CAueJEmS\nJLUIC54kSZIktQgLniRJkiS1iP8fVCRdtOetPG4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x8b759e4748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# let's make boxplots to visualise outliers in the continuous variables \n",
    "# Age and Fare\n",
    "\n",
    "plt.figure(figsize=(15,6))\n",
    "plt.subplot(1, 2, 1)\n",
    "fig = data.boxplot(column='Age')\n",
    "fig.set_title('')\n",
    "fig.set_ylabel('Age')\n",
    "\n",
    "plt.subplot(1, 2, 2)\n",
    "fig = data.boxplot(column='Fare')\n",
    "fig.set_title('')\n",
    "fig.set_ylabel('Fare')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Both Age and Fare contain outliers. Let's find which valuers are the outliers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x8b760fe1d0>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA38AAAF3CAYAAAAVe/LLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X2UXHd95/n3B9kBYcDYA+lRbAc5s4KMjZYnxSGQw3Zw\nCCYmmMMwHrGGI3O8o+zGE0xGu0EmD5DN8Y5mM2bDMGESBRhEwtoxBGKBGRKjoclyNoufcCI/4LHB\nMtgjWwyZAHKCiezv/lFXVresh+qurqp7+75f5/TpW7+qW/Wpn0v+1rfvU6oKSZIkSdLK9qRpB5Ak\nSZIkjZ/NnyRJkiT1gM2fJEmSJPWAzZ8kSZIk9YDNnyRJkiT1gM2fJEmSJPWAzZ8kSZIk9YDNnyRJ\nkiT1gM2fJEmSJPWAzZ8kSZIk9cAJ0w4wimc961m1du3aJa//8MMPc9JJJy1foDHqUlboVt4uZYVu\n5e1SVuhW3j5mvfnmm/9rVT17GSL1wqg1Err1OZsE5+MQ52Ih5+MQ52KhSczHYupjp5u/tWvXctNN\nNy15/bm5OWZnZ5cv0Bh1KSt0K2+XskK38nYpK3Qrbx+zJrlv9DT9MWqNhG59zibB+TjEuVjI+TjE\nuVhoEvOxmProbp+SJEmS1AM2f5IkSZLUAzZ/kiRJktQDY2v+knwoyb4kt80bOzXJ9Unubn6fMu++\ny5Pck+SuJK8eVy5JkiRJ6qNxbvn7MHDeYWNbgV1VtQ7Y1dwmyVnARuDsZp33J1k1xmySJEmS1Ctj\na/6q6s+Bvz5s+AJgR7O8A3j9vPGrq+qRqroXuAc4Z1zZJEmSJKlvJn3M30xV7W2WHwRmmuXTgG/M\ne9z9zZgkSZIkaRlM7Tp/VVVJarHrJdkMbAaYmZlhbm5uyRn2798/0vqT1KWs0K28XcoK3crbpazQ\nrbxmlSRJizXp5u+hJGuqam+SNcC+ZvwB4Ix5jzu9GXuCqtoObAfYsGFDjXLRxC5dhLJLWaFbebuU\nFbqVt0tZoVt5zSpJkhZr0rt97gQ2NcubgGvnjW9M8uQkZwLrgBsmnE2SJEmSVqyxbflLchUwCzwr\nyf3Au4BtwDVJLgHuAy4EqKrbk1wD3AEcAC6tqkfHlU2SJEmS+mZszV9Vvekod517lMdfAVwxrjyS\nJEmS1GeT3u1TkiRJkjQFUzvbp1a+tVuvG/k59mw7fxmSSJKWYvcD3+biEf9f7v/HJak93PInSZIk\nST1g8ydJkiRJPWDzJ0mSJEk9YPMnSZIkST1g8ydJkiRJPWDzJ0mSJEk9YPMnSZIkST1g8ydJkiRJ\nPWDzJ0mSJEk9YPMnSZIkST1g8ydJkiRJPWDzJ0mSJEk9YPMnSZIkST1g8ydJkiRJPWDzJ0mSJEk9\nYPMnSZIkST1g8ydJkiRJPWDzJ0mSJEk9YPMnSZIkST1g8ydJkiRJPWDzJ0mSJEk9YPMnSZIkST1g\n8ydJkiRJPWDzJ0mSJEk9YPMnSZIkST1g8ydJkiRJPWDzJ0mSJEk9YPMnSZIkST1g8ydJkiRJPWDz\nJ0mSJEk9YPMnSdIUJNmTZHeSW5Pc1IydmuT6JHc3v0+Z9/jLk9yT5K4kr55ecklSV9n8SZI0PT9V\nVS+sqg3N7a3ArqpaB+xqbpPkLGAjcDZwHvD+JKumEViS1F02f5IktccFwI5meQfw+nnjV1fVI1V1\nL3APcM4U8kmSOuyEaQeQJKmnCvhckkeB36uq7cBMVe1t7n8QmGmWTwP+v3nr3t+MLZBkM7AZYGZm\nhrm5uZECzqyGLesPjPQco2Zok/3796+o9zMK52Ih5+MQ52Khts2HzZ8kSdPxk1X1QJIfBK5P8pX5\nd1ZVJanFPGHTQG4H2LBhQ83Ozo4U8H0fvZYrd4/2VWHPRaNlaJO5uTlGndOVwrlYyPk4xLlYqG3z\n4W6fkiRNQVU90PzeB3ySwW6cDyVZA9D83tc8/AHgjHmrn96MSZI0NJs/SZImLMlJSZ5+cBn4GeA2\nYCewqXnYJuDaZnknsDHJk5OcCawDbphsaklS17nbpyRJkzcDfDIJDGrx/11Vn01yI3BNkkuA+4AL\nAarq9iTXAHcAB4BLq+rR6USXJHWVzZ8kSRNWVV8DXnCE8W8B5x5lnSuAK8YcTZK0grnbpyRJkiT1\ngFv+1Gprt163LM+zZ9v5y/I8kiRJUle55U+SJEmSesDmT5IkSZJ6wOZPkiRJknrAY/7UC8c6dnDL\n+gNcPMSxhR43KEmSpC5zy58kSZIk9YDNnyRJkiT1gM2fJEmSJPWAzZ8kSZIk9YDNnyRJkiT1gM2f\nJEmSJPWAzZ8kSZIk9cBUmr8kv5Tk9iS3JbkqyVOSnJrk+iR3N79PmUY2SZIkSVqJJt78JTkNeBuw\noaqeD6wCNgJbgV1VtQ7Y1dyWJEmSJC2Dae32eQKwOskJwFOB/wJcAOxo7t8BvH5K2SRJkiRpxZl4\n81dVDwD/Bvg6sBf4dlX9GTBTVXubhz0IzEw6myRJkiStVCdM+gWbY/kuAM4E/gb4WJI3z39MVVWS\nOsr6m4HNADMzM8zNzS05y/79+0daf5K6lBUGebesf3TaMYYysxq2rD9w3Me1Zf679FnoUlboVl6z\nSpKkxZp48wf8NHBvVX0TIMkngJcBDyVZU1V7k6wB9h1p5araDmwH2LBhQ83Ozi45yNzcHKOsP0ld\nygqDvFd+8eFpxxjKlvUHuHL38f8p7LlodvxhhtClz0KXskK38ppVkiQt1jSO+fs68NIkT00S4Fzg\nTmAnsKl5zCbg2ilkkyRJkqQVaeJb/qrqS0k+DtwCHAC+zGBL3tOAa5JcAtwHXDjpbDpk7dbrRlp/\nsBvlNDYsS5IkSTqSqXw7r6p3Ae86bPgRBlsBJUmSJEnLbFqXepAkSZIkTZDNnyRJkiT1gM2fJEmS\nJPWAzZ8kSZIk9YDNnyRJkiT1gM2fJEmSJPWAzZ8kSZIk9YDNnyRJkiT1gM2fJEmSJPWAzZ8kSZIk\n9YDNnyRJkiT1gM2fJEmSJPWAzZ8kSZIk9YDNnyRJkiT1gM2fJEmSJPWAzZ8kSZIk9YDNnyRJkiT1\nwAnTDiB1xdqt1438HHu2nb8MSSRJkqTFc8ufJEmSJPWAzZ8kSZIk9YDNnyRJkiT1gM2fJEmSJPWA\nzZ8kSZIk9YDNnyRJkiT1gM2fJElTkGRVki8n+XRz+9Qk1ye5u/l9yrzHXp7kniR3JXn19FJLkrrM\n5k+SpOm4DLhz3u2twK6qWgfsam6T5CxgI3A2cB7w/iSrJpxVkrQC2PxJkjRhSU4Hzgc+MG/4AmBH\ns7wDeP288aur6pGquhe4BzhnUlklSSuHzZ8kSZP328AvA4/NG5upqr3N8oPATLN8GvCNeY+7vxmT\nJGlRTph2AEmS+iTJa4F9VXVzktkjPaaqKkkt4bk3A5sBZmZmmJubGyUqM6thy/oDIz3HqBnaZP/+\n/Svq/YzCuVjI+TjEuViobfNh8ydJ0mS9HHhdkp8FngI8I8kfAg8lWVNVe5OsAfY1j38AOGPe+qc3\nY09QVduB7QAbNmyo2dnZkYK+76PXcuXu0b4q7LlotAxtMjc3x6hzulI4Fws5H4c4Fwu1bT6Ou9tn\nkpcnOalZfnOS9yR5zvijSZLUbkupkVV1eVWdXlVrGZzI5T9V1ZuBncCm5mGbgGub5Z3AxiRPTnIm\nsA64YQxvR5K0wg1zzN+/B/42yQuALcBXgY+MNZUkSd2wnDVyG/CqJHcDP93cpqpuB64B7gA+C1xa\nVY+OGlyS1D/D7MtxoDn24ALg31XVB5NcMu5gkiR1wEg1sqrmgLlm+VvAuUd53BXAFaPHlST12TDN\n33eTXA68GXhFkicBJ443liRJnWCNlCR1xjC7ff4z4BHgkqp6kMGB5r811lSSJHWDNVKS1BnH3PKX\nZBVwVVX91MGxqvo6HvMnSeo5a6QkqWuOueWvOaD8sSQnTyiPJEmdYI2UJHXNMMf87Qd2J7keePjg\nYFW9bWypJEnqBmukJKkzhmn+PtH8SJKkhayRkqTOOG7zV1U7kqwGfriq7ppAJkmSOsEaKUnqkuOe\n7TPJzwG3MriwLElemGTnuINJktR21khJUpcMc6mHdwPnAH8DUFW3Aj8yxkySJHXFu7FGSpI6Ypjm\n7++r6tuHjT02jjCSJHWMNVKS1BnDnPDl9iT/I7AqyTrgbcD/O95YkiR1gjVSktQZw2z5+0XgbOAR\n4CrgO8DbxxlKkqSOsEZKkjpjmLN9/i3wK82PJElqWCMlSV1y3OYvyaeAOmz428BNwO9V1ffGEUyS\npLazRkqSumSY3T6/BuwHfr/5+Q7wXeC5zW1JkvrKGilJ6oxhTvjysqr6sXm3P5Xkxqr6sSS3jyuY\nJEkdYI2UJHXGMFv+npbkhw/eaJaf1tz8/lhSSZLUDdZISVJnDLPlbwvwxSRfBQKcCfxCkpOAHeMM\nJ0lSy1kjJUmdMczZPj/TXLvoR5uhu+YdwP7bY0smSVLLWSMlSV0yzJY/gJcAa5vHvyAJVfWRsaWS\nJKk7rJGSpE4Y5lIPfwD8I+BW4NFmuAALmySp16yRkqQuGWbL3wbgrKo6/DpGkiT1nTVSktQZw5zt\n8zbgHy7niyZ5ZpKPJ/lKkjuT/ESSU5Ncn+Tu5vcpy/makiSNwbLXSEmSxmWYLX/PAu5IcgPwyMHB\nqnrdCK/7XuCzVfXGJD8APBV4J7CrqrYl2QpsBd4xwmtIkjRu46iRkiSNxTDN37uX8wWTnAy8ArgY\noKq+D3w/yQXAbPOwHcAcNn+SpHZ797QDSJI0rGEu9fCFJM8B1lXV55I8FVg1wmueCXwT+A9JXgDc\nDFwGzFTV3uYxDwIzI7yGJEljN4YaKUnS2Axzts9/DmwGTmVwRrPTgN8Fzh3hNV8M/GJVfSnJexns\n4vm4qqokRzx4PsnmJg8zMzPMzc0tMQbs379/pPUnadJZt6w/MNL6M6tHf45JmWTW5fhv6Od2fLqU\n16ztMIYaKUnS2Ayz2+elwDnAlwCq6u4kPzjCa94P3F9VX2puf5xB8/dQkjVVtTfJGmDfkVauqu3A\ndoANGzbU7OzskoPMzc0xyvqTNOmsF2+9bqT1t6w/wJW7h72M5HRNMuuei2ZHfg4/t+PTpbxmbY3l\nrpGSJI3NMGf7fKQ5Lg+AJCcwuIbRklTVg8A3kjyvGToXuAPYCWxqxjYB1y71NSRJmpBlrZGSJI3T\nMJs7vpDkncDqJK8CfgH41Iiv+4vAR5szfX4NeCuDRvSaJJcA9wEXjvgakiSN2zhqpCRJYzFM87cV\nuATYDfw88BngA6O8aFXdyuDCuIfzGAlJUpcse42UJGlchjnb52PA7wO/n+RU4PSqcpcWSVLvWSMl\nSV1y3GP+kswleUZT1G5mUOD+r/FHkySp3ayRkqQuGeaELydX1XeANwAfqaofx90zJUkCa6QkqUOG\naf5OaC69cCHw6THnkSSpS6yRkqTOGKb5+9+BPwXuqaobk/wIcPd4Y0mS1AnWSElSZwxzwpePAR+b\nd/trwD8ZZyhJkrrAGilJ6pJhTvjyfzYHs5+YZFeSbyZ58yTCSZLUZtZISVKXDLPb5880B7O/FtgD\n/HfA/zbOUJIkdYQ1UpLUGUOd8KX5fT7wsar69hjzSJLUJdZISVJnHPeYP+DTSb4C/B3wvyR5NvC9\n8caSJKkTrJGSpM447pa/qtoKvAzYUFV/DzwMXDDuYJIktZ01UpLUJcNs+QP4IeCnkzxl3thHxpBH\nkqSusUZKkjrhuM1fkncBs8BZwGeA1wBfxMImSeo5a6QkqUuGOeHLG4FzgQer6q3AC4CTx5pKkqRu\nsEZKkjpjmObv76rqMeBAkmcA+4AzxhtLkqROsEZKkjpjmObvpiTPBH4fuBm4BfiLsaaSJKkbFl0j\nkzwlyQ1J/jLJ7Ul+oxk/Ncn1Se5ufp8yb53Lk9yT5K4krx7nG5IkrVzHPeavqn6hWfzdJJ8FnlFV\nfzXeWJIktd8Sa+QjwCuran+SE4EvJvmPwBuAXVW1LclWYCvwjiRnARuBsxmcXOZzSZ5bVY+O5U1J\nklasoc72meQNwE8CxeBAdps/SZJYfI2sqgL2NzdPbH6KwSUiZpvxHcAc8I5m/OqqegS4N8k9wDm4\nF44kaZGOu9tnkvcD/zOwG7gN+PkkvzPuYJIktd1Sa2SSVUluZXCM4PVV9SVgpqr2Ng95EJhplk8D\nvjFv9fubMUmSFmWYLX+vBP5x85dKkuwAbh9rKkmSumFJNbLZZfOFzfGCn0zy/MPuryS12DBJNgOb\nAWZmZpibm1vsUywwsxq2rD8w0nOMmqFN9u/fv6Lezyici4Wcj0Oci4XaNh/DNH/3AD8M3NfcPqMZ\nkySp70aqkVX1N0k+D5wHPJRkTVXtTbKGwVZBgAdYeAbR05uxIz3fdmA7wIYNG2p2dnYRb+WJ3vfR\na7ly91BHiBzVnotGy9Amc3NzjDqnK4VzsZDzcYhzsVDb5mOYs30+HbgzyVxToO4AnpFkZ5Kd440n\nSVKrLbpGJnl2s8WPJKuBVwFfAXYCm5qHbQKubZZ3AhuTPDnJmcA64IaxvSNJ0oo1zJ/zfn3sKSRJ\n6qal1Mg1wI4kqxj8Efaaqvp0kr8ArklyCYMtiRcCVNXtSa5h0FgeAC71TJ+SpKUY5lIPX5hEEEmS\numYpNbK5FMSLjjD+LeDco6xzBXDFogNKkjTPMLt9SpIkSZI6zuZPkiRJknrgqM1fkl3N7389uTiS\nJLWfNVKS1EXHOuZvTZKXAa9LcjWQ+XdW1S1jTSZJUntZIyVJnXOs5u/XgV9jcD2h9xx2XzG4sK0k\nSX1kjZQkdc5Rm7+q+jjw8SS/VlW/OcFMkiS1mjVSktRFw1zq4TeTvA54RTM0V1WfHm8sSZLazxop\nSeqS457tM8m/Ai5jcHHZO4DLkvwf4w4mSVLbWSMlSV1y3C1/wPnAC6vqMYAkO4AvA+8cZzBJkjrA\nGilJ6oxhr/P3zHnLJ48jiCRJHWWNlCR1wjBb/v4V8OUkn2dwKutXAFvHmkqSpG6wRkqSOmOYE75c\nlWQO+LFm6B1V9eBYU0mS1AHWSElSlwyz5Y+q2gvsHHMWSZI6xxopSeqKYY/5kyRJkiR1mM2fJEmS\nJPXAMZu/JKuSfGVSYSRJ6gprpCSpa47Z/FXVo8BdSX54QnkkSeoEa6QkqWuGOeHLKcDtSW4AHj44\nWFWvG1sqSZK6wRopSeqMYZq/Xxt7Cqkn1m69buTn2LL+ALOjR5G0PKyRkqTOGOY6f19I8hxgXVV9\nLslTgVXjjyZJUrtZIyVJXXLcs30m+efAx4Hfa4ZOA/5knKEkSeoCa6QkqUuGudTDpcDLge8AVNXd\nwA+OM5QkSR1hjZQkdcYwzd8jVfX9gzeSnADU+CJJktQZ1khJUmcM0/x9Ick7gdVJXgV8DPjUeGNJ\nktQJ1khJUmcM0/xtBb4J7AZ+HvgM8KvjDCVJUkdYIyVJnTHM2T4fS7ID+BKDXVnuqip3aZEk9Z41\nUpLUJcdt/pKcD/wu8FUgwJlJfr6q/uO4w0mS1GbWSElSlwxzkfcrgZ+qqnsAkvwj4DrAwiZJ6jtr\npCSpM4Y55u+7B4ta42vAd8eUR5KkLrFGSpI646hb/pK8oVm8KclngGsYHM/wT4EbJ5BNkqRWskZK\nkrroWLt9/ty85YeA/6FZ/iawetQXTrIKuAl4oKpem+RU4I+AtcAe4MKq+m+jvo4kSWMw1hopSdI4\nHLX5q6q3jvm1LwPuBJ7R3N4K7KqqbUm2NrffMeYMkiQt2gRqpCRJy26Ys32eCfwigy1yjz++ql63\n1BdNcjpwPnAF8C+b4QuA2WZ5BzCHzZ8kqcXGUSMlSRqXYc72+SfAB4FPAY8t0+v+NvDLwNPnjc1U\n1d5m+UFgZpleS5KkcRlHjZQkaSyGaf6+V1X/drleMMlrgX1VdXOS2SM9pqoqyREvkptkM7AZYGZm\nhrm5uSVn2b9/P3Nzc+x+4NtLfo6D1p928sjPcSwHs07KlvUHRlp/ZvXozzEpXcoKg7yT/CyMYtKf\n21F1Ka9ZW2NZa6QkSeM0TPP33iTvAv4MeOTgYFXdssTXfDnwuiQ/CzwFeEaSPwQeSrKmqvYmWQPs\nO9LKVbUd2A6wYcOGmp2dXWKMwRfo2dlZLt563ZKf46A9Fy09xzAOZp2UUedky/oDXLl7mI/X9HUp\nKwzyXjjBz8IoJv25HVWX8pq1NZa7RkqSNDbDfONdD7wFeCWHdmmp5vaiVdXlwOUAzZa//7Wq3pzk\nt4BNwLbm97VLeX5JkiZoWWukJEnjNEzz90+BH6mq7485yzbgmiSXAPcBF4759SRJGtWkaqQkSSMb\npvm7DXgmR9kNcxRVNcfgrJ5U1beAc5f7NSRJGqOx1UhJkpbbMM3fM4GvJLmRhcczeBprSVLfWSMl\nSZ0xTPP3rrGnkCSpm6yRkqTOOG7zV1VfmEQQSZK6xhopSeqS4zZ/Sb7L4MxlAD8AnAg8XFXPGGcw\nSZLazhopSeqSYbb8Pf3gcpIAFwAvHWcoSZK6wBopSeqSJy3mwTXwJ8Crx5RHkqROskZKktpumN0+\n3zDv5pOADcD3xpZIkqSOsEZKkrpkmLN9/ty85QPAHga7tUiS1HeLrpFJzgA+AswwOF5we1W9N8mp\nwB8Ba5vnubCq/luzzuXAJcCjwNuq6k+X9V1IknphmGP+3jqJIJIkdc0Sa+QBYEtV3ZLk6cDNSa4H\nLgZ2VdW2JFuBrcA7kpwFbATOBn4I+FyS51bVo8vzLiRJfXHU5i/Jrx9jvaqq3xxDHkmSWm+UGllV\ne4G9zfJ3k9wJnMZgi+Fs87AdwBzwjmb86qp6BLg3yT3AOcBfjPg2JEk9c6wTvjx8hB8Y7HbyjjHn\nkiSpzZalRiZZC7wI+BIw0zSGAA8y2C0UBo3hN+atdn8zJknSohx1y19VXXlwudkt5TLgrcDVwJVH\nW0/S+K3det20IwCwZ9v5044gTcVy1MgkTwP+GHh7VX1ncKWIx5+/ktRRVz76c24GNgPMzMwwNze3\n2KdYYGY1bFl/YKTnGDVDm+zfv39FvZ9ROBcLOR+HOBcLtW0+jnnMX3Pw+b8ELmKwC8qLDx58LklS\nn41SI5OcyKDx+2hVfaIZfijJmqram2QNsK8ZfwA4Y97qpzdjT1BV24HtABs2bKjZ2dnFvanDvO+j\n13Ll7mHODXd0ey4aLUObzM3NMeqcrhTOxULOxyHOxUJtm4+j7vaZ5LeAG4HvAuur6t02fpIkjVYj\nm4vBfxC4s6reM++uncCmZnkTcO288Y1JnpzkTGAdcMMyvA1JUs8c6895W4BHgF8FfmXe7ihhsEfK\nM8acTZKkthqlRr4ceAuwO8mtzdg7gW3ANUkuAe4DLmTwZLcnuQa4g8GZQi/1TJ+SpKU41jF/xzoZ\njCRJvTVKjayqLzJoEo/k3KOscwVwxVJfU5IkOPbZPiVJkiRJK4TNnyRJkiT1gM2fJEmSJPWAzZ8k\nSZIk9YDNnyRJkiT1gM2fJEmSJPWAzZ8kSZIk9YDNnyRJkiT1wFEv8q5uWrv1umlHkCRJktRCbvmT\nJEmSpB6w+ZMkSZKkHrD5kyRJkqQe8Jg/SZI0Nst1LPqebecvy/NIUp+55U+SJEmSesAtf5KW7Hh/\n0d+y/gAXH+cx/jVfkiRpMtzyJ0mSJEk9YPMnSZIkST1g8ydJkiRJPWDzJ0mSJEk9YPMnSZIkST1g\n8ydJkiRJPWDzJ0mSJEk9YPMnSZIkST1g8ydJkiRJPWDzJ0mSJEk9YPMnSZIkST1g8ydJkiRJPWDz\nJ0mSJEk9YPMnSZIkST1g8ydJkiRJPWDzJ0mSJEk9YPMnSZIkST1g8ydJkiRJPWDzJ0mSJEk9YPMn\nSZIkST1g8ydJkiRJPWDzJ0mSJEk9YPMnSZIkST0w8eYvyRlJPp/kjiS3J7msGT81yfVJ7m5+nzLp\nbJIkSZK0Uk1jy98BYEtVnQW8FLg0yVnAVmBXVa0DdjW3JUmSJEnLYOLNX1XtrapbmuXvAncCpwEX\nADuah+0AXj/pbJIkSZK0Uk31mL8ka4EXAV8CZqpqb3PXg8DMlGJJkiRJ0opzwrReOMnTgD8G3l5V\n30ny+H1VVUnqKOttBjYDzMzMMDc3t+QM+/fvZ25uji3rDyz5OQ4aJccwDmY9nuV4L8thZnV7shxP\nl7JCt/IOk3Xc/3YWY9h/Z21gVkmStFhTaf6SnMig8ftoVX2iGX4oyZqq2ptkDbDvSOtW1XZgO8CG\nDRtqdnZ2yTnm5uaYnZ3l4q3XLfk5Dtpz0dJzDONg1uNZjveyHLasP8CVu6f2t4VF6VJW6FbeYbKO\n+9/OYgz776wNzCpJkhZrGmf7DPBB4M6qes+8u3YCm5rlTcC1k84mSZIkSSvVNDYfvBx4C7A7ya3N\n2DuBbcA1SS4B7gMunEI2SZIkSVqRJt78VdUXgRzl7nMnmUWSJEmS+mKqZ/uUJKmPknwoyb4kt80b\nOzXJ9Unubn6fMu++y5Pck+SuJK+eTmpJUtfZ/EmSNHkfBs47bGwrsKuq1gG7mtskOQvYCJzdrPP+\nJKsmF1WStFLY/EmSNGFV9efAXx82fAGwo1neAbx+3vjVVfVIVd0L3AOcM5GgkqQVxeZPkqR2mKmq\nvc3yg8BMs3wa8I15j7u/GZMkaVG6cbEwSZJ6pKoqSS12vSSbgc0AMzMzzM3NjZRjZvXgep1tMOp7\nWQ779+9vRY42cC4Wcj4OcS4Watt82PxJktQODyVZU1V7k6wB9jXjDwBnzHvc6c3YE1TVdmA7wIYN\nG2p2dnakQO/76LVcubsdXxX2XDQ77QjMzc0x6pyuFM7FQs7HIc7FQm2bD3f7lCSpHXYCm5rlTcC1\n88Y3JnlykjOBdcANU8gnSeq4dvw5T5KkHklyFTALPCvJ/cC7gG3ANUkuAe4DLgSoqtuTXAPcARwA\nLq2qR6cSXJLUaTZ/kqZq7dbrluV59mw7f1meR5qEqnrTUe469yiPvwK4YnyJJEl94G6fkiRJktQD\nNn+SJEmS1APu9tkix9r9bcv6A1y8TLvHSZIkSeoft/xJkiRJUg/Y/EmSJElSD7jb5zJZrjMWSpIk\nSdI4uOU1GLUxAAALcUlEQVRPkiRJknrA5k+SJEmSesDmT5IkSZJ6wGP+JK0Iy3Hc7YfPO2kZkkiS\nJLWTW/4kSZIkqQds/iRJkiSpB2z+JEmSJKkHPOZPkiS13nIc17tn2/nLkESSusstf5IkSZLUA275\nk6TG7ge+zcUjbl1wy4IkSWort/xJkiRJUg/Y/EmSJElSD9j8SZIkSVIP2PxJkiRJUg/Y/EmSJElS\nD9j8SZIkSVIP2PxJkiRJUg/Y/EmSJElSD9j8SZIkSVIP2PxJkiRJUg/Y/EmSJElSD9j8SZIkSVIP\nnDDtAJIkSZOwdut1I62/Zf0BZpcniiRNhVv+JEmSJKkHbP4kSZIkqQds/iRJkiSpB2z+JEmSJKkH\nbP4kSZIkqQc826cktdDxzkq4Zf0BLj7OY/ZsO385I0mSpI5zy58kSZIk9YDNnyRJkiT1gLt9StIy\nGvUi0pIkSePilj9JkiRJ6gGbP0mSJEnqAZs/SZIkSeoBj/mTJEmasOU4PtjLuUhaLJs/SZKkIXlS\nJ0ld5m6fkiRJktQDNn+SJEmS1APu9ilJkqSReAyj1A2ta/6SnAe8F1gFfKCqtk05kiRJU2d9lI7P\nJlQ6tlY1f0lWAb8DvAq4H7gxyc6qumO6ySSpe9pyYooPn3fStCN0nvVRkrQcWtX8AecA91TV1wCS\nXA1cAFjcJEl9Zn2UpJYZ5o+sW9Yf4OJjPG7SW5rb1vydBnxj3u37gR+fUhZJktrC+qgnWGm7OLZl\nbwVpJUtVTTvD45K8ETivqv6n5vZbgB+vqn8x7zGbgc3NzecBd43wks8C/usI609Sl7JCt/J2KSt0\nK2+XskK38vYx63Oq6tnL8DydM0x9bMaXs0ZCtz5nk+B8HOJcLOR8HOJcLDSJ+Ri6PrZty98DwBnz\nbp/ejD2uqrYD25fjxZLcVFUbluO5xq1LWaFbebuUFbqVt0tZoVt5zdo7x62PsLw1Evxvdzjn4xDn\nYiHn4xDnYqG2zUfbrvN3I7AuyZlJfgDYCOycciZJkqbN+ihJGlmrtvxV1YEk/wL4Uwansv5QVd0+\n5ViSJE2V9VGStBxa1fwBVNVngM9M6OWWbdeYCehSVuhW3i5lhW7l7VJW6FZes/bMhOvjQf63W8j5\nOMS5WMj5OMS5WKhV89GqE75IkiRJksajbcf8SZIkSZLGoJfNX5LzktyV5J4kW6ed53BJPpRkX5Lb\n5o2dmuT6JHc3v0+ZZsaDkpyR5PNJ7khye5LLmvG25n1KkhuS/GWT9zea8VbmBUiyKsmXk3y6ud3m\nrHuS7E5ya5KbmrFW5k3yzCQfT/KVJHcm+Yk2Zk3yvGY+D/58J8nb25j1oCS/1Pz7ui3JVc2/u9bm\n1ZG1vVYut8XW3iSXN3NzV5JXTyf1eCyltq/w+Vj0d4eVPB+wuO8mPZiLRX33mfZ89K75S7IK+B3g\nNcBZwJuSnDXdVE/wYeC8w8a2Aruqah2wq7ndBgeALVV1FvBS4NJmPtua9xHglVX1AuCFwHlJXkp7\n8wJcBtw573abswL8VFW9cN5pjdua973AZ6vqR4EXMJjj1mWtqrua+Xwh8BLgb4FP0sKsAElOA94G\nbKiq5zM4OclGWppXR9aRWrncPsyQtbeZi43A2c0672/mbKVYVG3vwXws6rtDD+YDhvxu0pO5gCG/\n+7RhPnrX/AHnAPdU1deq6vvA1cAFU860QFX9OfDXhw1fAOxolncAr59oqKOoqr1VdUuz/F0G/yM4\njfbmrara39w8sfkpWpo3yenA+cAH5g23MusxtC5vkpOBVwAfBKiq71fV39DCrIc5F/hqVd1Hu7Oe\nAKxOcgLwVOC/0O68eqLW18rltsjaewFwdVU9UlX3AvcwmLMVYQm1faXPx2K/O6zo+Vjkd5MVPRfH\n0Nr56GPzdxrwjXm372/G2m6mqvY2yw8CM9MMcyRJ1gIvAr5Ei/M2uyrcCuwDrq+qNuf9beCXgcfm\njbU1KwyK4eeS3JxkczPWxrxnAt8E/kOz28oHkpxEO7POtxG4qlluZdaqegD4N8DXgb3At6vqz2hp\nXh1VV2vlcjva57Y38zNkbV/x87HI7w4rfT4W891kpc8FLO67z9Tno4/NX+fV4BStrTpNa5KnAX8M\nvL2qvjP/vrblrapHm13oTgfOSfL8w+5vRd4krwX2VdXNR3tMW7LO85PN3L6GwW5Cr5h/Z4vyngC8\nGPj3VfUi4GEO2w2xRVkByODC3q8DPnb4fW3K2hzXcAGDBvuHgJOSvHn+Y9qUVxpWHz+3Xart49aV\n7w7j1tHvJuPWle8+QD+bvweAM+bdPr0Za7uHkqwBaH7vm3KexyU5kUFx+GhVfaIZbm3eg5rd/D7P\nYJ/rNuZ9OfC6JHsY7HL1yiR/SDuzAo9v9aGq9jE4Lu0c2pn3fuD+5i+3AB9n0Ay2MetBrwFuqaqH\nmtttzfrTwL1V9c2q+nvgE8DLaG9eHVlXa+VyO9rndsXPzyJr+4qfj4OG/O6wkudjsd9NVvJcAIv+\n7jP1+ehj83cjsC7Jmc1f0jcCO6ecaRg7gU3N8ibg2ilmeVySMDhu6s6qes+8u9qa99lJntksrwZe\nBXyFFuatqsur6vSqWsvgc/qfqurNtDArQJKTkjz94DLwM8BttDBvVT0IfCPJ85qhc4E7aGHWed7E\noV0+ob1Zvw68NMlTm/8/nMvgeKG25tWRdbVWLrejfW53AhuTPDnJmcA64IYp5BuLJdT2lT4fi/3u\nsGLnYwnfTVbsXMCSvvtMfz6qqnc/wM8C/xn4KvAr085zhHxXMThW5u8ZbKG4BPgHDM4WdDfwOeDU\naedssv4kg03ZfwXc2vz8bIvz/vfAl5u8twG/3oy3Mu+83LPAp9ucFfgR4C+bn9sP/ttqcd4XAjc1\nn4U/AU5pcdaTgG8BJ88ba2XWJttvMPhidBvwB8CT25zXn6P+d2x1rRzD+11U7QV+pZmbu4DXTDv/\nMs/Fomv7Cp+PRX93WMnzMe89DvXdZCXPxVK++0x7PtKEkCRJkiStYH3c7VOSJEmSesfmT5IkSZJ6\nwOZPkiRJknrA5k+SJEmSesDmT5IkSZJ6wOZPaqEkr09SSX502lkkSZq2JI8muXXez9ppZ5K6yEs9\nSC2U5I+AH2Jw8dR3TTuPJEnTlGR/VT1tCeudUFUHxpFJ6iK3/Ektk+RpDC6wewmwsRl7UpL3J/lK\nkuuTfCbJG5v7XpLkC0luTvKnSdZMMb4kSRORZG2S/yfJLc3Py5rx2WZ8J3BHM/bmJDc0Ww1/L8mq\nqYaXpsTmT2qfC4DPVtV/Br6V5CXAG4C1wFnAW4CfAEhyIvA+4I1V9RLgQ8AV0wgtSdIYrZ63y+cn\nm7F9wKuq6sXAPwP+7bzHvxi4rKqem+QfN/e/vKpeCDwKXDTJ8FJbnDDtAJKe4E3Ae5vlq5vbJwAf\nq6rHgAeTfL65/3nA84HrkwCsAvZONq4kSWP3d03jNt+JwL9LcrChe+68+26oqnub5XOBlwA3NrVy\nNYPGUeodmz+pRZKcCrwSWJ+kGDRzBXzyaKsAt1fVT0wooiRJbfFLwEPACxjszfa9efc9PG85wI6q\nunyC2aRWcrdPqV3eCPxBVT2nqtZW1RnAvcBfA/+kOfZvBphtHn8X8Owkj+8GmuTsaQSXJGnCTgb2\nNnvFvIXBH0yPZBfwxiQ/CIM/tCZ5zoQySq1i8ye1y5t44la+Pwb+IXA/gwPX/xC4Bfh2VX2fQcP4\nr5P8JXAr8LLJxZUkaWreD2xq6t+PsnBr3+Oq6g7gV4E/S/JXwPWAJ0dTL3mpB6kjkjytqvYn+QfA\nDQwOXH9w2rkkSZLUDR7zJ3XHp5M8E/gB4Ddt/CRJkrQYbvmTJEmSpB7wmD9JkiRJ6gGbP0mSJEnq\nAZs/SZIkSeoBmz9JkiRJ6gGbP0mSJEnqAZs/SZIkSeqB/x+5QtZKy9en8gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x8b760a9438>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# first we plot the distributions to find out if they are Gaussian or skewed.\n",
    "# Depending on the distribution, we will use the normal assumption or the interquantile\n",
    "# range to find outliers\n",
    "\n",
    "plt.figure(figsize=(15,6))\n",
    "plt.subplot(1, 2, 1)\n",
    "fig = data.Age.hist(bins=20)\n",
    "fig.set_ylabel('Number of passengers')\n",
    "fig.set_xlabel('Age')\n",
    "\n",
    "plt.subplot(1, 2, 2)\n",
    "fig = data.Fare.hist(bins=20)\n",
    "fig.set_ylabel('Number of passengers')\n",
    "fig.set_xlabel('Fare')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Age is quite Gaussian and Fare is skewed, so I will use the Gaussian assumption for Age, and the interquantile range for Fare"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Age outliers are values < -13.88037434994331 or > 73.27860964406095\n",
      "Fare outliers are values < -61.358399999999996 or > 100.2688\n"
     ]
    }
   ],
   "source": [
    "# find outliers\n",
    "\n",
    "# Age\n",
    "Upper_boundary = data.Age.mean() + 3* data.Age.std()\n",
    "Lower_boundary = data.Age.mean() - 3* data.Age.std()\n",
    "print('Age outliers are values < {lowerboundary} or > {upperboundary}'.format(lowerboundary=Lower_boundary, upperboundary=Upper_boundary))\n",
    "\n",
    "# Fare\n",
    "IQR = data.Fare.quantile(0.75) - data.Fare.quantile(0.25)\n",
    "Lower_fence = data.Fare.quantile(0.25) - (IQR * 3)\n",
    "Upper_fence = data.Fare.quantile(0.75) + (IQR * 3)\n",
    "print('Fare outliers are values < {lowerboundary} or > {upperboundary}'.format(lowerboundary=Lower_fence, upperboundary=Upper_fence))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We  should remove from the dataset Ages > 73 or Fares > 100.\n",
    "\n",
    "There are a few methods to handle outliers, one is top-coding, the other one is discretisation of variables. **I will use top-coding for Age, and discretisation for Fare**.\n",
    "\n",
    "#### Outlies in discrete variables\n",
    "\n",
    "Let's calculate the percentage of passengers for each  of the values that can take the discrete variables in the titanic dataset. I will call outliers, those values that are present in less than 1% of the passengers. This is exactly the same as finding rare labels in categorical variables. Discrete variables, in essence can be pre-processed / engineered as if they were categorical. Keep this in mind."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3    0.551066\n",
      "1    0.242424\n",
      "2    0.206510\n",
      "Name: Pclass, dtype: float64\n",
      "\n",
      "0    0.682379\n",
      "1    0.234568\n",
      "2    0.031425\n",
      "4    0.020202\n",
      "3    0.017957\n",
      "8    0.007856\n",
      "5    0.005612\n",
      "Name: SibSp, dtype: float64\n",
      "\n",
      "0    0.760943\n",
      "1    0.132435\n",
      "2    0.089787\n",
      "5    0.005612\n",
      "3    0.005612\n",
      "4    0.004489\n",
      "6    0.001122\n",
      "Name: Parch, dtype: float64\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# outlies in discrete variables\n",
    "for var in ['Pclass',  'SibSp', 'Parch']:\n",
    "    print(data[var].value_counts() / np.float(len(data)))\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Pclass** does not contain outliers, as all its numbers are present in at least 20% of the passengers.\n",
    "\n",
    "**SibSp**\n",
    "This variable indicates the number of of siblings / spouses aboard the Titanic. Values bigger than 4, are rare. So I will cap this variable at 4 (top coding).\n",
    "\n",
    "**Parch** \n",
    "This variable indicates the number of parents / children aboard the Titanic. We can see that values > 2 are rare (present in less than 1% of passengers). Thus I will cap this variable at 2 (top-coding).\n",
    "\n",
    "\n",
    "#### Number of labels: cardinality"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name  contains  891  labels\n",
      "Sex  contains  2  labels\n",
      "Ticket  contains  681  labels\n",
      "Cabin  contains  148  labels\n",
      "Embarked  contains  4  labels\n"
     ]
    }
   ],
   "source": [
    "for var in categorical:\n",
    "    print(var, ' contains ', len(data[var].unique()), ' labels')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The variables Name, Ticket and Cabin are highly cardinal, i.e., they contain a lot of labels. In addition, those variables are not usable as such, and they require some manual preprocessing. I will do that before proceeding with the data exploration"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pre-processing of mixed type of variables (section 11)\n",
    "\n",
    "The variables Cabin and Ticket contain both numbers and letters. Let's create 2 variables for each extracting the numerical and categorical part."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Cabin_numerical</th>\n",
       "      <th>Cabin_categorical</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>C85</td>\n",
       "      <td>85.0</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>C123</td>\n",
       "      <td>123.0</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Cabin  Cabin_numerical Cabin_categorical\n",
       "0   NaN              NaN               NaN\n",
       "1   C85             85.0                 C\n",
       "2   NaN              NaN               NaN\n",
       "3  C123            123.0                 C\n",
       "4   NaN              NaN               NaN"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Cabin\n",
    "data['Cabin_numerical'] = data.Cabin.str.extract('(\\d+)') # extracts number from string\n",
    "data['Cabin_numerical'] = data['Cabin_numerical'].astype('float') # parses the above variable to float type\n",
    "\n",
    "data['Cabin_categorical'] = data['Cabin'].str[0] # captures first letter of string (the letter of the cabin)\n",
    "\n",
    "# same for submission data\n",
    "submission['Cabin_numerical'] = submission.Cabin.str.extract('(\\d+)')\n",
    "submission['Cabin_numerical'] = submission['Cabin_numerical'].astype('float')\n",
    "\n",
    "submission['Cabin_categorical'] = submission['Cabin'].str[0]\n",
    "\n",
    "data[['Cabin', 'Cabin_numerical', 'Cabin_categorical']].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# drop the original variable\n",
    "data.drop(labels='Cabin', inplace=True, axis=1)\n",
    "submission.drop(labels='Cabin', inplace=True, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Ticket_numerical</th>\n",
       "      <th>Ticket_categorical</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>21171.0</td>\n",
       "      <td>A/5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>PC 17599</td>\n",
       "      <td>17599.0</td>\n",
       "      <td>PC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>3101282.0</td>\n",
       "      <td>STON/O2.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>113803</td>\n",
       "      <td>113803.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>373450</td>\n",
       "      <td>373450.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             Ticket  Ticket_numerical Ticket_categorical\n",
       "0         A/5 21171           21171.0                A/5\n",
       "1          PC 17599           17599.0                 PC\n",
       "2  STON/O2. 3101282         3101282.0           STON/O2.\n",
       "3            113803          113803.0                NaN\n",
       "4            373450          373450.0                NaN"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  Ticket\n",
    "# extract the last bit of ticket as number\n",
    "data['Ticket_numerical'] = data.Ticket.apply(lambda s: s.split()[-1])\n",
    "data['Ticket_numerical'] = np.where(data.Ticket_numerical.str.isdigit(), data.Ticket_numerical, np.nan)\n",
    "data['Ticket_numerical'] = data['Ticket_numerical'].astype('float')\n",
    "\n",
    "# extract the first part of ticket as category\n",
    "data['Ticket_categorical'] = data.Ticket.apply(lambda s: s.split()[0])\n",
    "data['Ticket_categorical'] = np.where(data.Ticket_categorical.str.isdigit(), np.nan, data.Ticket_categorical)\n",
    "\n",
    "# submission\n",
    "submission['Ticket_numerical'] = submission.Ticket.apply(lambda s: s.split()[-1])\n",
    "submission['Ticket_numerical'] = np.where(submission.Ticket_numerical.str.isdigit(), submission.Ticket_numerical, np.nan)\n",
    "submission['Ticket_numerical'] = submission['Ticket_numerical'].astype('float')\n",
    "\n",
    "# extract the first part of ticket as category\n",
    "submission['Ticket_categorical'] = submission.Ticket.apply(lambda s: s.split()[0])\n",
    "submission['Ticket_categorical'] = np.where(submission.Ticket_categorical.str.isdigit(), np.nan, submission.Ticket_categorical)\n",
    "\n",
    "data[['Ticket', 'Ticket_numerical', 'Ticket_categorical']].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['A/5', 'PC', 'STON/O2.', nan, 'PP', 'A/5.', 'C.A.', 'A./5.',\n",
       "       'SC/Paris', 'S.C./A.4.', 'A/4.', 'CA', 'S.P.', 'S.O.C.', 'SO/C',\n",
       "       'W./C.', 'SOTON/OQ', 'W.E.P.', 'STON/O', 'A4.', 'C', 'SOTON/O.Q.',\n",
       "       'SC/PARIS', 'S.O.P.', 'A.5.', 'Fa', 'CA.', 'LINE', 'F.C.C.', 'W/C',\n",
       "       'SW/PP', 'SCO/W', 'P/PP', 'SC', 'SC/AH', 'A/S', 'A/4', 'WE/P',\n",
       "       'S.W./PP', 'S.O./P.P.', 'F.C.', 'SOTON/O2', 'S.C./PARIS',\n",
       "       'C.A./SOTON'], dtype=object)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's explore the ticket categorical part a bit further\n",
    "data.Ticket_categorical.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Ticket_categorical</th>\n",
       "      <th>Ticket_categorical</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A</td>\n",
       "      <td>A/5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>PC</td>\n",
       "      <td>PC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>STONO</td>\n",
       "      <td>STON/O2.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>PP</td>\n",
       "      <td>PP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>A</td>\n",
       "      <td>A/5.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>861</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>862</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>863</th>\n",
       "      <td>CA</td>\n",
       "      <td>CA.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>864</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>865</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>866</th>\n",
       "      <td>SCPARIS</td>\n",
       "      <td>SC/PARIS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>867</th>\n",
       "      <td>PC</td>\n",
       "      <td>PC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>868</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>869</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>870</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>871</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>872</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>873</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>874</th>\n",
       "      <td>PPP</td>\n",
       "      <td>P/PP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>875</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>876</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>877</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>878</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>879</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>880</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>881</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>882</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>883</th>\n",
       "      <td>CASOTON</td>\n",
       "      <td>C.A./SOTON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>884</th>\n",
       "      <td>SOTONOQ</td>\n",
       "      <td>SOTON/OQ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>885</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>886</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>887</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>888</th>\n",
       "      <td>WC</td>\n",
       "      <td>W./C.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>889</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>890</th>\n",
       "      <td>nan</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>891 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    Ticket_categorical Ticket_categorical\n",
       "0                    A                A/5\n",
       "1                   PC                 PC\n",
       "2                STONO           STON/O2.\n",
       "3                  nan                NaN\n",
       "4                  nan                NaN\n",
       "5                  nan                NaN\n",
       "6                  nan                NaN\n",
       "7                  nan                NaN\n",
       "8                  nan                NaN\n",
       "9                  nan                NaN\n",
       "10                  PP                 PP\n",
       "11                 nan                NaN\n",
       "12                   A               A/5.\n",
       "13                 nan                NaN\n",
       "14                 nan                NaN\n",
       "15                 nan                NaN\n",
       "16                 nan                NaN\n",
       "17                 nan                NaN\n",
       "18                 nan                NaN\n",
       "19                 nan                NaN\n",
       "20                 nan                NaN\n",
       "21                 nan                NaN\n",
       "22                 nan                NaN\n",
       "23                 nan                NaN\n",
       "24                 nan                NaN\n",
       "25                 nan                NaN\n",
       "26                 nan                NaN\n",
       "27                 nan                NaN\n",
       "28                 nan                NaN\n",
       "29                 nan                NaN\n",
       "..                 ...                ...\n",
       "861                nan                NaN\n",
       "862                nan                NaN\n",
       "863                 CA                CA.\n",
       "864                nan                NaN\n",
       "865                nan                NaN\n",
       "866            SCPARIS           SC/PARIS\n",
       "867                 PC                 PC\n",
       "868                nan                NaN\n",
       "869                nan                NaN\n",
       "870                nan                NaN\n",
       "871                nan                NaN\n",
       "872                nan                NaN\n",
       "873                nan                NaN\n",
       "874                PPP               P/PP\n",
       "875                nan                NaN\n",
       "876                nan                NaN\n",
       "877                nan                NaN\n",
       "878                nan                NaN\n",
       "879                nan                NaN\n",
       "880                nan                NaN\n",
       "881                nan                NaN\n",
       "882                nan                NaN\n",
       "883            CASOTON         C.A./SOTON\n",
       "884            SOTONOQ           SOTON/OQ\n",
       "885                nan                NaN\n",
       "886                nan                NaN\n",
       "887                nan                NaN\n",
       "888                 WC              W./C.\n",
       "889                nan                NaN\n",
       "890                nan                NaN\n",
       "\n",
       "[891 rows x 2 columns]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# it contains several labels, some of them seem very similar apart from the punctuation\n",
    "# I will try to reduce this number of labels a bit further\n",
    "\n",
    "# remove non letter characters from string\n",
    "text = data.Ticket_categorical.apply(lambda x: re.sub(\"[^a-zA-Z]\", '', str(x)))\n",
    "\n",
    "# to visualise the output and compare with input\n",
    "pd.concat([text, data.Ticket_categorical], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['A', 'PC', 'STONO', 'NAN', 'PP', 'CA', 'SCPARIS', 'SCA', 'SP',\n",
       "       'SOC', 'WC', 'SOTONOQ', 'WEP', 'C', 'SOP', 'FA', 'LINE', 'FCC',\n",
       "       'SWPP', 'SCOW', 'PPP', 'SC', 'SCAH', 'AS', 'SOPP', 'FC', 'SOTONO',\n",
       "       'CASOTON'], dtype=object)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# set to upper case: we reduce the number of labels quite a bit\n",
    "text = text.str.upper()\n",
    "text.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# process the variable in submission as well\n",
    "data['Ticket_categorical'] = text\n",
    "\n",
    "submission['Ticket_categorical'] = submission.Ticket_categorical.apply(lambda x: re.sub(\"[^a-zA-Z]\", '', str(x)))\n",
    "submission['Ticket_categorical'] = submission['Ticket_categorical'].str.upper()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# drop the original variable\n",
    "data.drop(labels='Ticket', inplace=True, axis=1)\n",
    "submission.drop(labels='Ticket', inplace=True, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bonus: tailored preprocessing for the Titanic dataset\n",
    "\n",
    "The variable Name contains 891 different values, one for each of the passengers. We wouldn't be able to use this variable as is. However, we can extract some data from it, for example the title. See below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>Title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>Mr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>Mrs</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>Miss</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>Mrs</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>Mr</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                Name Title\n",
       "0                            Braund, Mr. Owen Harris    Mr\n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...   Mrs\n",
       "2                             Heikkinen, Miss. Laina  Miss\n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)   Mrs\n",
       "4                           Allen, Mr. William Henry    Mr"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def get_title(passenger):\n",
    "    # extracts the title from the name variable\n",
    "    line = passenger\n",
    "    if re.search('Mrs', line):\n",
    "        return 'Mrs'\n",
    "    elif re.search('Mr', line):\n",
    "        return 'Mr'\n",
    "    elif re.search('Miss', line):\n",
    "        return 'Miss'\n",
    "    elif re.search('Master', line):\n",
    "        return 'Master'\n",
    "    else:\n",
    "        return 'Other'\n",
    "    \n",
    "data['Title'] = data['Name'].apply(get_title)\n",
    "submission['Title'] = submission['Name'].apply(get_title)\n",
    "\n",
    "data[['Name', 'Title']].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# drop the original variable\n",
    "data.drop(labels='Name', inplace=True, axis=1)\n",
    "submission.drop(labels='Name', inplace=True, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1     0.602694\n",
      "2     0.180696\n",
      "3     0.114478\n",
      "4     0.032548\n",
      "6     0.024691\n",
      "5     0.016835\n",
      "7     0.013468\n",
      "11    0.007856\n",
      "8     0.006734\n",
      "Name: Family_size, dtype: float64\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x8b762a55c0>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD/CAYAAAAKVJb/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAD45JREFUeJzt3X+s3Xddx/Hna7eMMGCM2Ctgf9AaS5YZfgQuhcgSRmDY\nbcSCklhmBNHZ1FDBGI2NGIwhJlswUQyFpoFqiD8aQX4UVihGBVQg3A5rRzeKlzLXVhjlhyyDha7u\n7R/nO3I43vZ+b3t6z+1nz0fS3O/38/3k+32lPX31ez7nR1NVSJLactmkA0iSxs9yl6QGWe6S1CDL\nXZIaZLlLUoMsd0lqkOUuSQ2y3CWpQZa7JDXIcpekBq2Y1IVXrlxZ69atm9TlJemSdMcdd3yzqqYX\nmjexcl+3bh0HDx6c1OUl6ZKU5L/6zHNZRpIaZLlLUoMsd0lqkOUuSQ2y3CWpQb3KPcmmJEeTzCXZ\ncZY51yU5lORIkk+NN6YkaTEWfCtkkilgJ3A9cAKYTbKvqu4amnMV8E5gU1Xdm+THL1ZgSdLC+ty5\nbwTmqupYVZ0G9gKbR+bcDHygqu4FqKpvjDemJGkx+nyIaRVwfGj/BPCCkTnPAB6T5JPAE4G3V9V7\nxxFw3Y7bx3Ea7rn1prGcR5IuBeP6hOoK4HnAS4HHAZ9N8rmq+vLwpCRbga0Aa9euHdOlJUmj+izL\nnATWDO2v7saGnQAOVNX3quqbwKeBZ4+eqKp2V9VMVc1MTy/41QiSpPPUp9xngQ1J1ie5HNgC7BuZ\n82Hg2iQrklzBYNnm7vFGlST1teCyTFWdSbIdOABMAXuq6kiSbd3xXVV1d5KPA4eBh4F3V9UXL2Zw\nSdLZ9Vpzr6r9wP6RsV0j+28D3ja+aJKk8+UnVCWpQZa7JDXIcpekBlnuktQgy12SGmS5S1KDLHdJ\napDlLkkNstwlqUGWuyQ1yHKXpAZZ7pLUIMtdkhpkuUtSgyx3SWqQ5S5JDbLcJalBlrskNchyl6QG\nWe6S1CDLXZIaZLlLUoMsd0lqkOUuSQ2y3CWpQb3KPcmmJEeTzCXZMc/x65J8N8mh7tdbxh9VktTX\nioUmJJkCdgLXAyeA2ST7ququkan/UlWvuAgZJUmL1OfOfSMwV1XHquo0sBfYfHFjSZIuRJ9yXwUc\nH9o/0Y2N+pkkh5N8LMlPjyWdJOm8LLgs09MXgLVV9UCSG4EPARtGJyXZCmwFWLt27ZguLUka1efO\n/SSwZmh/dTf2Q1V1f1U90G3vBx6TZOXoiapqd1XNVNXM9PT0BcSWJJ1Ln3KfBTYkWZ/kcmALsG94\nQpKnJkm3vbE777fGHVaS1M+CyzJVdSbJduAAMAXsqaojSbZ1x3cBrwZ+I8kZ4EFgS1XVRcwtSTqH\nXmvu3VLL/pGxXUPb7wDeMd5okqTz5SdUJalBlrskNchyl6QGWe6S1CDLXZIaZLlLUoMsd0lqkOUu\nSQ2y3CWpQZa7JDXIcpekBlnuktQgy12SGmS5S1KDLHdJapDlLkkNstwlqUGWuyQ1yHKXpAZZ7pLU\nIMtdkhpkuUtSgyx3SWqQ5S5JDbLcJalBlrskNahXuSfZlORokrkkO84x7/lJziR59fgiSpIWa8Fy\nTzIF7ARuAK4BXpPkmrPMuw34xLhDSpIWp8+d+0ZgrqqOVdVpYC+weZ55vwn8PfCNMeaTJJ2HPuW+\nCjg+tH+iG/uhJKuAVwHvGl80SdL5GtcLqn8G/F5VPXyuSUm2JjmY5OCpU6fGdGlJ0qgVPeacBNYM\n7a/uxobNAHuTAKwEbkxypqo+NDypqnYDuwFmZmbqfENLks6tT7nPAhuSrGdQ6luAm4cnVNX6R7aT\n/CXw0dFilyQtnQXLvarOJNkOHACmgD1VdSTJtu74roucUZK0SH3u3Kmq/cD+kbF5S72qfuXCY0mS\nLoSfUJWkBlnuktQgy12SGmS5S1KDLHdJapDlLkkNstwlqUGWuyQ1yHKXpAZZ7pLUIMtdkhpkuUtS\ngyx3SWqQ5S5JDbLcJalBlrskNchyl6QGWe6S1CDLXZIaZLlLUoMsd0lqkOUuSQ2y3CWpQZa7JDXI\ncpekBvUq9ySbkhxNMpdkxzzHNyc5nORQkoNJrh1/VElSXysWmpBkCtgJXA+cAGaT7Kuqu4am/SOw\nr6oqybOAvwOuvhiBJUkL63PnvhGYq6pjVXUa2AtsHp5QVQ9UVXW7jwcKSdLE9Cn3VcDxof0T3diP\nSPKqJF8Cbgd+dTzxJEnnY2wvqFbVB6vqauCVwFvnm5Nka7cmf/DUqVPjurQkaUSfcj8JrBnaX92N\nzauqPg38ZJKV8xzbXVUzVTUzPT296LCSpH76lPsssCHJ+iSXA1uAfcMTkvxUknTbzwUeC3xr3GEl\nSf0s+G6ZqjqTZDtwAJgC9lTVkSTbuuO7gF8AXpvkIeBB4BeHXmCVJC2xBcsdoKr2A/tHxnYNbd8G\n3DbeaJKk8+UnVCWpQZa7JDXIcpekBlnuktQgy12SGmS5S1KDLHdJapDlLkkNstwlqUGWuyQ1yHKX\npAZZ7pLUIMtdkhpkuUtSgyx3SWqQ5S5JDbLcJalBlrskNchyl6QGWe6S1CDLXZIaZLlLUoMsd0lq\nkOUuSQ2y3CWpQb3KPcmmJEeTzCXZMc/xX0pyOMmdST6T5NnjjypJ6mvBck8yBewEbgCuAV6T5JqR\naV8FXlxVzwTeCuwed1BJUn997tw3AnNVdayqTgN7gc3DE6rqM1X1nW73c8Dq8caUJC1Gn3JfBRwf\n2j/RjZ3NrwEfu5BQkqQLs2KcJ0vyEgblfu1Zjm8FtgKsXbt2nJeWJA3pc+d+ElgztL+6G/sRSZ4F\nvBvYXFXfmu9EVbW7qmaqamZ6evp88kqSeuhT7rPAhiTrk1wObAH2DU9Ishb4APDLVfXl8ceUJC3G\ngssyVXUmyXbgADAF7KmqI0m2dcd3AW8Bfgx4ZxKAM1U1c/FiS5LOpdeae1XtB/aPjO0a2r4FuGW8\n0SRJ58tPqEpSgyx3SWrQWN8K+WixbsftYznPPbfeNJbzSNIo79wlqUGWuyQ1yHKXpAZZ7pLUIMtd\nkhpkuUtSgyx3SWqQ5S5JDbLcJalBlrskNchyl6QGWe6S1CDLXZIaZLlLUoMsd0lqkOUuSQ2y3CWp\nQZa7JDXIcpekBlnuktQgy12SGmS5S1KDVkw6gMZj3Y7bx3aue269aWznkjQZve7ck2xKcjTJXJId\n8xy/Oslnk/wgye+MP6YkaTEWvHNPMgXsBK4HTgCzSfZV1V1D074NvBF45UVJKUlalD537huBuao6\nVlWngb3A5uEJVfWNqpoFHroIGSVJi9Sn3FcBx4f2T3RjkqRlaknfLZNka5KDSQ6eOnVqKS8tSY8q\nfcr9JLBmaH91N7ZoVbW7qmaqamZ6evp8TiFJ6qFPuc8CG5KsT3I5sAXYd3FjSZIuxILvlqmqM0m2\nAweAKWBPVR1Jsq07vivJU4GDwJXAw0l+C7imqu6/iNklSWfR60NMVbUf2D8ytmto++sMlmskScuA\nXz8gSQ2y3CWpQZa7JDXIcpekBlnuktQgy12SGmS5S1KDLHdJapDlLkkNstwlqUGWuyQ1yHKXpAZZ\n7pLUIMtdkhpkuUtSgyx3SWqQ5S5JDbLcJalBvf6bPel8rNtx+9jOdc+tN43tXNKjgXfuktQgy12S\nGuSyjB5VXCrSo4V37pLUIO/cpWVgXM8ofDahR1jukublPziXtl7LMkk2JTmaZC7JjnmOJ8mfd8cP\nJ3nu+KNKkvpa8M49yRSwE7geOAHMJtlXVXcNTbsB2ND9egHwru6nJI2NL4j312dZZiMwV1XHAJLs\nBTYDw+W+GXhvVRXwuSRXJXlaVX1t7IklaRlZrv/g9FmWWQUcH9o/0Y0tdo4kaYks6QuqSbYCW7vd\nB5IcHdOpVwLfPOe1bxvTlfq7JDPBkudajpngEv3zMxOwPB9T48z09D6T+pT7SWDN0P7qbmyxc6iq\n3cDuPsEWI8nBqpoZ93kvhJn6WY6ZYHnmMlM/ZhrosywzC2xIsj7J5cAWYN/InH3Aa7t3zbwQ+K7r\n7ZI0OQveuVfVmSTbgQPAFLCnqo4k2dYd3wXsB24E5oDvA6+/eJElSQvpteZeVfsZFPjw2K6h7QLe\nMN5oizL2pZ4xMFM/yzETLM9cZurHTEAGvSxJaolfHCZJDbLcJalBlrskNchyH5MkVyd5aZInjIxv\nmmCmjUme321fk+S3k9w4qTzzSfLeCV//BUmu7LYfl+SPknwkyW1JnjTJbMOSXNv9+b18ghnemGTN\nwjMf3ZJcnuS1SV7W7d+c5B1J3pDkMUuWo6UXVJO8vqr+YgLXfSODdwvdDTwHeFNVfbg79oWqWvJv\nyUzyhwy+0G0F8A8Mvsjtnxl8AdyBqvrjCWQa/XxEgJcA/wRQVT83gUxHgGd3b/ndzeCtvO8HXtqN\n//xSZ+pyfb6qNnbbv87g8fVB4OXAR6rq1glk+i7wPeArwN8C76uqU0udY7lL8tcM/t5dAfwP8ATg\nAwweU6mq1y1JkKpq5hdw74SueyfwhG57HXCQQcED/PsEM011D7D7gSu78ccBhyeU6QvAXwHXAS/u\nfn6t237xhDLdPZxv5NihSWQafdww+CDhdLf9eODOSWVi8Gz/5cB7gFPAx4HXAU+c1O/VOfJ+bELX\nPdz9XAHcB0x1+1nKv3uX3H/WkeTw2Q4BT1nKLEMuq6oHAKrqniTXAe9P8vQu1yScqar/Bb6f5CtV\ndX+X78EkD08o0wzwJuDNwO9W1aEkD1bVpyaUB+CLQ8/4/iPJTFUdTPIM4KEJ5rosyZMZlOlUdXfI\nVfW9JGcmlKmq6mHgE8AnuiWGG4DXAH8CTC91oHP83xFh8Cx6Ei7rPs3/eAY3V08Cvg08FliyZZlL\nrtwZFPjPAt8ZGQ/wmaWPA8B9SZ5TVYcAquqBJK8A9gDPnFCm00muqKrvA897ZLBbR55IuXfF8KdJ\n3tf9vI/JPwZvAd6e5A8YfLHTZ5McZ/Atp7dMMNeTgDsYPK7rka/Q7l7TmdQNw49ct6oeYvDVI/uS\nXDGZSMwCn2L+35OrljjLI94DfInBM+c3A+9Lcgx4IbB3qUJccmvuSd4D/EVV/es8x/6mqm6eQKbV\nDO6Uvz7PsRdV1b9NINNjq+oH84yvBJ5WVXcudaZ5stwEvKiqfn8ZZLkSWM/gH5sTVXXfhCPNqyvR\np1TVVydw7WdU1ZeX+rrnkuSLwKuq6j/nOXa8qibyAnCSnwCoqv9OchXwMgbLxp9fsgyXWrlL0iOS\nvJrBaxD/7+vDk7yyqj40gVjLwqSfEkvSeauq95/j8JOXLMgy5J27pCYlubeq1k46x6R45y7pkrVM\n3z23LFjuki5ly/Hdc8uC5S7pUvZRBh8gPDR6IMknlz7O8uGauyQ1yC8Ok6QGWe6S1CDLXZIaZLlL\nUoMsd0lq0P8B6t0EmuIpZiYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x8b762aacf8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# create a variable indicating family size (including the passenger)\n",
    "# sums siblings and parents\n",
    "\n",
    "data['Family_size'] = data['SibSp']+data['Parch']+1\n",
    "submission['Family_size'] = submission['SibSp']+submission['Parch']+1\n",
    "\n",
    "print(data.Family_size.value_counts()/ np.float(len(data)))\n",
    "\n",
    "(data.Family_size.value_counts() / np.float(len(data))).plot.bar()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The new variable Family size is discrete, because it is the sum of 2 discrete variables. It takes a finite number of values, and large families were rare on the Titanic. In fact, families larger than 7 people were rare, so I will **cap family size at 7**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sex</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Age</th>\n",
       "      <th>is_mother</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>male</td>\n",
       "      <td>0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>female</td>\n",
       "      <td>0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>female</td>\n",
       "      <td>0</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>female</td>\n",
       "      <td>0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>male</td>\n",
       "      <td>0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Sex  Parch   Age  is_mother\n",
       "0    male      0  22.0          0\n",
       "1  female      0  38.0          0\n",
       "2  female      0  26.0          0\n",
       "3  female      0  35.0          0\n",
       "4    male      0  35.0          0"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# variable indicating if passenger was a mother\n",
    "data['is_mother'] = np.where((data.Sex =='female')&(data.Parch>=1)&(data.Age>18),1,0)\n",
    "submission['is_mother'] = np.where((submission.Sex =='female')&(submission.Parch>=1)&(submission.Age>18),1,0)\n",
    "\n",
    "data[['Sex', 'Parch', 'Age', 'is_mother']].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sex</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Age</th>\n",
       "      <th>is_mother</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>female</td>\n",
       "      <td>2</td>\n",
       "      <td>27.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>female</td>\n",
       "      <td>5</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>88</th>\n",
       "      <td>female</td>\n",
       "      <td>2</td>\n",
       "      <td>23.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>female</td>\n",
       "      <td>1</td>\n",
       "      <td>34.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>136</th>\n",
       "      <td>female</td>\n",
       "      <td>2</td>\n",
       "      <td>19.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Sex  Parch   Age  is_mother\n",
       "8    female      2  27.0          1\n",
       "25   female      5  38.0          1\n",
       "88   female      2  23.0          1\n",
       "98   female      1  34.0          1\n",
       "136  female      2  19.0          1"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.loc[data.is_mother==1, ['Sex', 'Parch', 'Age', 'is_mother']].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "there were 64 mothers in the Titanic\n"
     ]
    }
   ],
   "source": [
    "print('there were {} mothers in the Titanic'.format(data.is_mother.sum()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Types of problems within variables II (section 3)\n",
    "\n",
    "Let's look for missing data, outliers, cardinality and rare labels in the newly created variables.\n",
    "\n",
    "#### New numerical variables: Missing values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Cabin_numerical     0.775533\n",
       "Ticket_numerical    0.004489\n",
       "is_mother           0.000000\n",
       "Family_size         0.000000\n",
       "dtype: float64"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[['Cabin_numerical', 'Ticket_numerical', 'is_mother', 'Family_size']].isnull().mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cabin_numerical, as expected contains the same amount of missing data than the original variable Cabin. \n",
    "\n",
    "Ticket, also contains a small percentage of missing values. The other newly created variables do not contain missing data, as expected.\n",
    "\n",
    "#### New numerical variables: Outliers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x8b7662d438>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3gAAAF3CAYAAAD3p+myAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu85nVd7/3XW0BAVA6pqxGooULb6CTqRB663UvxQGKS\n3W5vDA3UGu8dedh72jloKW43OyqxA5U6JolJEpoGgWlILrstBUGB4RioozJxMNuio4kOfO4/rt/k\nNcu11lxrrev0+83r+Xisx7qu7+/0/l7XWuu7PtfvlKpCkiRJktR+95t0AEmSJEnScFjgSZIkSVJH\nWOBJkiRJUkdY4EmSJElSR1jgSZIkSVJHWOBJkiRJUkdY4EmSJElSR1jgSZIkSVJHWOBJkiRJUkdY\n4EmSJElSR+w96QCDeMhDHlJr165d1jLf/OY3OeCAA0YTaMTMPjltzm/2yWlz/mnLftVVV/1rVT10\n0jnaYiXj40Km7edg2Lrcvy73Dexf23W5f5Po26BjZCsKvLVr13LllVcua5m5uTlmZ2dHE2jEzD45\nbc5v9slpc/5py57ki5PO0CYrGR8XMm0/B8PW5f51uW9g/9quy/2bRN8GHSM9RFOSJEmSOsICT5Ik\nSZI6wgJPkiRJkjrCAk+SJEmSOsICT5IkSZI6wgJPkiRJkjrCAk+SJEmSOsICT5IkSZI6wgJPkiRJ\nkjpiZAVeknOS3JXkur62301yU5Jrk3wwyUGj2r4kSZIk7WlGuQfvXcBx89ouBR5dVT8B/DNw2gi3\nL0mSJEl7lJEVeFX1D8C/zWv7u6ra0Tz9FHDYqLYvSZIkSXuaSZ6D91Lgbye4fUmSJEnqlL0nsdEk\nrwN2AOctMc8GYAPAzMwMc3Nzy9rG9u3bB15my7a7F5227tADl7XdYVhO9mnT5uzQ7vxmn5w2529z\ndg3Plm13c8qmSxactvXM48ecRpK0GmMv8JKcAjwHOLaqarH5qmozsBlg/fr1NTs7u6ztzM3NMegy\niw1qAFtPWt52h2E52adNm7NDu/ObfXLanL/N2SVJ0vcba4GX5Djg14H/XFXfGue2JUmSJKnrRnmb\nhPcCnwQemeS2JC8D/gh4EHBpkquTvG1U25ckSZKkPc3I9uBV1QsXaH7nqLYnSZIkSXu6SV5FU5Ik\nSZI0RBZ4kiRJktQRFniSJEmS1BEWeJIkSZLUERZ4kiSNUJK9knw2ycXN80OSXJrklub7wX3znpbk\n1iQ3J3nW5FJLktrKAk+SpNF6FXBj3/NNwGVVdSRwWfOcJEcBJwKPAo4D/iTJXmPOKklqOQs8SZJG\nJMlhwPHAn/Y1nwCc2zw+F/i5vvbzq+qeqvoCcCtwzLiySpK6wQJPkqTR+X3g14H7+tpmqur25vEd\nwEzz+FDgy33z3da0SZI0sJHd6FySpD1ZkucAd1XVVUlmF5qnqipJLXO9G4ANADMzM8zNza02KjP7\nw8Z1OxacNoz1T9r27ds70Y+FdLlvYP/arsv9m+a+WeBJkjQaTwaem+TZwH7Ag5O8B7gzyZqquj3J\nGuCuZv5twOF9yx/WtO2iqjYDmwHWr19fs7Ozqw569nkXctaWhf8l2HrS6tc/aXNzcwzjdZpGXe4b\n2L+263L/prlvHqIpSdIIVNVpVXVYVa2ld/GUv6+qFwEXASc3s50MXNg8vgg4Mcm+SY4AjgSuGHNs\nSVLLuQdPkqTxOhO4IMnLgC8CLwCoquuTXADcAOwATq2qeycXU5LURhZ4kiSNWFXNAXPN468Cxy4y\n3xnAGWMLJknqHA/RlCRJkqSOsMCTJEmSpI6wwJMkSZKkjrDAkyRJkqSOsMCTJEmSpI6wwJMkSZKk\njrDAkyRJkqSOsMCTJEmSpI7wRudTZu2mS9i4bgenbLrk+6ZtPfP4oW9rMcPe1mq0JackSZI0ae7B\nkyRJkqSOsMCTJEmSpI6wwJMkSZKkjrDAkyRJkqSOsMCTJEmSpI6wwJMkSZKkjrDAkyRJkqSOsMCT\nJEmSpI6wwJMkSZKkjrDAkyRJkqSOsMCTJEmSpI6wwJMkSZKkjrDAkyRJkqSOsMCTJEmSpI6wwJMk\nSZKkjrDAkyRJkqSOsMCTJEmSpI6wwJMkSZKkjrDAkyRJkqSOsMCTJGkEkuyX5Iok1yS5Pskbm/bT\nk2xLcnXz9ey+ZU5LcmuSm5M8a3LpJUlttfekA0iS1FH3AE+rqu1J9gE+keRvm2m/V1Vv7p85yVHA\nicCjgIcDH03yiKq6d6ypJUmtNrI9eEnOSXJXkuv62g5JcmmSW5rvB49q+5IkTVL1bG+e7tN81RKL\nnACcX1X3VNUXgFuBY0YcU5LUMaM8RPNdwHHz2jYBl1XVkcBlzXNJkjopyV5JrgbuAi6tqsubSa9I\ncm3zYejODzsPBb7ct/htTZskSQMb2SGaVfUPSdbOaz4BmG0enwvMAa8ZVQZJkiapObzy6CQHAR9M\n8mjgrcCb6O3NexNwFvDSQdeZZAOwAWBmZoa5ublV55zZHzau27HgtGGsf9K2b9/eiX4spMt9A/vX\ndl3u3zT3bdzn4M1U1e3N4zuAmTFvX5KksauqryX5GHBc/7l3Sd4BXNw83QYc3rfYYU3b/HVtBjYD\nrF+/vmZnZ1ed7+zzLuSsLQv/S7D1pNWvf9Lm5uYYxus0jbrcN7B/bdfl/k1z3yZ2kZWqqiSLnouw\n2k8ol1NVL/apJYz/k8uN63Ys+knq2edduOhy6w49cEXbWsxKtzWKTzPG+f5M86cxu2P2yWlz/jZn\nn3ZJHgp8tynu9geeAfx2kjV9H3Y+D9h5rvpFwF8keQu9i6wcCVwx7tySpHYbd4F3586BLckaeuck\nLGi1n1Aup6o+ZdMli04b9yeXp2y6hI3rdiz6SepiVpJzqX6vdFuj+DRjnO/PNH8asztmn5w2529z\n9hZYA5ybZC9657xfUFUXJ/nzJEfTO0RzK/BygKq6PskFwA3ADuBUr6ApSVqucRd4FwEnA2c23xff\nTSRJUotV1bXAYxdof/ESy5wBnDHKXJKkbhvlbRLeC3wSeGSS25K8jF5h94wktwBPb55LkiRJkoZg\nlFfRfOEik44d1TYlSZIkaU82yvvgSZIkSZLGyAJPkiRJkjrCAk+SJEmSOsICT5IkSZI6wgJPkiRJ\nkjrCAk+SJEmSOsICT5IkSZI6wgJPkiRJkjrCAk+SJEmSOsICT5IkSZI6wgJPkiRJkjrCAk+SJEmS\nOsICT5IkSZI6wgJPkiRJkjrCAk+SJEmSOsICT5IkSZI6wgJPkiRJkjrCAk+SJEmSOsICT5IkSZI6\nwgJPkiRJkjrCAk+SJEmSOmLvSQcYl7WbLpl0BEmSJEkaKffgSZIkSVJHWOBJkiRJUkdY4EmSJElS\nR1jgSZIkSVJHWOBJkjQCSfZLckWSa5Jcn+SNTfshSS5Nckvz/eC+ZU5LcmuSm5M8a3LpJUltZYEn\nSdJo3AM8raoeAxwNHJfkCcAm4LKqOhK4rHlOkqOAE4FHAccBf5Jkr4kklyS1lgWeJEkjUD3bm6f7\nNF8FnACc27SfC/xc8/gE4PyquqeqvgDcChwzxsiSpA6wwJMkaUSS7JXkauAu4NKquhyYqarbm1nu\nAGaax4cCX+5b/LamTZKkge0xNzqXJGncqupe4OgkBwEfTPLoedMrSS1nnUk2ABsAZmZmmJubW3XO\nmf1h47odC04bxvonbfv27Z3ox0K63Dewf23X5f5Nc98s8CRJGrGq+lqSj9E7t+7OJGuq6vYka+jt\n3QPYBhzet9hhTdv8dW0GNgOsX7++ZmdnV53v7PMu5KwtC/9LsPWk1a9/0ubm5hjG6zSNutw3sH9t\n1+X+TXPfPERTkqQRSPLQZs8dSfYHngHcBFwEnNzMdjJwYfP4IuDEJPsmOQI4ErhivKklSW3nHjxJ\nkkZjDXBucyXM+wEXVNXFST4JXJDkZcAXgRcAVNX1SS4AbgB2AKc2h3hKkjQwCzxJkkagqq4FHrtA\n+1eBYxdZ5gzgjBFHkyR12G4P0Uzy5CQHNI9flOQtSX549NEkSZo8x0FJUpsMcg7eW4FvJXkMsBH4\nHPDukaaSJGl6OA5KklpjkAJvR1XtvDHrH1XVHwMPGm0sSZKmhuOgJKk1BjkH7xtJTgNeBDwlyf2A\nfUYbS5KkqeE4KElqjUH24P0/wD3Ay6rqDnr35fndkaaSJGl6OA5KklpjyT14zaWd31tVT93ZVlVf\nwnMPJEl7AMdBSVLbLLkHr7n/zn1JDhxTHkmSpobjoCSpbQY5B287sCXJpcA3dzZW1StHlkqSpOnh\nOChJao1BCrwPNF9Dk+S/Ab8EFLAFeElVfXuY25AkaUiGPg5KkjQquy3wqurcJPsDP1RVN692g0kO\nBV4JHFVV/57kAuBE4F2rXbckScM27HFQkqRR2u1VNJP8LHA18OHm+dFJLlrldvcG9k+yN/AA4F9W\nuT5JkkZiROOgJEkjMchtEk4HjgG+BlBVVwM/stINVtU24M3Al4Dbgbur6u9Wuj5JkkbsdIY4DkqS\nNEqDnIP33aq6O0l/230r3WCSg4ETgCPoDZbvS/KiqnrPvPk2ABsAZmZmmJubW9Z2tm/fvssyG9ft\nWFHe5W53EFu23b3otI3rYGb/5eddSc5RvCbzX/dhWCrnsLc1ivzjYvbJaXP+Nmcfo6GOg5IkjdIg\nBd71SX4B2CvJkfTOn/unVWzz6cAXquorAEk+ADwJ2KXAq6rNwGaA9evX1+zs7LI2Mjc3R/8yp2y6\nZEVht560vO0OYndZNq7bwVlbBnlrvmclOUfxmsx/3YdhqZzDfn9GkX9czD45bc7f5uxjNOxxUJKk\nkRnkEM1XAI8C7gHeC3wdePUqtvkl4AlJHpDex6HHAjeuYn2SJI3SsMdBSZJGZpCraH4LeF3ztWpV\ndXmS9wOfAXYAn6XZUydJ0rQZ9jgoSdIo7bbAS/I39O5X1+9u4Erg7Su5f11VvQF4w3KXkyRp3EYx\nDkqSNCqDHKL5eWA78I7m6+vAN4BHNM8lSeoyx0FJUmsMciWPJ1XVT/Y9/5skn66qn0xy/aiCSZI0\nJRwHJUmtMcgevAcm+aGdT5rHD2yefmckqSRJmh6Og5Kk1hhkD95G4BNJPgeE3v3rfiXJAcC5owwn\nSdIUcByUJLXGIFfR/FBz358fb5pu7juh/PdHlkySpCngOChJapNB76b9eGBtM/9jklBV7x5ZKkmS\npovjoCSpFQa5TcKfAz8KXA3c2zQX4MAmSeo8x0FJUpsMsgdvPXBUVc2/B5AkSXsCx0FJUmsMchXN\n64AfHHUQSZKmlOOgJKk1BtmD9xDghiRXAPfsbKyq544slSRJ08NxUJLUGoMUeKePOoQkSVPs9EkH\nkCRpUIPcJuHjSX4YOLKqPprkAcBeo48mSdLkOQ5Kktpkt+fgJfll4P3A25umQ4G/HmUoSZKmxUrH\nwSSHJ/lYkhuSXJ/kVU376Um2Jbm6+Xp23zKnJbk1yc1JnjWK/kiSum2QQzRPBY4BLgeoqluSPGyk\nqSRJmh4rHQd3ABur6jNJHgRcleTSZtrvVdWb+2dOchRwIvAo4OHAR5M8oqruRZKkAQ1yFc17quo7\nO58k2Zve/X8kSdoTrGgcrKrbq+ozzeNvADfS2/u3mBOA86vqnqr6AnArvcJSkqSBDVLgfTzJa4H9\nkzwDeB/wN6ONJUnS1Fj1OJhkLfBYmr2AwCuSXJvknCQHN22HAl/uW+w2li4IJUn6PoMcorkJeBmw\nBXg58CHgT0cZSpKkKbKqcTDJA4G/Al5dVV9P8lbgTfT2Ar4JOAt46TLWtwHYADAzM8Pc3Nygiy5q\nZn/YuG7HgtOGsf5J2759eyf6sZAu9w3sX9t1uX/T3LdBrqJ5H/AO4B1JDgEOqyoP0ZQk7RFWMw4m\n2YdecXdeVX2gWd+dfdPfAVzcPN0GHN63+GFN2/w8m4HNAOvXr6/Z2dnldun7nH3ehZy1ZeF/Cbae\ntPr1T9rc3BzDeJ2mUZf7Bvav7brcv2nu2yBX0ZxL8uBmULuK3gD3e6OPJknS5K10HEwS4J3AjVX1\nlr72NX2zPQ+4rnl8EXBikn2THAEcCVwxrH5IkvYMgxyieWBzSMkvAe+uqjckuXbUwSRJmhIrHQef\nDLwY2JLk6qbttcALkxxN7xDNrfQO+6Sqrk9yAXADvStwnuoVNCVJyzVIgbd382njC4DXjTiPJEnT\nZkXjYFV9AsgCkz60xDJnAGcsO6EkSY1BCrz/CXwE+ERVfTrJjwC3jDaW1C5rN12y6LStZx4/xiSS\nRsBxUJLUGoNcZOV99C4JvfP554H/e5ShJEmaFo6DkqQ2GeQiK7/TnFy+T5LLknwlyYvGEU6SpElz\nHJQktckgNzp/ZlV9HXgOvZPBfwz4H6MMJUnSFHEclCS1xiAF3s7DOI8H3ldVd48wjyRJ08ZxUJLU\nGoNcZOXiJDcB/w781yQPBb492liSJE0Nx0FJUmvsdg9eVW0CngSsr6rvAt8EThh1MEmSpoHjoCSp\nTQbZgwfwcODpSfbra3v3CPJIkjSNHAfHwFvOSNLq7bbAS/IGYBY4it7NWX8G+AQObJKkPYDjoCSp\nTQa5yMrzgWOBO6rqJcBjgANHmkqSpOnhOChJao1BCrx/r6r7gB1JHgzcBRw+2liSJE0Nx0FJUmsM\ncg7elUkOAt4BXAVsBz450lSSJE0Px0FJUmvstsCrql9pHr4tyYeBB1fVtaONJUnSdHAclCS1yUBX\n0Uzy88BPA0XvxHIHNknSHsNxUJLUFrs9By/JnwD/L7AFuA54eZI/HnUwSZKmgeOgJKlNBtmD9zTg\nP1VVASQ5F7h+pKkkSZoejoOSpNYY5CqatwI/1Pf88KZNkqQ9geOgJKk1BtmD9yDgxiRX0Dv34Bh6\nVxS7CKCqnjvCfJIkTZrjoCSpNQYp8F4/8hSSJE0vx0FJUmsMcpuEj48jiCRJ08hxUJLUJoOcgydJ\nkiRJagELPEmSJEnqiEULvCSXNd9/e9gbTXJQkvcnuSnJjUmeOOxtSJK0GqMcByVJGpWlzsFbk+RJ\nwHOTnA+kf2JVfWYV2/0D4MNV9fwk9wcesIp1SZI0CqMcByVJGomlCrzXA78JHAa8Zd60onfj12VL\nciDwFOAUgKr6DvCdlaxLkqQRGsk4KEnSKC1a4FXV+4H3J/nNqnrTELd5BPAV4M+SPAa4CnhVVX2z\nf6YkG4ANADMzM8zNzS1rI9u3b99lmY3rdqwo7NnnXbjotHWHHriide4uy8z+y8+7WM6lMq70NVnq\nvZj/ug9qy7a7F522cd3KsqzESvMv9VoOO+NiVpp9Uvrf85n9v/czvNLfq0lq22vfr83ZR22E46Ak\nSSMzyG0S3pTkufT2ugHMVdXFq9zm44BXVNXlSf4A2ETvU9L+7W4GNgOsX7++Zmdnl7WRubk5+pc5\nZdMlq4i8sK0nze52noXsLsvGdTs4a8sgtyjcvaUyrvQ1WWqd81/3QY0iy0qMIv+wMy5mpdknpf81\n6/+ZH9frNUxte+37tTn7uIxgHJQkaWR2exXNJL8FvAq4ofl6VZL/vYpt3gbcVlWXN8/fT6/gkyRp\n6oxgHJQkaWQGuU3C8cAzquqcqjoHOA54zko3WFV3AF9O8sim6Vh6A6YkSdNoReNgksOTfCzJDUmu\nT/Kqpv2QJJcmuaX5fnDfMqcluTXJzUmeNbIeSZI6a9D74B3U93gYJ8i8AjgvybXA0YCfhEqSptlK\nxsEdwMaqOgp4AnBqkqPonZZwWVUdCVzWPKeZdiLwKHpF5J8k2WtI+SVJe4hBTvT6LeCzST5G7xLR\nT6EZjFaqqq4G1q9mHZIkjcmKxsGquh24vXn8jSQ3AocCJwCzzWznAnPAa5r286vqHuALSW4FjgE+\nOczOSJK6bZCLrLw3yRzwk03Ta5rDLCVJ6rxhjINJ1gKPBS4HZpriD+AOYKZ5fCjwqb7FbmvaJEka\n2ECXamwGootGnEWSpKm0mnEwyQOBvwJeXVVfT753v/SqqiS1zPWt6jZCC1nq9jzjvI3GqG450+Xb\ngXS5b2D/2q7L/Zvmvg3nWvySJOn7JNmHXnF3XlV9oGm+M8maqro9yRrgrqZ9G3B43+KHNW27WO1t\nhBZy9nkXLnp7nnHeumRUt5zp8u1Autw3sH9t1+X+TXPfBr3IiiRJWob0dtW9E7ixqt7SN+ki4OTm\n8cnAhX3tJybZN8kRwJHAFePKK0nqhiULvCR7JblpXGEkSZomqxwHnwy8GHhakqubr2cDZwLPSHIL\n8PTmOVV1PXABvVsHfRg4taruXXUnJEl7lCUP0ayqe5t78fxQVX1pXKEkSZoGqxkHq+oT9K66uZBj\nF1nmDOCMZcaUJOk/DHIO3sHA9UmuAL65s7GqnjuyVJIkTQ/HQUlSawxS4P3myFNIkjS9HAclSa0x\nyH3wPp7kh4Ejq+qjSR4A7DX6aJIkTZ7joCSpTXZ7Fc0kvwy8H3h703Qo8NejDCVJ0rRwHJQktckg\nt0k4ld6VwL4OUFW3AA8bZShJkqaI46AkqTUGKfDuqarv7HySZG+gRhdJkqSp4jgoSWqNQQq8jyd5\nLbB/kmcA7wP+ZrSxJEmaGo6DkqTWGKTA2wR8BdgCvBz4EPAbowwlSdIUcRyUJLXGIFfRvC/JucDl\n9A5JubmqPDRFkrRHcByUJLXJbgu8JMcDbwM+BwQ4IsnLq+pvRx1OkqRJcxyUJLXJIDc6Pwt4alXd\nCpDkR4FLAAc2SdKewHFQktQag5yD942dg1rj88A3RpRHkqRp4zgoSWqNRffgJfn55uGVST4EXEDv\n3IP/Anx6DNkkSZoYx0FJUhstdYjmz/Y9vhP4z83jrwD7jyyRJEnTwXFQktQ6ixZ4VfWScQbRaKzd\ndMmkI2gEFnpfN67bwSmbLmHrmcdPIJHUPY6DkqQ2GuQqmkcArwDW9s9fVc8dXSxJkqaD46AkqU0G\nuYrmXwPvBP4GuG+0cSRJmjqOg5Kk1hikwPt2Vf3hyJNIkjSdHAclSa0xSIH3B0neAPwdcM/Oxqr6\nzMhSSZI0PRwHJUmtMUiBtw54MfA0vndoSjXPJUnqOsdBSVJrDFLg/RfgR6rqO6MOI0nSFHIclCS1\nxv0GmOc64KBRB5EkaUo5DkqSWmOQPXgHATcl+TS7nnvg5aElSXsCx0FJUmsMUuC9YeQpJEmaXo6D\nkqTW2G2BV1UfH0cQSZKmkeOgJKlNdlvgJfkGvauFAdwf2Af4ZlU9eJTBJEmaBo6DkqQ2GWQP3oN2\nPk4S4ATgCaMMJUnStHAclCS1ySBX0fwP1fPXwLNGlEeSpKnlOChJmnaDHKL5831P7wesB749skSS\nJE2RlY6DSc4BngPcVVWPbtpOB34Z+Eoz22ur6kPNtNOAlwH3Aq+sqo8Mqw+SpD3HIFfR/Nm+xzuA\nrfQOT5EkaU+w0nHwXcAfAe+e1/57VfXm/oYkRwEnAo8CHg58NMkjqureFWaWJO2hBjkH7yXjCCJJ\n0jRa6ThYVf+QZO2As58AnF9V9wBfSHIrcAzwyZVsW5K051q0wEvy+iWWq6p60wjySJI0FUY4Dr4i\nyS8CVwIbq+r/AIcCn+qb57amTZKkZVlqD943F2g7gN75AT8AWOBJkrpsFOPgW5vlqvl+FvDS5awg\nyQZgA8DMzAxzc3MriLGrmf1h47odC04bxvoHtViG1ebYvn37WPsxTl3uG9i/tuty/6a5b4sWeFV1\n1s7HSR4EvAp4CXA+vQFJkqTOGsU4WFV39q3zHcDFzdNtwOF9sx7WtC20js3AZoD169fX7OzsSqLs\n4uzzLuSsLQv/S7D1pNWvf1CnbLpk0WmryTE3N8cwXqdp1OW+gf1ruy73b5r7tuRtEpIckuR/AdfS\nKwYfV1Wvqaq7xpJOkqQJGvY4mGRN39PnAdc1jy8CTkyyb5IjgCOBK1YRXZK0h1rqHLzfBX6e3qeE\n66pq+9hSSZI0YasdB5O8F5gFHpLkNuANwGySo+kdorkVeDlAVV2f5ALgBnpX6jzVK2hKklZiqXPw\nNgL3AL8BvC7JzvbQO7n8wavZcJK96J1gvq2qnrOadUmSNAKrGger6oULNL9zifnPAM5YWVRJknqW\nOgdvycM3h+BVwI3AqgpFSZJGYQzjoCRJQzeRwSvJYcDxwJ9OYvuSJEmS1EWT+nTy94FfB+6b0PYl\nSZIkqXOWOgdvJJI8B7irqq5KMrvEfKu6z8/8e1MsdW+dlVrpvS92l2Wp+xFNg7PPu3DRaUccuNei\nr8uWbXcvutzGdSvLMuz7j6z0niajunfTcra38+dmqfdn3aEHrmh7S713K9X/nvf/zE/rPWWWMv/n\nZqWv10rfn9WY5vv4SJKk5Rt7gQc8GXhukmcD+wEPTvKeqnpR/0yrvc/P/HtTLHVvnZVa6T15dpdl\n47odi96PaNq967gDFr0nyDS9B4tZ6T1NRnXvpuVsb5Cfm1H9zK5Wf/Zx3nNrWIb192YSfZ/m+/hI\nkqTlG/shmlV1WlUdVlVrgROBv59f3EmSJEmSls8rhEmSJElSR0z0OMCqmgPmJplBkiRJkrrCPXiS\nJEmS1BEWeJIkSZLUERZ4kiRJktQRFniSJEmS1BEWeJIkSZLUERZ4kiRJktQRFniSJEmS1BEWeJIk\nSZLUERZ4kiRJktQRFniSJEmS1BEWeJIkSZLUERZ4kiRJktQRFniSJEmS1BEWeJIkSZLUERZ4kiRJ\nktQRFniSJEmS1BEWeJIkSZLUERZ4kiRJktQRFniSJEmS1BEWeJIkSZLUERZ4kiSNQJJzktyV5Lq+\ntkOSXJrklub7wX3TTktya5KbkzxrMqklSW2396QDdNXaTZdMOsJEbNl2N6eMse/Dfp03rtuxZP6t\nZx4/1O2N++dkqe0Nu28r1YaM0oDeBfwR8O6+tk3AZVV1ZpJNzfPXJDkKOBF4FPBw4KNJHlFV9445\nsySp5dyDJ0nSCFTVPwD/Nq/5BODc5vG5wM/1tZ9fVfdU1ReAW4FjxhJUktQpFniSJI3PTFXd3jy+\nA5hpHh8KfLlvvtuaNkmSlsVDNCVJmoCqqiS13OWSbAA2AMzMzDA3N7fqLDP79w5RX8gw1j+oxTKs\nNsf27duLqIPmAAATYUlEQVTH2o9x6nLfwP61XZf7N819s8CTJGl87kyypqpuT7IGuKtp3wYc3jff\nYU3b96mqzcBmgPXr19fs7OyqQ5193oWctWXhfwm2nrT69Q9qyXOgV5Fjbm6OYbxO06jLfQP713Zd\n7t80981DNCVJGp+LgJObxycDF/a1n5hk3yRHAEcCV0wgnySp5dyDJ0nSCCR5LzALPCTJbcAbgDOB\nC5K8DPgi8AKAqro+yQXADcAO4FSvoClJWgkLPEmSRqCqXrjIpGMXmf8M4IzRJZIk7Qk8RFOSJEmS\nOsICT5IkSZI6wgJPkiRJkjrCAk+SJEmSOsICT5IkSZI6wgJPkiRJkjrCAk+SJEmSOsICT5IkSZI6\nwgJPkiRJkjrCAk+SJEmSOsICT5IkSZI6wgJPkiRJkjrCAk+SJEmSOsICT5IkSZI6Yu9xbzDJ4cC7\ngRmggM1V9QfjziFJknZv7aZLFp229czjx5hEkjSIsRd4wA5gY1V9JsmDgKuSXFpVN0wgiyRJkiR1\nxtgP0ayq26vqM83jbwA3AoeOO4ckSZIkdc1Ez8FLshZ4LHD5JHNIkiRJUhdM4hBNAJI8EPgr4NVV\n9fUFpm8ANgDMzMwwNze3rPVv3759l2U2rtuxirQLO/u8CxedtnHdytc7s/9o8o5Dm7PD7vMv9p4v\n9X6P6udkvtW+9kv9jo36PR00+3L/DozSlm13/8fjmf13fZ9X+r5Oon/z/1ZKkqR2m0iBl2QfesXd\neVX1gYXmqarNwGaA9evX1+zs7LK2MTc3R/8ypyxxkvi02bhuB2dtmVjtvSptzg7tzr/a7FtPml10\n2qh/fwbNvlTGcet/TYb1czOJ/s3/WylJktpt7IdoJgnwTuDGqnrLuLcvSZIkSV01iXPwngy8GHha\nkqubr2dPIIckSZIkdcrYj0Wrqk8AGfd2JUmSJKnrJnoVTUmSJEnS8LTzahKSJGmqrV3k4kxbzzx+\nzEkkac/iHjxJkiRJ6ggLPEmSJEnqCAs8SZIkSeoICzxJkiRJ6ggvsiJJ0gQk2Qp8A7gX2FFV65Mc\nAvwlsBbYCrygqv7PpDJKktrHPXiSJE3OU6vq6Kpa3zzfBFxWVUcClzXPJUkamAWeJEnT4wTg3Obx\nucDPTTCLJKmFLPAkSZqMAj6a5KokG5q2maq6vXl8BzAzmWiSpLbyHDxJkibjp6tqW5KHAZcmual/\nYlVVkpq/UFMMbgCYmZlhbm5u1UFm9oeN63Yse7mltr3Y+layzO6W253t27cP5XWaRl3uG9i/tuty\n/6a5bxZ4kiRNQFVta77fleSDwDHAnUnWVNXtSdYAdy2w3GZgM8D69etrdnZ21VnOPu9Cztqy/H8J\ntp60+LZP2XTJ0JbZ3XK7Mzc3xzBep2nU5b6B/Wu7LvdvmvvmIZqSJI1ZkgOSPGjnY+CZwHXARcDJ\nzWwnAxdOJqEkqa3cgydJ0vjNAB9MAr2x+C+q6sNJPg1ckORlwBeBF0wwoySphSzwJEkas6r6PPCY\nBdq/Chw7/kSSpNVau9Rh5mceP7YcHqIpSZIkSR1hgSdJkiRJHWGBJ0mSJEkdYYEnSZIkSR1hgSdJ\nkiRJHWGBJ0mSJEkd4W0SJAFLX9p3Wowi41KXLR73azItl1deShsySpK0J3MPniRJkiR1hAWeJEmS\nJHWEh2hKkqSp5+HBkjQY9+BJkiRJUkdY4EmSJElSR3iIpiRJGps2XLFXktrMPXiSJEmS1BEWeJIk\nSZLUERZ4kiRJktQRFniSJEmS1BEWeJIkSZLUERZ4kiRJktQRFniSJEmS1BEWeJIkSZLUERZ4kiRJ\nktQRFniSJEmS1BEWeJIkSZLUERZ4kiRJktQRFniSJEmS1BEWeJIkSZLUERZ4kiRJktQREynwkhyX\n5OYktybZNIkMkiRNI8dISdJqjL3AS7IX8MfAzwBHAS9MctS4c0iSNG0cIyVJqzWJPXjHALdW1eer\n6jvA+cAJE8ghSdK0cYyUJK3KJAq8Q4Ev9z2/rWmTJGlP5xgpSVqVVNV4N5g8Hziuqn6pef5i4Keq\n6lfnzbcB2NA8fSRw8zI39RDgX1cZd1LMPjltzm/2yWlz/mnL/sNV9dBJh5iUQcbIIYyPC5m2n4Nh\n63L/utw3sH9t1+X+TaJvA42Re48jyTzbgMP7nh/WtO2iqjYDm1e6kSRXVtX6lS4/SWafnDbnN/vk\ntDl/m7N31G7HyNWOjwvp+s9Bl/vX5b6B/Wu7Lvdvmvs2iUM0Pw0cmeSIJPcHTgQumkAOSZKmjWOk\nJGlVxr4Hr6p2JPlV4CPAXsA5VXX9uHNIkjRtHCMlSas1iUM0qaoPAR8a8WaGevjKmJl9ctqc3+yT\n0+b8bc7eSWMaI+fr+s9Bl/vX5b6B/Wu7Lvdvavs29ousSJIkSZJGYxLn4EmSJEmSRqBzBV6S45Lc\nnOTWJJsmnWcpSQ5P8rEkNyS5PsmrmvZDklya5Jbm+8GTzrqYJHsl+WySi5vnbcp+UJL3J7kpyY1J\nntiW/En+W/Mzc12S9ybZb5qzJzknyV1JrutrWzRvktOa3+GbkzxrMqn/I8tC2X+3+bm5NskHkxzU\nN21qsjd5vi9/37SNSSrJQ/rapiq/Rmvax8wkW5NsSXJ1kiubtmX/7Ujy+GY9tyb5wyRp2vdN8pdN\n++VJ1vYtc3KzjVuSnDyk/gzlb+Ew+5PeBX0ub5b5y/Qu7jPM/p2eZFvzHl6d5Nlt7F9W8D9bR/rX\nlfdvvyRXJLmm6d8bm/ZOvH/fp6o680XvhPTPAT8C3B+4Bjhq0rmWyLsGeFzz+EHAPwNHAb8DbGra\nNwG/PemsS/ThvwN/AVzcPG9T9nOBX2oe3x84qA356d30+AvA/s3zC4BTpjk78BTgccB1fW0L5m1+\nB64B9gWOaH6n95qy7M8E9m4e//a0Zl8sf9N+OL0LeXwReMi05vdrpD8bUz9mAlt3/nz2tS37bwdw\nBfAEIMDfAj/TtP8K8Lbm8YnAXzaPDwE+33w/uHl88BD6M5S/hcPsD70x5MTm8duA/zrk/p0O/NoC\n87aqfyzzf7YO9a8r71+ABzaP9wEubzJ24v37vv4OYyXT8gU8EfhI3/PTgNMmnWsZ+S8EnkHvprVr\nmrY1wM2TzrZI3sOAy4Cn8b0Cry3ZD6RXJGVe+9Tnp1fgfbn5Q7E3cDG9gmOqswNr2XXQXzDv/N9b\nekXIE6cp+7xpzwPOm9bsi+UH3g88hr5/oKc1v18j+7mY+jGThQu8Zf3taOa5qa/9hcDb++dpHu9N\n76bF6Z+nmfZ24IVD6tOq/hYOsz/NtH/lex9Y7fIzMaT+nc7CBUIr+9e3jSX/Z+tQ/zr3/gEPAD4D\n/FRX37+uHaK58x/fnW5r2qZesxv3sfQ+UZipqtubSXcAMxOKtTu/D/w6cF9fW1uyHwF8Bfiz9A4x\n/dMkB9CC/FW1DXgz8CXgduDuqvo7WpB9nsXytu33+KX0PsGDlmRPcgKwraqumTepFfk1NG14vwv4\naJKrkmxo2pb7t+PQ5vH89l2WqaodwN3ADyyxrlGYZH9+APhaM+/8dQ3TK9I7pP2cvkPgWtu/Af9n\n60r/oCPvX3qnFV0N3AVcWlWdfP+gg+fgtVGSBwJ/Bby6qr7eP616JX1NJNgSkjwHuKuqrlpsnmnN\n3tib3mEkb62qxwLfpLdr/j9Ma/7mj+sJ9IrUhwMHJHlR/zzTmn0xbcu7U5LXATuA8yadZVBJHgC8\nFnj9pLNIA/jpqjoa+Bng1CRP6Z/Y1r8di+lafxpvpXcY8NH0PpQ8a7JxVqeN/7MtxwL968z7V1X3\nNn9PDgOOSfLoedNb//7t1LUCbxu980p2Oqxpm1pJ9qH3i3ReVX2gab4zyZpm+hp6nzRMmycDz02y\nFTgfeFqS99CO7ND7lOS25tMb6B2u9jjakf/pwBeq6itV9V3gA8CTaEf2fovlbcXvcZJTgOcAJzWD\nArQj+4/S+3Dgmub39zDgM0l+kHbk1/BM/fvdHLFAVd0FfBA4huX/7djWPJ7fvssySfamd/j+V5dY\n1yhMsj9fBQ5q5p2/rqGoqjubf6zvA95B7z3cJeu8bU9t/5b5P1sn+tel92+nqvoa8DHgODr0/vXr\nWoH3aeDI5oo096d3guNFE860qOaqO+8Ebqyqt/RNugg4uXl8Mr3joKdKVZ1WVYdV1Vp6r/PfV9WL\naEF2gKq6A/hykkc2TccCN9CO/F8CnpDkAc3P0LHAjbQje7/F8l4EnNhcjeoI4Eh6JzRPjSTH0Ts8\n+blV9a2+SVOfvaq2VNXDqmpt8/t7G70T6++gBfk1VFM9ZiY5IMmDdj6md67xdSzzb0dz+NXXkzyh\n+Zv5i/OW2bmu59Mby4reuTTPTHJwc9TEM5u2UZhYf5ppH2vmnb/9odj5z3PjefTew9b1bwX/s3Wi\nfx16/x6a5orXSfand37hTXTk/fs+wziRb5q+gGfTu/LP54DXTTrPbrL+NL1dwdcCVzdfz6Z3TO5l\nwC3AR4FDJp11N/2Y5XsXWWlNdnqHG1zZvP5/Te/KRq3ID7yR3h+m64A/p3eVp6nNDryX3qEd36VX\nULxsqbzA65rf4Ztprk41ZdlvpXc8/c7f27dNY/bF8s+bvpW+i1hMW36/Rv7zMbVjJr3Dwq5pvq7f\nmW8lfzuA9c3fy88Bf0RzgS1gP+B9ze/0FcCP9C3z0qb9VuAlQ+rTUP4WDrM/zet8RdP+PmDfIffv\nz4Et9Mbai2guaNG2/rGC/9k60r+uvH8/AXy26cd1wOun4e/JsPo3/2tnIEmSJElSy3XtEE1JkiRJ\n2mNZ4EmSJElSR1jgSZIkSVJHWOBJkiRJUkdY4EmSJElSR1jgSfMk+cEk5yf5XJKrknwoySOWmH9t\nkusWmfanSY4aXdrBLJVRkqTdSfIDSa5uvu5Isq3v+T/tZtm5JOuXsa1XJ3nA6lMPLsnpSX5tnNuU\nRmXv3c8i7Tmam1Z+EDi3qk5s2h4DzNC7V9SyVNUvDTfhZCTZu6p2TDqHJGkyquqr9O4fS5LTge1V\n9eYRbe7VwHuAb41o/UPV/O+Qqrpv0lkkcA+eNN9Tge9W1dt2NlTVNVX1/yV5YJLLknwmyZYkJ/Qt\nt3eS85LcmOT9Oz957P/UMsn2JGckuSbJp5LMzN948wniOc1yn0/yyqZ9lz1wSX6tGWB3buP3klzZ\nbP8nk3wgyS1J/tcAGR+f5OPN3sqPJFnTt97fT3Il8Kohvb6SpI5Jsr3v8WuaMfKaJGfOm+9+Sd61\nc2xK8swkn2zG1fc14+wrgYcDH0vysQW2tTXJG/vG4h9v2nfZA5fkumbsXJvkpma7/9yMg09P8o/N\nOHlM3+of0+S5Jckv963rfyT5dJJrk7yxaVub5OYk76Z30+vDh/JiSkNggSft6tHAVYtM+zbwvKp6\nHL1C8KzmUzuARwJ/UlX/Cfg68CsLLH8A8KmqegzwD8AvLzAPwI8DzwKOAd6QZJ8Bcn+nqtYDbwMu\nBE5t+nJKkh9YLGOz7rOB51fV44FzgDP61nv/qlpfVWcNkEGStAdL8jPACcBPNWPd7/RN3hs4D7il\nqn4jyUOA3wCe3oyrVwL/var+EPgX4KlV9dRFNvWvzTJvBQY5rPLHgLPoja8/DvwC8NPNsq/tm+8n\ngKcBTwRen+ThSZ4JHElvTD4aeHySpzTzH0lvXH1UVX1xgBzSWHiIpjS4AP+7+cN+H3AovUM3Ab5c\nVf/YPH4P8Epg/qEr3wEubh5fBTxjke1cUlX3APckuatvG0u5qPm+Bbi+qm4HSPJ5ep8qfm2RjB+m\nVwhe2tSqewG39633LwfYtiRJAE8H/qyqvgVQVf/WN+3twAVVtfNDxCcARwH/2Iw/9wc+OeB2PtB8\nvwr4+QHm/0JVbQFIcj1wWVVVki3A2r75Lqyqfwf+vdl7eAy9QvCZwGebeR5Ir7D7EvDFqvrUgJml\nsbHAk3Z1PfD8RaadBDwUeHxVfTfJVmC/ZlrNm3f+c+gd+rmz/V4W//27p+/xzvl2sOse9/3Y1c5l\n7pu3/H1921koY+gVhE9cJMs3F2mXJGk5/gl4apKzqurb9MafS6vqhStY185xrn8sXWqcnD8u9o+Z\n/WPxYuPkb1XV2/snJFmLY6SmlIdoSrv6e2DfJBt2NiT5iST/F3AgcFdT3D0V+OG+5X4oyc4i6ReA\nTww5153Aw9K7itm+wHNWsI6FMt4MPHRne5J9kjxqKIklSXuaS4GX9J3jfUjftHcCHwIuSLI38Cng\nyUl+rJn3gHzvitXfAB60zG1vBR7XrOtxwBEryH9Ckv2aUxtmgU8DHwFemuSBzboPTfKwFaxbGhsL\nPKlPs4ftecDT07tNwvXAbwF30Dt3YH1zSMcvAjf1LXozcGqSG4GD6Z0XMMxc3wX+J3AFvQH0pqWX\nWND3Zayq79DbY/nbSa4BrgaeNJzUkqQ9SVV9mN4pA1cmuZp558dV1VvoHer458BXgVOA9ya5lt7h\nmT/ezLoZ+PBCF1lZwl8BhzTj9q+ygitfA9cCH6NXfL6pqv6lqv4O+Avgk834/36WX3xKY5XvHTEm\nSZIkSWoz9+BJkiRJUkdY4EmSJElSR1jgSZIkSVJHWOBJkiRJUkdY4EmSJElSR1jgSZIkSVJHWOBJ\nkiRJUkdY4EmSJElSR/z/jBOisTmCcygAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x8b762aaba8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# first we plot the distributions to find out if they are Gaussian or skewed.\n",
    "# Depending on the distribution, we will use the normal assumption or the interquantile\n",
    "# range to find outliers\n",
    "\n",
    "plt.figure(figsize=(15,6))\n",
    "plt.subplot(1, 2, 1)\n",
    "fig = data.Cabin_numerical.hist(bins=50)\n",
    "fig.set_ylabel('Number of passengers')\n",
    "fig.set_xlabel('Cabin number')\n",
    "\n",
    "plt.subplot(1, 2, 2)\n",
    "fig = data.Ticket_numerical.hist(bins=50)\n",
    "fig.set_ylabel('Number of passengers')\n",
    "fig.set_xlabel('Ticket number')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x8b767d8a90>"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA38AAAFpCAYAAAAsp5EgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuYZXV95/v3R26i3NHUIcDYHEVjQ7x2gIx5ZgowQIwj\nekYNJqNtQuzMI8dozCRALoOKTGBOlNE4knSEgMQIiHrgiIAtdMVEA9LgpblI6AgIHS5JNwJtFGn8\nnj/2r2RTVlXXpWvv6l7v1/PsZ6/1Xeu31rf62bj8rvVbv1+qCkmSJEnS9u1pw05AkiRJkrTwLP4k\nSZIkqQMs/iRJkiSpAyz+JEmSJKkDLP4kSZIkqQMs/iRJkiSpAyz+JEmSJKkDLP4kSZIkqQMs/iRJ\nkiSpA3YcdgLz8axnPauWLFky7DSkgfve977HM5/5zGGnIQ3UjTfe+K9V9exh57Gt8BqpLvL6qC6a\nzfVxmy7+lixZwpo1a4adhjRwY2NjjI6ODjsNaaCS3D3sHLYlXiPVRV4f1UWzuT7a7VOSJEmSOsDi\nT5IkSZI6wOJPkiRJkjrA4k+SJEmSOsDiT5IkSZI6wOJPkiRJkjrA4k+SJEmSOsDiT5IkSZI6wOJP\nkiRJkjrA4k+SJEmSOmDHYScgSZIkzUeSn4hV1RAykRY3n/xJkiRpm9Vf+B111FGTxiX1WPxJkiRp\nm1dV/PEf/7FP/KRpWPxJQ5Jkzp8jjzxyzm0lSdrevPnNb552XVKPxZ80JFU1589zTv7cnNtKkrS9\nufDCC6ddl9Rj8SdJkqRtXhJOP/10e7lI07D4kyRJ0jarv1fLtddeO2lcUo/FnyRJkrZp4682rF69\n2tccpGlY/EmSJElSB1j8SZLUJHl6kq8m+UaSW5K8t8X3SbIqyR3te+++NqcmWZfk9iTH9sVfnmRt\n2/bhtBeRkuyS5OIWvz7Jkr42y9s57kiyvC9+UNt3XWu78yD+PSRJ2xeLP0mSnvQYcFRVvRh4CXBc\nkiOAU4Brqupg4Jq2TpKlwAnAIcBxwEeT7NCOdQ7wNuDg9jmuxU8EHqqq5wFnA2e1Y+0DnAYcDhwG\nnNZXZJ4FnN3aPNSOIUnSrFj8SZLUVM+mtrpT+xRwPHBBi18AvLYtHw9cVFWPVdWdwDrgsCT7AXtU\n1XXVe/no4xPajB/rUuDo9lTwWGBVVW2sqoeAVfSKzwBHtX0nnl+SpBnbcdgJSJK0mLQndzcCzwP+\nd1Vdn2Skqu5ru9wPjLTl/YHr+prf22KPt+WJ8fE29wBU1eYkDwP79scntNkX+G5VbZ7kWBNzXwGs\nABgZGWFsbGzmf7i0Hdi0aZO/e2kaFn+SJPWpqieAlyTZC/hskkMnbK8ki3IowapaCawEWLZsWY2O\njg43IWnAxsbG8HcvTc1un5IkTaKqvguspveu3gOtKyft+8G223rgwL5mB7TY+rY8Mf6UNkl2BPYE\nNkxzrA3AXm3ficeSJGnGLP4kSWqSPLs98SPJrsAvAt8CLgfGR99cDlzWli8HTmgjeB5Eb2CXr7Yu\noo8kOaK9s/eWCW3Gj/V64Nr2XuDVwDFJ9m4DvRwDXN22rW77Tjy/JEkztmDFX5LzkjyY5OZJtv1u\nkkryrL7YpENlS5I0QPsBq5N8E7iB3gAsnwPOBH4xyR3AK9s6VXULcAlwK3AVcFLrNgrwduBj9AaB\n+SfgyhY/F9g3yTrg3bSRQ6tqI3B6O+8NwPtaDOBk4N2tzb7tGJIkzcpCvvN3PvAReiOc/ViSA+nd\nzfxOX6x/qOyfBr6Y5Pl9F1BJkhZcVX0TeOkk8Q3A0VO0OQM4Y5L4GuDQSeI/AN4wxbHOA86bJP5t\netM/SJI0Zwv25K+qvgRsnGTT2cDv0xs6e9ykQ2UvVG6SJEmS1DUDHe0zyfHA+qr6Ru8ViB+baqjs\nyY7hMNYS+NuXJEnSrAys+EvyDOAP6HX5nDOHsZaAq65wKGtJkiTNyiCf/D0XOAgYf+p3AHBTksOY\nenhrSZIkSdJWMLCpHqpqbVX9VFUtqaol9Lp2vqyq7meKobIHlZskSZIkbe8WcqqHTwL/ALwgyb1J\nTpxq3y0MlS1JkiRJmqcF6/ZZVW/awvYlE9YnHSpbkiRJkjR/A+v2KUmSJEkaHos/SZIkSeoAiz9J\nkiRJ6gCLP0mSJEnqAIs/SZIkSeoAiz9JkiRJ6gCLP0mSJEnqAIs/SZIkSeoAiz9JkiRJ6gCLP0mS\nJEnqAIs/SZIkSeoAiz9JkiRJ6gCLP0mSJEnqAIs/SZIkSeoAiz9JkiRJ6gCLP0mSJEnqAIs/SZIk\nSeoAiz9JkiRJ6gCLP0mSJEnqAIs/SZIkSeoAiz9JkiRJ6gCLP0mSJEnqAIs/SZIkSeoAiz9JkiRJ\n6gCLP0mSJEnqAIs/SZIkSeoAiz9JkiRJ6gCLP0mSJEnqAIs/SZIkSeoAiz9JkiRJ6gCLP0mSJEnq\ngAUr/pKcl+TBJDf3xf6fJN9K8s0kn02yV9+2U5OsS3J7kmMXKi9JkiRJ6qKFfPJ3PnDchNgq4NCq\nehHwj8CpAEmWAicAh7Q2H02ywwLmJknST0hyYJLVSW5NckuSd7b4e5KsT/L19nlVX5tJb14meXmS\ntW3bh5OkxXdJcnGLX59kSV+b5UnuaJ/lffGD2r7rWtudB/HvIUnavixY8VdVXwI2Toh9oao2t9Xr\ngAPa8vHARVX1WFXdCawDDluo3CRJmsJm4HerailwBHBSu0EJcHZVvaR9Pg9bvHl5DvA24OD2Gb8h\neiLwUFU9DzgbOKsdax/gNOBwetfA05Ls3dqc1c7/POChdgxJkmZlxyGe+zeAi9vy/vSKwXH3tthP\nSLICWAEwMjLC2NjYAqYoLV7+9qWtr6ruA+5ry48muY0prkfNj29eAncmWQccluQuYI+qug4gyceB\n1wJXtjbvae0vBT7SngoeC6yqqo2tzSrguCQXAUcBv9raXNDan7M1/mZJUncMpfhL8of07q5+YrZt\nq2olsBJg2bJlNTo6unWTk7YFV12Bv31pYbXumC8FrgdeAbwjyVuANfSeDj7E1DcvH2/LE+O073sA\nqmpzkoeBffvjE9rsC3y3r+eMN0ilKWzatMnfvTSNgRd/Sd4KvBo4uqqqhdcDB/btdkCLSZI0cEl2\nAz4NvKuqHklyDnA6UO37A/R6sCwq3iBV142NjXlzVJrGQKd6SHIc8PvAa6rq3/o2XQ6c0F6CP4je\nuxFfHWRukiQBJNmJXuH3iar6DEBVPVBVT1TVj4C/5Mn30qe6ebmeJ99r748/pU2SHYE9gQ3THGsD\nsFfbd+KxJEmasYWc6uGTwD8AL0hyb5ITgY8AuwOr2mhpfw5QVbcAlwC3AlcBJ1XVEwuVmyRJk2nv\n3p0L3FZVH+yL79e32+uA8WmMJr152d4dfCTJEe2YbwEu62szPpLn64FrW0+Yq4FjkuzdBno5Bri6\nbVvd9qW1HT+WJEkztmDdPqvqTZOEz51m/zOAMxYqH0mSZuAVwJuBtUm+3mJ/ALwpyUvodfu8C/gt\n6N28TDJ+83IzT715+XZ60x7tSm+glytb/FzgwjY4zEZ6o4VSVRuTnA7c0PZ73/jgL8DJwEVJ3g98\njWmup5IkTWWYo31KkrSoVNXfA5lk0+enaTPpzcuqWgMcOkn8B8AbpjjWecB5k8S/jVMgSZLmaaDv\n/EmSJEmShsPiT5IkSZI6wOJPkiRJkjrA4k+SJEmSOsDiT5IkSZI6wOJPkiRJkjrA4k+SJEmSOsDi\nT5IkSZI6wOJPkiRJkjrA4k+SJEmSOsDiT5IkSZI6wOJPkiRJkjrA4k+SJEmSOsDiT5IkSZI6wOJP\nkiRJkjrA4k+SJEmSOsDiT5IkSZI6wOJPkiRJkjrA4k+SJEmSOsDiT5IkSZI6wOJPkiRJkjrA4k+S\nJEmSOsDiT5IkSZI6wOJPkiRJkjrA4k+SJEmSOsDiT5IkSZI6wOJPkiRJkjrA4k+SJEmSOsDiT5Ik\nSZI6wOJPkiRJkjrA4k+SJEmSOmDBir8k5yV5MMnNfbF9kqxKckf73rtv26lJ1iW5PcmxC5WXJEmS\nJHXRQj75Ox84bkLsFOCaqjoYuKatk2QpcAJwSGvz0SQ7LGBukiRJktQpC1b8VdWXgI0TwscDF7Tl\nC4DX9sUvqqrHqupOYB1w2ELlJkmSJElds+OAzzdSVfe15fuBkba8P3Bd3373tthPSLICWAEwMjLC\n2NjYwmQqLXL+9iVJkjQbgy7+fqyqKknNod1KYCXAsmXLanR0dGunJi1+V12Bv31JkiTNxqBH+3wg\nyX4A7fvBFl8PHNi33wEtJkmSJEnaCgZd/F0OLG/Ly4HL+uInJNklyUHAwcBXB5ybJEmSJG23Fqzb\nZ5JPAqPAs5LcC5wGnAlckuRE4G7gjQBVdUuSS4Bbgc3ASVX1xELlJkmSJElds5Cjfb6pqvarqp2q\n6oCqOreqNlTV0VV1cFW9sqo29u1/RlU9t6peUFVXLlRekiRNJcmBSVYnuTXJLUne2eKznqc2ycuT\nrG3bPpwkLb5Lkotb/PokS/raLG/nuCPJ8r74QW3fda3tzoP495AkbV8G3e1TkqTFbDPwu1W1FDgC\nOKnNRTuXeWrPAd5G71WGg3ly7tsTgYeq6nnA2cBZ7Vj70Oslczi96Y5O6ysyzwLObm0easeQJGlW\nLP4kSWqq6r6quqktPwrcRm/qoVnNU9sGNdujqq6rqgI+PqHN+LEuBY5uTwWPBVZV1caqeghYBRzX\nth3V9p14fkmSZmxoUz1IkrSYte6YLwWuZ/bz1D7elifGx9vcA1BVm5M8DOzbH5/QZl/gu1W1eZJj\nTczZuXDVaZs2bfJ3L03D4k+SpAmS7AZ8GnhXVT3SXtcD5j5P7SA4F666bmxszHlwpWnY7VOSpD5J\ndqJX+H2iqj7TwrOdp3Z9W54Yf0qbJDsCewIbpjnWBmCvtu/EY0mSNGMWf5IkNe39unOB26rqg32b\nZjVPbesi+kiSI9ox3zKhzfixXg9c294LvBo4JsnebaCXY4Cr27bVbd+J55ckacbs9ilJ0pNeAbwZ\nWJvk6y32B8xtntq3A+cDuwJXtg/0issLk6wDNtIbLZSq2pjkdOCGtt/7+qZEOhm4KMn7ga+1Y0iS\nNCsWf5IkNVX190Cm2Hz0FG3OAM6YJL4GOHSS+A+AN0xxrPOA8yaJf5ve9A+SJM2Z3T4lSZIkqQMs\n/iRJkiSpAyz+JEnblfQcuOU9JUnqlmmLvyQ7JPmdQSUjSdJ8tdExPz/sPCRJWmymLf7aiGVvGlAu\nkiRtLTcl+blhJyFJ0mIyk9E+v5zkI8DFwPfGg1V104JlJUnS/BwO/FqSu+ldu0LvoeCLhpuWJEnD\nM5Pi7yXt+319sQKO2vrpSJK0VRw77AQkSVpstlj8VdWRg0hEkqStparuTvILwMFV9VdJng3sNuy8\nJEkapi2O9plkJMm5Sa5s60uTnLjwqUmSNDdJTgNOBk5toZ2Avx5eRpIkDd9Mpno4H7ga+Om2/o/A\nuxYqIUmStoLXAa+hvateVf8M7D7UjCRJGrKZFH/PqqpLgB8BVNVm4IkFzUqSpPn5YZvyoQCSPHPI\n+UiSNHQzKf6+l2RfnryAHgE8vKBZSZI0P5ck+QtgryRvA74I/OWQc5IkaahmMtrnu4HLgecm+TLw\nbOD1C5qVJEnzUFV/muQXgUeA5wP/vapWDTktSZKGaiajfd6U5D8CL6A3T9LtVfX4gmcmSdL8rAV2\npddzZe2Qc5EkaehmMtrn04HfBk4H3guc1GKSJC1KSX4T+Crwf9HrrXJdkt8YblaSJA3XTLp9fhx4\nFPiztv6rwIXAGxYqKUmS5un3gJdW1QaA9u76V4DzhpqVJElDNJPi79CqWtq3vjrJrQuVkCRJW8EG\nejcuxz3aYpIkddZMir+bkhxRVdcBJDkcWLOwaUmSNHtJ3t0W1wHXJ7mM3jt/xwPfHFpikiQtAlMW\nf0nW0rtg7gR8Jcl32qZ/B3xrALlJkjRb4xO5/1P7jLtsCLlIkrSoTPfk79UDy0KSpK2gqt477Bwk\nSVqspiz+quru8eUkewMHTtj/7p9oJEnSIpBkGfCHwHPou3ZV1YuGlpQkSUO2xXf+kpwOvJVe95lq\n4QKOWri0JEmal0/QG/FzLfCjIeciSdKiMJMBX94IPLeqfrjQyUiStJX8S1VdPuwkJElaTGZS/N0M\n7AU8uMC5SJK0tZyW5GPANcBj48Gq+szwUpIkabhmUvz9CfC1JDfz1Avoa+Z60iS/A/wmve6ja4Ff\nB54BXAwsAe4C3lhVD831HJKkTvt14GfojVg93u2zAIs/SVJnzaT4uwA4i6303kSS/YHfBpZW1feT\nXAKcACwFrqmqM5OcApwCnDzf80mSOunnquoFw05CkqTFZCbF379V1YcX4Ly7Jnmc3hO/fwZOBUbb\n9guAMSz+JElz85UkS6vq1mEnIknSYjGT4u/vkvwJcDlP7fZ501xOWFXrk/wp8B3g+8AXquoLSUaq\n6r622/3AyGTtk6wAVgCMjIwwNjY2lzSkbZ6/fWlaRwBfT3InvWtXgHKqB0lSl82k+Htp+z6iLzbn\nqR7anIHHAwcB3wU+leS/9O9TVZWkJmtfVSuBlQDLli2r0dHRuaQhbduuugJ/+9K0jht2ApIkLTZb\nLP6q6sitfM5XAndW1b8AJPkM8O+BB5LsV1X3JdkPRxeVJM3dpDcQJUnqsplM8v7fJ4tX1fvmeM7v\nAEckeQa9bp9HA2uA7wHLgTPb92VzPL4kSVfQKwADPJ1eb5PbgUOGmZQkScM0k26f3+tbfjrwauC2\nuZ6wqq5PcilwE7AZ+Bq9bpy7AZckORG4m97k8pIkzVpV/Wz/epKXAW8fUjqSJC0KM+n2+YH+9TZY\ny9XzOWlVnQacNiH8GL2ngNI25cXv/QIPf//xgZ93ySlXDOxce+66E9847ZiBnU/a2qrqpiSHDzsP\nSZKGaSZP/iZ6BnDA1k5E2lY9/P3HuevMXx7oOcfGxgY64MsgC01pa0jy7r7VpwEvozetkCRJnTWT\nd/7W8uSL8zsAzwbm+r6fJEmDsHvf8mZ67wB+eki5SJK0KMzkyd+r+5Y3Aw9U1eYFykeSpHmrqvcO\nOwdJkhabmbzzd3eSHehNur4j8NNJqKrvLHh2kiTNQZLnA/8NWELfta6q5jRHrSRJ24OZdPt8B73B\nWR4AftTCBbxoAfOSJGk+PgX8OfAx4Ikh5yJJ0qIwk26f7wReUFUbFjoZSZK2ks1Vdc6wk5AkaTF5\n2gz2uQd4eKETkSRpK/r/krw9yX5J9hn/bKlRkvOSPJjk5r7Ye5KsT/L19nlV37ZTk6xLcnuSY/vi\nL0+ytm37cJK0+C5JLm7x65Ms6WuzPMkd7bO8L35Q23dda7vz/P95JEldNJMnf98GxpJcQW8uPgCq\n6oMLlpUkSfMzXjz9Xl+sgP9zC+3OBz4CfHxC/Oyq+tP+QJKlwAnAIcBPA19M8vyqegI4B3gbcD3w\neeA44ErgROChqnpekhOAs4BfaYXpacCylueNSS6vqofaPmdX1UVJ/rwdw6eakqRZm8mTv+8Aq4Cd\n6Q2dPf6RJGlRqqqDJvlsqfCjqr4EbJzhaY4HLqqqx6rqTmAdcFiS/YA9quq6qip6heRr+9pc0JYv\nBY5uTwWPBVZV1cZW8K0Cjmvbjmr70tqOH0uSpFmZyWifDpctSeq6dyR5C7AG+N1WoO0PXNe3z70t\n9nhbnhinfd8DUFWbkzwM7Nsfn9BmX+C7fVMs9R/rJyRZAawAGBkZYWxsbNZ/qLQt27Rpk797aRoz\n6fYpSVKXnQOcTq875unAB4DfGGpGU6iqlcBKgGXLltXo6OhwE5IGbGxsDH/30tRm0u1TkqTOqqoH\nquqJqvoR8JfAYW3TeuDAvl0PaLH1bXli/CltkuwI7AlsmOZYG4C92r4TjyVJ0qxY/EmStjtJrplJ\nbIbH2q9v9XXA+EiglwMntBE8DwIOBr5aVfcBjyQ5or2z9xbgsr4244PRvB64tr0XeDVwTJK9k+wN\nHANc3batbvvS2o4fS5KkWZnJJO/Ppjdi2ZL+/atqUXZ5kSR1V5KnA88AntWKqLRNezDNu3J97T8J\njLb299IbgXM0yUvodfu8C/gtgKq6JcklwK3AZuCkNtInwNvpjRy6K71RPq9s8XOBC5OsozewzAnt\nWBuTnA7c0PZ7X1WNDzxzMnBRkvcDX2vHkCRp1mbyzt9lwN8BXwSe2MK+kiQN028B76I39cJNffFH\n6E3hMK2qetMk4SmLrao6Azhjkvga4NBJ4j8A3jDFsc4Dzpsk/m2e7GoqSdKczaT4e0ZVnbzgmUiS\nNE9V9SHgQ0neUVV/Nux8JElaTGbyzt/nkrxqwTORJGnrOS/JHyVZCZDk4CSvHnZSkiQN00yKv3fS\nKwC/n+SRJI8meWShE5MkaR7OA34I/Pu2vh54//DSkSRp+GYyyfvug0hEkqSt6LlV9StJ3gRQVf/W\nRt6UJKmzpiz+kvxMVX0rycsm215VN00WlyRpEfhhkl3pjdBJkucCjw03JUmShmu6J3/vBlYAH5hk\nWwFHLUhGkiTN32nAVcCBST4BvAJ461AzkiRpyKYs/qpqRfs+cnDpSJI0f1W1KslNwBH05vp7Z1X9\n65DTkiRpqLY44EuSpyd5d5LPJPl0kne1SXQlSVqUkryvqjZU1RVV9TlgY3sCKElSZ81ktM+PA4cA\nf0ZvgtxDgAsXMilJkubpwCSnAiTZBfgscMdwU5IkabhmMsn7oVW1tG99dZJbFyohSZK2gt8APtEK\nwCOBK6vq7CHnJEnSUM3kyd9NSY4YX0lyOLBm4VKSJGlukrysjVL9UuBDwK/Qe+L3t1ONXi1JUldM\nN9XDWnqjeu4EfCXJd9r6c4BvDSY9SZJmZeII1Q8BS1vckaolSZ02XbfPVw8sC0mStgJHqJYkaWpT\ndvusqrv7P8D36d01Hf9IkrQoJfkfSfbqW987yfuHmZMkScM2k6keXpPkDuBO4G+Bu4ArFzgvSZLm\n45eq6rvjK1X1EPCqIeYjSdLQzWTAl9PpTZL7j1V1EHA0cN2CZiVJ0vzs0KZ4ACDJrsAu0+wvSdJ2\nbybF3+NVtQF4WpKnVdVqYNl8TppkrySXJvlWktuS/HySfZKsSnJH+957PueQJHXaJ4BrkpyY5ERg\nFXDBkHOSJGmoZlL8fTfJbsCX6M2Z9CHge/M874eAq6rqZ4AXA7cBpwDXVNXBwDVtXZKkWauqs4Az\ngBe2z+lV9T+Hm5UkScM1k0nej6c32MvvAL8G7Am8b64nTLIn8B+AtwJU1Q+BHyY5Hhhtu10AjAEn\nz/U8kqRuq6or8R11SZJ+bLp5/p4HjFTVl1voR8AFSX4B2AvYMMdzHgT8C/BXSV4M3Ai8s53rvrbP\n/cDIFHmtAFYAjIyMMDY2Nsc0pK1n0L/DTZs2Dfyc/rembUGSv6+qX0jyKE8dmTpAVdUeQ0pNkqSh\nm+7J3/8CTp0k/nDb9p/mcc6XAe+oqutbN9KndPGsqkoy6XQSVbUSWAmwbNmyGh0dnWMa0lZy1RUM\n+nc4NjY22HMO4W+U5uiZAFW1+7ATkSRpsZnunb+Rqlo7MdhiS+ZxznuBe6vq+rZ+Kb1i8IEk+wG0\n7wfncQ5JUjc5D60kSVOY7snfXtNs23WuJ6yq+5Pck+QFVXU7vakjbm2f5cCZ7fuyuZ5DktRZP5Xk\n3VNtrKoPDjIZSZIWk+mKvzVJ3lZVf9kfTPKb9N7Tm4930Bs5dGfg28Cv03sKeUkbkvtu4I3zPIck\nqXt2AHaj946fJEnqM13x9y7gs0l+jSeLvWXAzsDr5nPSqvo6k88VePR8jitJ6rz7qmrOI1JLkrQ9\nm7L4q6oHgH+f5Ejg0Ba+oqquHUhmkiTNnk/8JEmawhbn+auq1cDqAeQiSdJ82YNEkqQpTDfapyRJ\n25Sq2jjsHCRJWqws/iRJkiSpAyz+JEmSJKkDLP4kSZIkqQMs/iRJkiSpAyz+JEmSJKkDLP4kSZIk\nqQMs/iRJkiSpAyz+JEmSJKkDLP4kSZIkqQMs/iRJkiSpAyz+JElqkpyX5MEkN/fF9kmyKskd7Xvv\nvm2nJlmX5PYkx/bFX55kbdv24SRp8V2SXNzi1ydZ0tdmeTvHHUmW98UPavuua213Xuh/B0nS9sni\nT5KkJ50PHDchdgpwTVUdDFzT1kmyFDgBOKS1+WiSHVqbc4C3AQe3z/gxTwQeqqrnAWcDZ7Vj7QOc\nBhwOHAac1ldkngWc3do81I4hSdKsWfxJktRU1ZeAjRPCxwMXtOULgNf2xS+qqseq6k5gHXBYkv2A\nParquqoq4OMT2owf61Lg6PZU8FhgVVVtrKqHgFXAcW3bUW3fieeXJGlWdhx2ApIkLXIjVXVfW74f\nGGnL+wPX9e13b4s93pYnxsfb3ANQVZuTPAzs2x+f0GZf4LtVtXmSY/2EJCuAFQAjIyOMjY3N+I+U\ntgebNm3ydy9Nw+JPkqQZqqpKUsPOYypVtRJYCbBs2bIaHR0dbkLSgI2NjeHvXpqa3T4lSZreA60r\nJ+37wRZfDxzYt98BLba+LU+MP6VNkh2BPYEN0xxrA7BX23fisSRJmhWLP0mSpnc5MD765nLgsr74\nCW0Ez4PoDezy1dZF9JEkR7R39t4yoc34sV4PXNveC7waOCbJ3m2gl2OAq9u21W3fieeXJGlW7PYp\nSVKT5JPAKPCsJPfSG4HzTOCSJCcCdwNvBKiqW5JcAtwKbAZOqqon2qHeTm/k0F2BK9sH4FzgwiTr\n6A0sc0I71sYkpwM3tP3eV1XjA8+cDFyU5P3A19oxJEmaNYs/SZKaqnrTFJuOnmL/M4AzJomvAQ6d\nJP4D4A1THOs84LxJ4t+mN/2DJEnzYrdPSZIkSeoAiz9JkiRJ6gCLP0mSJEnqAIs/SZIkSeoAiz9J\nkiRJ6gCLP0mSJEnqAIs/SZIkSeoA5/mT5mn3F57Cz15wyuBPfMHgTrX7CwF+eXAnlCRJ0lZn8SfN\n06O3ncldZw62MBobG2N0dHRg51tyyhUDO5ckSZIWht0+JUmSJKkDhlb8JdkhydeSfK6t75NkVZI7\n2vfew8pEcjvZAAAPF0lEQVRNkiRJkrY3w3zy907gtr71U4Brqupg4Jq2LkmSJEnaCoZS/CU5gN7o\nER/rCx/Pk0NYXAC8dtB5SZIkSdL2algDvvwv4PeB3ftiI1V1X1u+HxiZrGGSFcAKgJGREcbGxhYw\nTWlmBv073LRp08DP6X9rkiRJ27aBF39JXg08WFU3JhmdbJ+qqiQ1xbaVwEqAZcuW1SBHPJQmddUV\nAx15EwY/2ucw/kZJkiRtXcN48vcK4DVJXgU8HdgjyV8DDyTZr6ruS7If8OAQcpMkSZKk7dLA3/mr\nqlOr6oCqWgKcAFxbVf8FuBxY3nZbDlw26NwkSZIkaXu1mOb5OxP4xSR3AK9s65IkSZKkrWBYA74A\nUFVjwFhb3gAcPcx8JEmSJGl7tZie/EmSJEmSFojFnyRJkiR1gMWfJEmSJHWAxZ8kSZIkdYDFnyRJ\nkiR1gMWfJEmSJHWAxZ8kSZIkdYDFnyRJkiR1gMWfJEmSJHWAxZ8kSZIkdYDFnyRJkiR1gMWfJEmS\nJHWAxZ8kSZIkdYDFnyRJkiR1gMWfJEmSJHWAxZ8kSZIkdYDFnyRJkiR1gMWfJEmSJHWAxZ8kSZIk\ndYDFnyRJkiR1gMWfJEkzkOSuJGuTfD3JmhbbJ8mqJHe077379j81yboktyc5ti/+8nacdUk+nCQt\nvkuSi1v8+iRL+tosb+e4I8nywf3VkqTticWfJEkzd2RVvaSqlrX1U4Brqupg4Jq2TpKlwAnAIcBx\nwEeT7NDanAO8DTi4fY5r8ROBh6rqecDZwFntWPsApwGHA4cBp/UXmZIkzZTFnyRJc3c8cEFbvgB4\nbV/8oqp6rKruBNYBhyXZD9ijqq6rqgI+PqHN+LEuBY5uTwWPBVZV1caqeghYxZMFoyRJM7bjsBOQ\nJGkbUcAXkzwB/EVVrQRGquq+tv1+YKQt7w9c19f23hZ7vC1PjI+3uQegqjYneRjYtz8+SZunSLIC\nWAEwMjLC2NjY7P9KaRu2adMmf/fSNCz+JEmamV+oqvVJfgpYleRb/RurqpLUkHIbz2ElsBJg2bJl\nNTo6Osx0pIEbGxvD3700Nbt9SpI0A1W1vn0/CHyW3vt3D7SunLTvB9vu64ED+5of0GLr2/LE+FPa\nJNkR2BPYMM2xJEmaFYs/SZK2IMkzk+w+vgwcA9wMXA6Mj765HLisLV8OnNBG8DyI3sAuX21dRB9J\nckR7n+8tE9qMH+v1wLXtvcCrgWOS7N0GejmmxSRJmhW7fUqStGUjwGfbrAw7An9TVVcluQG4JMmJ\nwN3AGwGq6pYklwC3ApuBk6rqiXastwPnA7sCV7YPwLnAhUnWARvpjRZKVW1McjpwQ9vvfVW1cSH/\nWEnS9sniT5KkLaiqbwMvniS+ATh6ijZnAGdMEl8DHDpJ/AfAG6Y41nnAebPLWpKkp7LbpyRJkiR1\ngE/+pK1gySlXDP6kVw3unHvuutPAziVJkqSFMfDiL8mB9Ca1HaE3Z9LKqvpQkn2Ai4ElwF3AG9tk\nttKidteZvzzwcy455YqhnFeSJEnbrmF0+9wM/G5VLQWOAE5KshQ4Bbimqg4GrmnrkiRJkqStYOBP\n/tow1/e15UeT3AbsDxwPjLbdLgDGgJMHnZ8kSZKGq42sO1C9mVWk7dtQB3xJsgR4KXA9MNIKQ4D7\n6XULlSRJUsdU1Zw+zzn5c3NuK3XB0AZ8SbIb8GngXVX1SP8dnqqqJJP+V5hkBbACYGRkhLGxsQFk\nKy0+/vYlSZI0G0Mp/pLsRK/w+0RVfaaFH0iyX1Xdl2Q/4MHJ2lbVSmAlwLJly2p0dHQQKUuLy1VX\n4G9fkiRJszHwbp/pPeI7F7itqj7Yt+lyYHlbXg5cNujcJEmSJGl7NYwnf68A3gysTfL1FvsD4Ezg\nkiQnAncDbxxCbpIkSdpKXvzeL/Dw9x8f6DkHPffunrvuxDdOO2ag55Tmahijff49MNUQTkcPMhdJ\nkiQtnIe///hA56UdGxsb+GsRgy42pfkY6mifkiRJkqTBsPiTJEmSpA6w+JMkSZKkDrD4kyRJkqQO\nsPiTJEmSpA6w+JMkSZKkDrD4kyRJkqQOsPiTJEmSpA6w+JMkSZKkDrD4kyRJkqQOsPiTJEmSpA6w\n+JMkSZKkDrD4kyRJkqQOsPiTJEmSpA6w+JMkSZKkDrD4kyRJkqQOsPiTJEmSpA6w+JMkSZKkDrD4\nkyRJkqQOsPiTJEmSpA7YcdgJSJIkafu0+wtP4WcvOGWwJ71gsKfb/YUAvzzYk0pzZPEnSZKkBfHo\nbWdy15mDK4zGxsYYHR0d2PkAlpxyxUDPJ82H3T4lSZIkqQMs/iRJkiSpAyz+JEmSJKkDLP4kSZIk\nqQMs/iRJkiSpAyz+JEmSJKkDLP4kSdoGJDkuye1J1iUZ8MRpkqTtgcWfJEmLXJIdgP8N/BKwFHhT\nkqXDzUqStK1xkndJkha/w4B1VfVtgCQXAccDtw41K2kGBj4J+lWDPd+eu+400PNJ82HxJw1Jkvm1\nP2tu7apqXueVNBT7A/f0rd8LHD5xpyQrgBUAIyMjjI2NDSQ5aSrnH/fMObU78sgjt3ImW7Z69eo5\nt/W/NW0rFl3xl+Q44EPADsDHqurMIackLYj5FGFjY2OMjo5uvWQkbReqaiWwEmDZsmXl/05oWzXX\na6TXR2l6i+qdP99pkCRpUuuBA/vWD2gxSZJmbFEVf/S901BVPwTG32mQJKnLbgAOTnJQkp2BE4DL\nh5yTJGkbs9i6fW7xnQbfZ5Bg06ZN/valDqmqzUn+b+Bqeq9FnFdVtww5LUnSNmaxFX9b5PsMku80\nSF1UVZ8HPj/sPCRJ267F1u3TdxokSZIkaQEstuLPdxokSZIkaQEsqm6fvtMgSZIkSQtjURV/4DsN\nkiRJkrQQFlu3T0mSJEnSArD4kyRJkqQOsPiTJEmSpA6w+JMkSZKkDrD4kyRJkqQOSFUNO4c5S/Iv\nwN3DzkMagmcB/zrsJKQBe05VPXvYSWwrvEaqo7w+qotmfH3cpos/qauSrKmqZcPOQ5KkxcTrozQ9\nu31KkiRJUgdY/EmSJElSB1j8SdumlcNOQJKkRcjrozQN3/mTJEmSpA7wyZ8kSZIkdYDFnyRJkiR1\ngMWfJEmSJHWAxZ80QZL/I8lFSf4pyY1JPp/k+VPsuyTJzVNs+1iSpQub7cJof/Nec2j31iQfWYic\nJEnDkWTfJF9vn/uTrO9b/8oW2o4lmfG8e0neleQZ88964SRZluTDc2w7q38PaWvbcdgJSItJkgCf\nBS6oqhNa7MXACPCPszlWVf3m1s9wYbW/P1X1qmHnIklaHKpqA/ASgCTvATZV1Z8u0OneBfw18G8L\ndPx5SbJjVa0B1gw7F2kufPInPdWRwONV9efjgar6BvC1JNckuSnJ2iTH97XZMcknktyW5NLxO5b9\nd/eSbEpyRpJvJLkuychUCSQ5P8mHk3wlybeTvL7FR5N8rm+/jyR5a1u+K8mftLuwa5K8LMnV7enl\nf+1r83tJbkjyzSTvbbElSW5P8nHgZuDAdrxnte1vaft/I8mFLfafklyf5GtJvjjd3yNJ2n4l2dS3\nfHK7Rn4jyZkT9ntau769v60fk+Qf2nX1U0l2S/LbwE8Dq5Osnu6ck11T2/FfPzG3dv382ySXtevq\nmUl+LclXW77Pbfs9O8mn23XyhiSvaPH3JLkwyZeBC/uvxy3vv2rH+WaS/9zi57Tr8S3j11tpMbD4\nk57qUODGSeI/AF5XVS+jVyB+oD0lA3gB8NGqeiHwCPD2Sdo/E7iuql4MfAl42xby2A/4BeDVwJlb\n2Hfcd6rqJcDfAecDrweOAMaLvGOAg4HD6N3BfXmS/9DaHtz+hkOq6u7xAyY5BPgj4KiW+zvbpr8H\njqiqlwIXAb8/wxwlSduhJL8EHA8c3q4X/7Nv847AJ4A7quqP2s3FPwJe2a6ra4B3V9WHgX8Gjqyq\nI6c53WyvqQAvBv4r8ELgzcDzq+ow4GPAO9o+HwLOrqqfA/5z2zZuacv3TROO+8fAw1X1s1X1IuDa\nFv/DqloGvAj4j0leNIMcpQVnt09pZgL8j1Ys/QjYn15XUIB7qurLbfmvgd8GJnaH+SEw/tTuRuAX\nt3C+/7eqfgTcOounape377XAblX1KPBoksfSe3/vmPb5WttvN3pF33eAu6vqukmOeRTwqar6V4Cq\n2tjiBwAXJ9kP2Bm4c4Y5SpK2T68E/qqq/g2ecr0A+Avgkqo6o60fQa+Y+nK7j7oz8A+zONdsr6kA\nN1TVfQBJ/gn4QouvpXdTd/xvWPrkvV32SLJbW768qr4/yXFfCZwwvlJVD7XFNyZZQe//a+9H7+/9\n5gzylBaUxZ/0VLfQe2I20a8BzwZeXlWPJ7kLeHrbVhP2nbgOva6k4/En2PJ/e4/1LY9fhTbz1Kf1\nT+epxtv8aEL7H7XzBfiTqvqL/kZJlgDf20I+E/0Z8MGqujzJKPCeWbaXJHXHV4Ajk3ygqn5A73q0\napKnaDM11TX1x9fJJE+jV1SOm3hd7L9mjrd/Gr1eLT/oP1krBmd8nUxyEPDfgJ+rqoeSnM9PXrOl\nobDbp/RU1wK7tLt1ALSuGs8BHmyF35Ftfdy/S/LzbflX6XWJXAh307sjuUt7knf0LNtfDfzG+F3M\nJPsn+akttLkWeEOSfVubfVp8T2B9W14+yzwkSdufVcCv58n33vfp23Yu8HngkiQ7AtcBr0jyvLbv\nM/PkqNqPArvPMYe7gJe35dcAO82y/Rd4sgsoSV4ygzargJP62uwN7EGvWHy49d75pVnmIS0Yiz+p\nT7uT+DrglW2wlFuAP6F30VqWZC3wFuBbfc1uB05KchuwN3DOAuV2D3AJvUFZLuHJ7pszbf8F4G+A\nf2h/x6Vs4QJbVbcAZwB/m+QbwAfbpvcAn0pyI/Cvs8lDkrT9qaqr6L1+sCbJ1+k9+erf/kF6160L\ngQ3AW4FPJvkmvS6fP9N2XQlcNd2AL9P4S3rv130D+Hlm36vlt+ld67+Z5FZ67whuyfuBvZPc3M57\n5PhAcfT+v8LfAF+e7gDSIOXJp+aSJEmSpO2VT/4kSZIkqQMc8EUakiR/CLxhQvhTfaOhSZLUWUmu\nB3aZEH5zVa0dRj7S9sBun5IkSZLUAXb7lCRJkqQOsPiTJEmSpA6w+JMkSZKkDrD4kyRJkqQO+P8B\nVvheh/uPW4gAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x8b762aacc0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# let's visualise outliers with the boxplot and whiskers\n",
    "plt.figure(figsize=(15,6))\n",
    "plt.subplot(1, 2, 1)\n",
    "fig = data.boxplot(column='Cabin_numerical')\n",
    "fig.set_title('')\n",
    "fig.set_ylabel('Cabin number')\n",
    "\n",
    "plt.subplot(1, 2, 2)\n",
    "fig = data.boxplot(column='Ticket_numerical')\n",
    "fig.set_title('')\n",
    "fig.set_ylabel('Ticket number')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cabin_numerical does not contain outliers. Ticket_numerical seems to contain a few outliers. Let's find out more about it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ticket number outliers are values < -981730.0 or > 1343691.0\n",
      "Number of passengers with ticket values higher than 1343691.0: 0.04938271604938271\n"
     ]
    }
   ],
   "source": [
    "# Ticket numerical\n",
    "IQR = data.Ticket_numerical.quantile(0.75) - data.Ticket_numerical.quantile(\n",
    "    0.25)\n",
    "Lower_fence = data.Ticket_numerical.quantile(0.25) - (IQR * 3)\n",
    "Upper_fence = data.Ticket_numerical.quantile(0.75) + (IQR * 3)\n",
    "print(\n",
    "    'Ticket number outliers are values < {lowerboundary} or > {upperboundary}'.\n",
    "    format(lowerboundary=Lower_fence, upperboundary=Upper_fence))\n",
    "passengers = len(data[data.Ticket_numerical > Upper_fence]) / np.float(\n",
    "    len(data))\n",
    "print('Number of passengers with ticket values higher than {upperboundary}: {passengers}'.format(upperboundary=Upper_fence, \\\n",
    "                                                                                                 passengers=passengers))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I will use equal width discretisation for this variable."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### New categorical variables: Missing values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Cabin_categorical     0.771044\n",
       "Ticket_categorical    0.000000\n",
       "Title                 0.000000\n",
       "dtype: float64"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[['Cabin_categorical', 'Ticket_categorical', 'Title']].isnull().mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected, Cabin contains the same amount of missing data as the original Cabin variable.\n",
    "\n",
    "The other 2 variables do not show missing data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### New categorical variables: cardinality"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cabin_categorical  contains  9  labels\n",
      "Ticket_categorical  contains  28  labels\n",
      "Title  contains  5  labels\n"
     ]
    }
   ],
   "source": [
    "for var in ['Cabin_categorical', 'Ticket_categorical', 'Title']:\n",
    "    print(var, ' contains ', len(data[var].unique()), ' labels')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Title and Cabin are not highly cardinal, Ticket on the other hand has quite a few labels. Let's explore the percentage of passengers within each label to identify rare labels.\n",
    "\n",
    "#### New categorical variables: rare labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C    0.066218\n",
      "B    0.052750\n",
      "D    0.037037\n",
      "E    0.035915\n",
      "A    0.016835\n",
      "F    0.014590\n",
      "G    0.004489\n",
      "T    0.001122\n",
      "Name: Cabin_categorical, dtype: float64\n",
      "\n",
      "NAN        0.741863\n",
      "PC         0.067340\n",
      "CA         0.046016\n",
      "A          0.031425\n",
      "STONO      0.020202\n",
      "SOTONOQ    0.016835\n",
      "SCPARIS    0.012346\n",
      "WC         0.011223\n",
      "SOC        0.006734\n",
      "C          0.005612\n",
      "FCC        0.005612\n",
      "LINE       0.004489\n",
      "SCAH       0.003367\n",
      "WEP        0.003367\n",
      "SOPP       0.003367\n",
      "PP         0.003367\n",
      "PPP        0.002245\n",
      "SWPP       0.002245\n",
      "SOTONO     0.002245\n",
      "SC         0.001122\n",
      "AS         0.001122\n",
      "CASOTON    0.001122\n",
      "SCOW       0.001122\n",
      "FA         0.001122\n",
      "SCA        0.001122\n",
      "SOP        0.001122\n",
      "SP         0.001122\n",
      "FC         0.001122\n",
      "Name: Ticket_categorical, dtype: float64\n",
      "\n",
      "Mr        0.581369\n",
      "Miss      0.202020\n",
      "Mrs       0.144781\n",
      "Master    0.044893\n",
      "Other     0.026936\n",
      "Name: Title, dtype: float64\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# rare / infrequent labels (less than 1% of passengers)\n",
    "for var in ['Cabin_categorical', 'Ticket_categorical', 'Title']:\n",
    "    print(data[var].value_counts() / np.float(len(data)))\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Cabin contains the rare labels G and T: replace by most frequent category\n",
    "- Ticket contains a lot of infrequent labels: replace by rare\n",
    "- Title does not contain rare labels\n",
    "\n",
    "Because the number of passengers in the rare cabins is so small, grouping them into a new category called rare, will be in itself rare, and may be prone to over-fitting. This, in cabin, I will replace rare labels by the most frequent category.\n",
    "\n",
    "In ticket_categorical, on the other hand, the number of infrequent labels is high, therefore grouping them into a new label makes sense. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Separate train and test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((712, 16), (179, 16))"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's separate into train and test set\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(data, data.Survived, test_size=0.2,\n",
    "                                                    random_state=0)\n",
    "X_train.shape, X_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# let's group again the variables into categorical or numerical\n",
    "# now considering the newly created variables\n",
    "\n",
    "def find_categorical_and_numerical_variables(dataframe):\n",
    "    cat_vars = [col for col in data.columns if data[col].dtypes == 'O']\n",
    "    num_vars  = [col for col in data.columns if data[col].dtypes != 'O']\n",
    "    return cat_vars, num_vars\n",
    "                 \n",
    "categorical, numerical = find_categorical_and_numerical_variables(data)       "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Sex', 'Embarked', 'Cabin_categorical', 'Ticket_categorical', 'Title']"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "categorical"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Pclass',\n",
       " 'Age',\n",
       " 'SibSp',\n",
       " 'Parch',\n",
       " 'Fare',\n",
       " 'Cabin_numerical',\n",
       " 'Ticket_numerical',\n",
       " 'Family_size',\n",
       " 'is_mother']"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numerical = [var for var in numerical if var not in ['Survived','PassengerId']]\n",
    "numerical"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Engineering missing values in numerical variables (section 5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Age 0.198033707865\n",
      "Cabin_numerical 0.776685393258\n",
      "Ticket_numerical 0.00561797752809\n"
     ]
    }
   ],
   "source": [
    "# print variables with missing data\n",
    "for col in numerical:\n",
    "    if X_train[col].isnull().mean()>0:\n",
    "        print(col, X_train[col].isnull().mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Age and ticket contains < 50% NA: create additional variable with NA + random sample imputation\n",
    "- Cabin_numerical contains > 50% NA: impute NA by value far in the distribution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def impute_na(X_train, df, variable):\n",
    "    # make temporary df copy\n",
    "    temp = df.copy()\n",
    "    \n",
    "    # extract random from train set to fill the na\n",
    "    random_sample = X_train[variable].dropna().sample(temp[variable].isnull().sum(), random_state=0)\n",
    "    \n",
    "    # pandas needs to have the same index in order to merge datasets\n",
    "    random_sample.index = temp[temp[variable].isnull()].index\n",
    "    temp.loc[temp[variable].isnull(), variable] = random_sample\n",
    "    return temp[variable]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Age and ticket\n",
    "# add variable indicating missingness\n",
    "for df in [X_train, X_test, submission]:\n",
    "    for var in ['Age', 'Ticket_numerical']:\n",
    "        df[var+'_NA'] = np.where(df[var].isnull(), 1, 0)\n",
    "    \n",
    "# replace by random sampling\n",
    "for df in [X_train, X_test, submission]:\n",
    "    for var in ['Age', 'Ticket_numerical']:\n",
    "        df[var] = impute_na(X_train, df, var)\n",
    "    \n",
    "\n",
    "# Cabin numerical\n",
    "extreme = X_train.Cabin_numerical.mean() + X_train.Cabin_numerical.std()*3\n",
    "for df in [X_train, X_test, submission]:\n",
    "    df.Cabin_numerical.fillna(extreme, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Engineering Missing Data in categorical variables (section 6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Embarked 0.00280898876404\n",
      "Cabin_categorical 0.77106741573\n"
     ]
    }
   ],
   "source": [
    "# print variables with missing data\n",
    "for col in categorical:\n",
    "    if X_train[col].isnull().mean()>0:\n",
    "        print(col, X_train[col].isnull().mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Embarked NA imputed by most frequent category, because NA is low\n",
    "- Cabin_categorical imputed by 'Missing', because NA is high"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# add label indicating 'Missing' to Cabin categorical\n",
    "# or replace by most frequent label in Embarked\n",
    "\n",
    "for df in [X_train, X_test, submission]:\n",
    "    df['Embarked'].fillna(X_train['Embarked'].mode()[0], inplace=True)\n",
    "    df['Cabin_categorical'].fillna('Missing', inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PassengerId            0\n",
       "Survived               0\n",
       "Pclass                 0\n",
       "Sex                    0\n",
       "Age                    0\n",
       "SibSp                  0\n",
       "Parch                  0\n",
       "Fare                   0\n",
       "Embarked               0\n",
       "Cabin_numerical        0\n",
       "Cabin_categorical      0\n",
       "Ticket_numerical       0\n",
       "Ticket_categorical     0\n",
       "Title                  0\n",
       "Family_size            0\n",
       "is_mother              0\n",
       "Age_NA                 0\n",
       "Ticket_numerical_NA    0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check absence of null values\n",
    "X_train.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PassengerId            0\n",
       "Survived               0\n",
       "Pclass                 0\n",
       "Sex                    0\n",
       "Age                    0\n",
       "SibSp                  0\n",
       "Parch                  0\n",
       "Fare                   0\n",
       "Embarked               0\n",
       "Cabin_numerical        0\n",
       "Cabin_categorical      0\n",
       "Ticket_numerical       0\n",
       "Ticket_categorical     0\n",
       "Title                  0\n",
       "Family_size            0\n",
       "is_mother              0\n",
       "Age_NA                 0\n",
       "Ticket_numerical_NA    0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PassengerId            0\n",
       "Pclass                 0\n",
       "Sex                    0\n",
       "Age                    0\n",
       "SibSp                  0\n",
       "Parch                  0\n",
       "Fare                   1\n",
       "Embarked               0\n",
       "Cabin_numerical        0\n",
       "Cabin_categorical      0\n",
       "Ticket_numerical       0\n",
       "Ticket_categorical     0\n",
       "Title                  0\n",
       "Family_size            0\n",
       "is_mother              0\n",
       "Age_NA                 0\n",
       "Ticket_numerical_NA    0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "submission.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Fare in the submission dataset contains one null value, I will replace it by the median \n",
    "submission.Fare.fillna(X_train.Fare.median(), inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Outliers in Numerical variables (section 8 and 15)\n",
    "\n",
    "As I was analysing the outliers at the beginning of the notebook, I was taking a note on the preprocessing that I thought would be more convenient for each one of them. The notes are summarised here:\n",
    "\n",
    "- Age: top-coding (73)\n",
    "- Fare: equal frequency binning\n",
    "- Sibsp: top-coding (4)\n",
    "- Parch: top-coding (2)\n",
    "- Family Size: top-coding (7)\n",
    "- Ticket_number: equal frequency binning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def top_code(df, variable, top):\n",
    "    return np.where(df[variable]>top, top, df[variable])\n",
    "\n",
    "for df in [X_train, X_test, submission]:\n",
    "    df['Age'] = top_code(df, 'Age', 73)\n",
    "    df['SibSp'] = top_code(df, 'SibSp', 4)\n",
    "    df['Parch'] = top_code(df, 'Parch', 2)\n",
    "    df['Family_size'] = top_code(df, 'Family_size', 7)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Age  max value:  73.0\n",
      "SibSp  max value:  4\n",
      "Parch  max value:  2\n",
      "Family_size  max value:  7\n"
     ]
    }
   ],
   "source": [
    "# let's check that it worked\n",
    "for var in ['Age',  'SibSp', 'Parch', 'Family_size']:\n",
    "    print(var, ' max value: ', X_train[var].max())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Age  max value:  73.0\n",
      "SibSp  max value:  4\n",
      "Parch  max value:  2\n",
      "Family_size  max value:  7\n"
     ]
    }
   ],
   "source": [
    "# let's check that it worked\n",
    "for var in ['Age',  'SibSp', 'Parch', 'Family_size']:\n",
    "    print(var, ' max value: ', submission[var].max())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# find quantiles and discretise train set\n",
    "X_train['Fare'], bins = pd.qcut(x=X_train['Fare'], q=8, retbins=True, precision=3, duplicates='raise')\n",
    "X_test['Fare'] = pd.cut(x = X_test['Fare'], bins=bins, include_lowest=True)\n",
    "submission['Fare'] = pd.cut(x = submission['Fare'], bins=bins, include_lowest=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "submission.Fare.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x8b766495c0>"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAswAAAHBCAYAAAB9p+1ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xu4XWV97v3vTQiEiAJCqkDABAQkEgQMqEVb0GpB+hq1\narHiAXWnHhCtW0u08Fbt3n3Rsi3QImxExDN2g1Z2QUW34OFVhACRgxwMiBAOElCQgyiH3/5jjsTF\nIozMtTLXGpkr38915cqY4zDXb41rzDnv9cxnPE+qCkmSJEmrt0HXBUiSJEnrMgOzJEmS1MLALEmS\nJLUwMEuSJEktDMySJElSCwOzJEmS1MLALEmSJLUwMEuSJEktDMySJElSiw27LmB1ttpqq5ozZ07X\nZUiSJGkKu/jii++oqllr2m+dDMxz5sxhyZIlXZchSZKkKSzJL/rZzy4ZkiRJUgsDsyRJktTCwCxJ\nkiS1WCf7MEuSJE11Dz74IMuXL+eBBx7oupQpb8aMGcyePZvp06eP63gDsyRJUgeWL1/OE5/4RObM\nmUOSrsuZsqqKO++8k+XLlzN37txxPYddMiRJkjrwwAMPsOWWWxqWJ1gSttxyy7VqyTcwS5IkdcSw\nPDnW9jwbmCVJkqQW9mGWJElaB8xZfPZAn++Gow9q3X7XXXfxxS9+kXe84x1jet6XvvSlfPGLX2Tz\nzTdfm/KGii3MkiRJ66G77rqLT3ziE49Z/9BDD7Ued84556xXYRlsYZYkSVovLV68mOuuu4499tiD\n6dOnM2PGDLbYYguuvvpqrr32Wl7+8pdz00038cADD/Dud7+bRYsWATBnzhyWLFnCvffey4EHHsjz\nn/98fvjDH7Ltttvyta99jU022aTj32zwbGGWJElaDx199NHsuOOOLF26lH/+53/mkksu4bjjjuPa\na68F4NRTT+Xiiy9myZIlHH/88dx5552PeY6f/exnvPOd7+TKK69k880358wzz5zsX2NS2MIsSZIk\n9tlnn0eNU3z88cfz1a9+FYCbbrqJn/3sZ2y55ZaPOmbu3LnsscceADz72c/mhhtumLR6J5OBWZIk\nSTzhCU9YtXz++efz7W9/mx/96EfMnDmT/fbbb7XjGG+88carlqdNm8Zvf/vbSal1shmYJ8D8z8wf\n13GXv/HyAVciSZK0ek984hO55557Vrvt7rvvZosttmDmzJlcffXVXHDBBZNc3brFwCxJkrQOWNMw\ncIO25ZZbsu+++7LbbruxySab8JSnPGXVtgMOOICTTjqJXXfdlV122YXnPve5k1rbusbALEmStJ76\n4he/uNr1G2+8MV//+tdXu21lP+WtttqKK664YtX6973vfQOvb13hKBmSJElSCwOzJEmS1MLALEmS\nJLUwMEuSJEktDMySJElSCwOzJEmS1MJh5SRJktYFH9pswM93d+vmu+66iy9+8Yu84x3vGPNTH3vs\nsSxatIiZM2eOt7qh0ldgTnIAcBwwDTilqo4etf0ZwKeBvYC/r6pjRmzbHDgF2A0o4M1V9aPBlC9J\n0hQ33hC1hrAk3XXXXXziE58Yd2A+5JBDDMwrJZkGnAC8GFgOXJTkrKr66YjdfgUcDrx8NU9xHPCN\nqnpVko2A9ePMSpIkrcMWL17Mddddxx577MGLX/xi/uiP/oh///d/53e/+x2veMUr+PCHP8x9993H\na17zGpYvX87DDz/MUUcdxS9/+UtuueUW9t9/f7baaivOO++8rn+VCddPC/M+wLKquh4gyenAQmBV\nYK6q24HbkzxqTsckmwF/Aryp2e/3wO8HUrkkSZLG7eijj+aKK65g6dKlnHvuuZxxxhlceOGFVBUv\ne9nL+N73vseKFSvYZpttOPvsswG4++672Wyzzfj4xz/Oeeedx1ZbbdXxbzE5+rnpb1vgphGPlzfr\n+jEXWAF8OsmlSU5J8oQx1ihJkqQJdO6553Luueey5557stdee3H11Vfzs5/9jPnz5/Otb32LI444\ngu9///tsttmA+1kPiYkeJWNDev2aT6yqPYH7gMWr2zHJoiRLkixZsWLFBJclSZKklaqKD3zgAyxd\nupSlS5eybNky3vKWt7DzzjtzySWXMH/+fI488kg+8pGPdF1qJ/oJzDcD2414PLtZ14/lwPKq+nHz\n+Ax6AfoxqurkqlpQVQtmzZrV59NLkiRpPJ74xCdyzz33APDnf/7nnHrqqdx7770A3Hzzzdx+++3c\ncsstzJw5k0MOOYT3v//9XHLJJY85dn3QTx/mi4CdksylF5QPBv66nyevqtuS3JRkl6q6BngRI/o+\nS5IkqTHJI5tsueWW7Lvvvuy2224ceOCB/PVf/zXPe97zANh00035/Oc/z7Jly3j/+9/PBhtswPTp\n0znxxBMBWLRoEQcccADbbLPNenHTX6pqzTslLwWOpTes3KlV9d+TvA2gqk5K8lRgCfAk4BHgXmBe\nVf0myR70hpXbCLgeOLSqft328xYsWFBLlixZi1+rW/M/M39cx13+xssHXIkkaeg5rNyUddVVV7Hr\nrrt2XcZ6Y3XnO8nFVbVgTcf2NQ5zVZ0DnDNq3Ukjlm+j11VjdccuBdZYiCRJkrQuWm9m+puz+Owx\nH3PD0QeteSdJkiRNaetNYB6X8X4NNnf7wdYhSZKkzkz0sHKSJEnSUDMwS5IkSS0MzJIkSVIL+zBL\nkiStA8Y7LO3jGfRwtR/60IfYdNNNed/73jeu40866SRmzpzJG97whr6PueWWWzj88MM544wzxvUz\nB8XALEmSpAn3tre9bczHbLPNNp2HZbBLhiRJ0nrrvvvu46CDDuJZz3oWu+22G1/+8peZM2cOd9xx\nBwBLlixhv/32W7X/T37yE573vOex00478clPfhKA888/nz/90z9l4cKF7LDDDixevJgvfOEL7LPP\nPsyfP5/rrrsO6LVQH3PMMQAcf/zxzJs3j913352DDz4YgO9+97vsscce7LHHHuy5557cc8893HDD\nDey2224APPDAAxx66KHMnz+fPffcc9UMg6eddhqvfOUrOeCAA9hpp534u7/7u4GfJ1uYJUmS1lPf\n+MY32GabbTj77N58FXfffTdHHHHE4+5/2WWXccEFF3Dfffex5557ctBBvTkrfvKTn3DVVVfx5Cc/\nmR122IG3vvWtXHjhhRx33HH867/+K8cee+yjnufoo4/m5z//ORtvvDF33XUXAMcccwwnnHAC++67\nL/feey8zZsx41DEnnHACSbj88su5+uqreclLXsK1114LwNKlS7n00kvZeOON2WWXXXjXu97Fdttt\nN7DzZAuzJEnSemr+/Pl861vf4ogjjuD73/8+m23WPgfFwoUL2WSTTdhqq63Yf//9ufDCCwHYe++9\n2Xrrrdl4443ZcccdeclLXrLq+W+44YbHPM/uu+/O6173Oj7/+c+z4Ya99tt9992X9773vRx//PHc\nddddq9av9IMf/IBDDjkEgGc84xk87WlPWxWYX/SiF7HZZpsxY8YM5s2bxy9+8Yu1Oi+jGZglSZLW\nUzvvvDOXXHIJ8+fP58gjj+QjH/kIG264IY888gjQ6wYxUpLVPt54441Xrdtggw1WPd5ggw146KGH\nHvNzzz77bN75zndyySWXsPfee/PQQw+xePFiTjnlFH7729+y7777cvXVV/f9e4z8+dOmTVvtz1wb\nBmZJkqT11C233MLMmTM55JBDeP/7388ll1zCnDlzuPjiiwE488wzH7X/1772NR544AHuvPNOzj//\nfPbee+8x/8xHHnmEm266if3335+PfvSj3H333dx7771cd911zJ8/nyOOOIK99977MYH5BS94AV/4\nwhcAuPbaa7nxxhvZZZddxvmbj419mCVJktYBgx4Grq+fefnlvP/972eDDTZg+vTpnHjiifz2t7/l\nLW95C0cdddSjbviDXleK/fffnzvuuIOjjjqKbbbZZlW3iH49/PDDHHLIIdx9991UFYcffjibb745\nRx11FOeddx4bbLABz3zmMznwwAO59dZbVx33jne8g7e//e3Mnz+fDTfckNNOO+1RLcsTKVU1KT9o\nLBYsWFBLliwZ6HPOWXz2mI+5YcZfj+tnzZ+7/biO6+KFIklax32ovU/p4x9392Dr0MBdddVV7Lrr\nrl2Xsd5Y3flOcnFVLVjTsXbJkCRJkloYmCVJkqQWBmZJkqSOrItdY6eitT3PBmZJkqQOzJgxgzvv\nvNPQPMGqijvvvPMxE6GMhaNkSJIkdWD27NksX76cFStWdF3KlDdjxgxmz5497uMNzJIkSR2YPn06\nc+fO7boM9cEuGZIkSVILA7MkSZLUwi4ZkiRJGrv1aFIdW5glSZKkFgZmSZIkqYWBWZIkSWphYJYk\nSZJaGJglSZKkFn0F5iQHJLkmybIki1ez/RlJfpTkd0net5rt05JcmuQ/B1G0JEmSNFnWGJiTTANO\nAA4E5gGvTTJv1G6/Ag4Hjnmcp3k3cNVa1ClJkiR1op8W5n2AZVV1fVX9HjgdWDhyh6q6vaouAh4c\nfXCS2cBBwCkDqFeSJEmaVP0E5m2Bm0Y8Xt6s69exwN8Bj4zhGEmSJGmdMKEz/SX5C+D2qro4yX5r\n2HcRsAhg++23n8iypKlhPZphSZKkLvXTwnwzsN2Ix7Obdf3YF3hZkhvodeV4YZLPr27Hqjq5qhZU\n1YJZs2b1+fSSJEnSxOonMF8E7JRkbpKNgIOBs/p58qr6QFXNrqo5zXHfqapDxl2tJEmSNMnW2CWj\nqh5KchjwTWAacGpVXZnkbc32k5I8FVgCPAl4JMl7gHlV9ZsJrF2SJEmacH31Ya6qc4BzRq07acTy\nbfS6arQ9x/nA+WOuUJIkSeqQM/1JkiRJLQzMkiRJUgsDsyRJktTCwCxJkiS1MDBLkiRJLQzMkiRJ\nUosJnRpbkqa6+Z+ZP+ZjLn/j5RNQiSRpotjCLEmSJLUwMEuSJEktDMySJElSC/swS5IkadKM594P\n6Pb+D1uYJUmSpBa2MEuSVmvO4rPHddwNRx804EokqVu2MEuSJEktDMySJElSC7tkSJLWCcN4I5Ck\n9YMtzJIkSVILA7MkSZLUwsAsSZIktbAPsyRJ0hrYx379ZmCWJGkSjHtc6xkDLkTSmNklQ5IkSWph\nYJYkSZJa2CVD6phf0w6W0zlLkgbNFmZJkiSphS3MkiRJ6zG/6VwzW5glSZKkFgZmSZIkqUVfgTnJ\nAUmuSbIsyeLVbH9Gkh8l+V2S941Yv12S85L8NMmVSd49yOIlSZKkibbGPsxJpgEnAC8GlgMXJTmr\nqn46YrdfAYcDLx91+EPAf62qS5I8Ebg4ybdGHStJkiSts/ppYd4HWFZV11fV74HTgYUjd6iq26vq\nIuDBUetvrapLmuV7gKuAbQdSuSRJkjQJ+gnM2wI3jXi8nHGE3iRzgD2BH4/1WEmSJKkrk3LTX5JN\ngTOB91TVbx5nn0VJliRZsmLFiskoS5IkSVqjfsZhvhnYbsTj2c26viSZTi8sf6GqvvJ4+1XVycDJ\nAAsWLKh+n1+SJK1/nNVTk6mfFuaLgJ2SzE2yEXAwcFY/T54kwKeAq6rq4+MvU5IkSerGGluYq+qh\nJIcB3wSmAadW1ZVJ3tZsPynJU4ElwJOAR5K8B5gH7A68Hrg8ydLmKT9YVedMwO8iSZIkDVxfU2M3\nAfecUetOGrF8G72uGqP9AMjaFChJkiR1qa/ALElT3oc2G99xc7cfbB2SpHWOU2NLkiRJLQzMkiRJ\nUgsDsyRJktTCwCxJkiS1MDBLkiRJLQzMkiRJUgsDsyRJktTCwCxJkiS1MDBLkiRJLQzMkiRJUgsD\nsyRJktTCwCxJkiS1MDBLkiRJLQzMkiRJUgsDsyRJktTCwCxJkiS1MDBLkiRJLTbsugBJkjR48z8z\nf1zHXf7GywdciTT8bGGWJEmSWhiYJUmSpBYGZkmSJKmFfZi1zrMfniRJ6pItzJIkSVILA7MkSZLU\nwsAsSZIktTAwS5IkSS0MzJIkSVKLvgJzkgOSXJNkWZLFq9n+jCQ/SvK7JO8by7GSJEnSumyNgTnJ\nNOAE4EBgHvDaJPNG7fYr4HDgmHEcK0mSJK2z+mlh3gdYVlXXV9XvgdOBhSN3qKrbq+oi4MGxHitJ\nkiSty/oJzNsCN414vLxZ14+1OVaSJEnq3Dpz01+SRUmWJFmyYsWKrsuRJEmSgP4C883AdiMez27W\n9aPvY6vq5KpaUFULZs2a1efTS5IkSROrn8B8EbBTkrlJNgIOBs7q8/nX5lhJkiSpcxuuaYeqeijJ\nYcA3gWnAqVV1ZZK3NdtPSvJUYAnwJOCRJO8B5lXVb1Z37ET9MpIkSdKgrTEwA1TVOcA5o9adNGL5\nNnrdLfo6VlJ35n9m/riOu/yNlw+4EkmShsM6c9OfJEmStC4yMEuSJEktDMySJElSCwOzJEmS1KKv\nm/6kgfjQZuM7bu72g61D0sTytS5pirGFWZIkSWphYJYkSZJaGJglSZKkFgZmSZIkqYU3/WnM5iw+\ne1zH3TBjwIVIkiRNAluYJUmSpBYGZkmSJKmFgVmSJElqYWCWJEmSWhiYJUmSpBaOkiFJktYfTt2u\ncbCFWZIkSWphYJYkSZJaGJglSZKkFgZmSZIkqYWBWZIkSWphYJYkSZJaGJglSZKkFgZmSZIkqYWB\nWZIkSWphYJYkSZJaGJglSZKkFgZmSZIkqUVfgTnJAUmuSbIsyeLVbE+S45vtlyXZa8S2v01yZZIr\nknwpyYxB/gKSJEnSRFpjYE4yDTgBOBCYB7w2ybxRux0I7NT8WwSc2By7LXA4sKCqdgOmAQcPrHpJ\nkiRpgvXTwrwPsKyqrq+q3wOnAwtH7bMQ+Gz1XABsnmTrZtuGwCZJNgRmArcMqHZJkiRpwvUTmLcF\nbhrxeHmzbo37VNXNwDHAjcCtwN1Vde74y5UkSZIm14Te9JdkC3qtz3OBbYAnJDnkcfZdlGRJkiUr\nVqyYyLIkSZKkvvUTmG8GthvxeHazrp99/gz4eVWtqKoHga8Af7y6H1JVJ1fVgqpaMGvWrH7rlyRJ\nkiZUP4H5ImCnJHOTbETvpr2zRu1zFvCGZrSM59LrenErva4Yz00yM0mAFwFXDbB+SZIkaUJtuKYd\nquqhJIcB36Q3ysWpVXVlkrc1208CzgFeCiwD7gcObbb9OMkZwCXAQ8ClwMkT8YtIkiRJE2GNgRmg\nqs6hF4pHrjtpxHIB73ycY/8B+Ie1qFGSJEnqjDP9SZIkSS0MzJIkSVILA7MkSZLUwsAsSZIktTAw\nS5IkSS0MzJIkSVILA7MkSZLUwsAsSZIktTAwS5IkSS0MzJIkSVILA7MkSZLUwsAsSZIktTAwS5Ik\nSS0MzJIkSVILA7MkSZLUwsAsSZIktTAwS5IkSS0MzJIkSVILA7MkSZLUwsAsSZIktTAwS5IkSS0M\nzJIkSVILA7MkSZLUwsAsSZIktTAwS5IkSS0MzJIkSVILA7MkSZLUwsAsSZIktTAwS5IkSS36CsxJ\nDkhyTZJlSRavZnuSHN9svyzJXiO2bZ7kjCRXJ7kqyfMG+QtIkiRJE2mNgTnJNOAE4EBgHvDaJPNG\n7XYgsFPzbxFw4ohtxwHfqKpnAM8CrhpA3ZIkSdKk6KeFeR9gWVVdX1W/B04HFo7aZyHw2eq5ANg8\nydZJNgP+BPgUQFX9vqruGmD9kiRJ0oTqJzBvC9w04vHyZl0/+8wFVgCfTnJpklOSPGF1PyTJoiRL\nkixZsWJF37+AJEmSNJEm+qa/DYG9gBOrak/gPuAxfaABqurkqlpQVQtmzZo1wWVJkiRJ/eknMN8M\nbDfi8exmXT/7LAeWV9WPm/Vn0AvQkiRJ0lDoJzBfBOyUZG6SjYCDgbNG7XMW8IZmtIznAndX1a1V\ndRtwU5Jdmv1eBPx0UMVLkiRJE23DNe1QVQ8lOQz4JjANOLWqrkzytmb7ScA5wEuBZcD9wKEjnuJd\nwBeasH39qG2SJEnSOm2NgRmgqs6hF4pHrjtpxHIB73ycY5cCC9aiRkmSJKkzzvQnSZIktTAwS5Ik\nSS0MzJIkSVILA7MkSZLUwsAsSZIktTAwS5IkSS0MzJIkSVILA7MkSZLUwsAsSZIktTAwS5IkSS0M\nzJIkSVILA7MkSZLUwsAsSZIktTAwS5IkSS0MzJIkSVILA7MkSZLUwsAsSZIktTAwS5IkSS0MzJIk\nSVILA7MkSZLUwsAsSZIktTAwS5IkSS0MzJIkSVILA7MkSZLUwsAsSZIktTAwS5IkSS0MzJIkSVIL\nA7MkSZLUoq/AnOSAJNckWZZk8Wq2J8nxzfbLkuw1avu0JJcm+c9BFS5JkiRNhjUG5iTTgBOAA4F5\nwGuTzBu124HATs2/RcCJo7a/G7hqrauVJEmSJlk/Lcz7AMuq6vqq+j1wOrBw1D4Lgc9WzwXA5km2\nBkgyGzgIOGWAdUuSJEmTop/AvC1w04jHy5t1/e5zLPB3wCNtPyTJoiRLkixZsWJFH2VJkiRJE29C\nb/pL8hfA7VV18Zr2raqTq2pBVS2YNWvWRJYlSZIk9a2fwHwzsN2Ix7Obdf3ssy/wsiQ30OvK8cIk\nnx93tZIkSdIk6ycwXwTslGRuko2Ag4GzRu1zFvCGZrSM5wJ3V9WtVfWBqppdVXOa475TVYcM8heQ\nJEmSJtKGa9qhqh5KchjwTWAacGpVXZnkbc32k4BzgJcCy4D7gUMnrmRJkiRp8qwxMANU1Tn0QvHI\ndSeNWC7gnWt4jvOB88dcoSRJktQhZ/qTJEmSWhiYJUmSpBYGZkmSJKmFgVmSJElqYWCWJEmSWhiY\nJUmSpBYGZkmSJKmFgVmSJElqYWCWJEmSWhiYJUmSpBYGZkmSJKmFgVmSJElqYWCWJEmSWhiYJUmS\npBYGZkmSJKmFgVmSJElqYWCWJEmSWhiYJUmSpBYGZkmSJKmFgVmSJElqYWCWJEmSWhiYJUmSpBYG\nZkmSJKmFgVmSJElqYWCWJEmSWhiYJUmSpBYGZkmSJKmFgVmSJElq0VdgTnJAkmuSLEuyeDXbk+T4\nZvtlSfZq1m+X5LwkP01yZZJ3D/oXkCRJkibSGgNzkmnACcCBwDzgtUnmjdrtQGCn5t8i4MRm/UPA\nf62qecBzgXeu5lhJkiRpndVPC/M+wLKqur6qfg+cDiwctc9C4LPVcwGweZKtq+rWqroEoKruAa4C\nth1g/ZIkSdKE6icwbwvcNOLxch4bete4T5I5wJ7Aj1f3Q5IsSrIkyZIVK1b0UZYkSZI08Sblpr8k\nmwJnAu+pqt+sbp+qOrmqFlTVglmzZk1GWZIkSdIa9ROYbwa2G/F4drOur32STKcXlr9QVV8Zf6mS\nJEnS5OsnMF8E7JRkbpKNgIOBs0btcxbwhma0jOcCd1fVrUkCfAq4qqo+PtDKJUmSpEmw4Zp2qKqH\nkhwGfBOYBpxaVVcmeVuz/STgHOClwDLgfuDQ5vB9gdcDlydZ2qz7YFWdM9hfQ5IkSZoYawzMAE3A\nPWfUupNGLBfwztUc9wMga1mjJEmS1Bln+pMkSZJaGJglSZKkFgZmSZIkqYWBWZIkSWphYJYkSZJa\nGJglSZKkFgZmSZIkqYWBWZIkSWphYJYkSZJaGJglSZKkFgZmSZIkqYWBWZIkSWphYJYkSZJaGJgl\nSZKkFgZmSZIkqYWBWZIkSWphYJYkSZJaGJglSZKkFgZmSZIkqYWBWZIkSWphYJYkSZJaGJglSZKk\nFgZmSZIkqYWBWZIkSWphYJYkSZJaGJglSZKkFgZmSZIkqYWBWZIkSWrRV2BOckCSa5IsS7J4NduT\n5Phm+2VJ9ur3WEmSJGldtsbAnGQacAJwIDAPeG2SeaN2OxDYqfm3CDhxDMdKkiRJ66x+Wpj3AZZV\n1fVV9XvgdGDhqH0WAp+tnguAzZNs3eexkiRJ0jqrn8C8LXDTiMfLm3X97NPPsZIkSdI6a8OuC1gp\nySJ63TkA7k1yTZf1AKR981bAHavfdMX4ft6b1vATh5znc7A8n4M1mefTc+m1ORaez8HyfA7OFDmX\nT+tnp34C883AdiMez27W9bPP9D6OBaCqTgZO7qOedUKSJVW1oOs6pgrP52B5PgfL8zk4nsvB8nwO\nludzcKbaueynS8ZFwE5J5ibZCDgYOGvUPmcBb2hGy3gucHdV3drnsZIkSdI6a40tzFX1UJLDgG8C\n04BTq+rKJG9rtp8EnAO8FFgG3A8c2nbshPwmkiRJ0gToqw9zVZ1DLxSPXHfSiOUC3tnvsVPE0HQf\nGRKez8HyfA6W53NwPJeD5fkcLM/n4Eypc5le1pUkSZK0Ok6NLUmSJLUwMEuSJEkt1plxmLX+SPLK\nPnZ7oOn/rjVI8uQ+dnukqu6a8GKGXJL39rHbfVX1Pye8mCnAa3OwPJ+DlaSfUbt+VVVvmuhaht36\ncG3ah7kPSX6zpl2AW6tq58moZ9gluRP4Gu1jnv9JVe04SSUNtSQPALfQfj6nVdX2k1TS0EpyK3Ai\n7efydb7W++O1OViez8FK8jPgrW27ACdU1TMnqaShtT5cm7Yw9+e6qtqzbYckl05WMVPA16vqzW07\nJPn8ZBUzBVzl9Tkwn6uqj7TtkOQJk1XMFOC1OViez8H6+6r6btsOST48WcUMuSl/bdrC3IckO1TV\n9Wu7jzQRksyoqgfWdh9p0Lw2B8vzqXXV+nBtetNfH/oJwoblsUnyJ0l2aZb3TfK+JAd1XdcwWt0b\n0Oj+ZMP8JjWZkjwnyZOa5U2SfCTJ/07y0SSbdV3fsPHaHCzP52AlOSzJVs3y05N8L8ldSX6cZH7X\n9Q2TldddkllJ9kyye5JNV7fPsDIw9yHJm0csz07yf5oX1Q+T2JdxjJIcCxwNfC7JPwL/DGwC/G2S\nf+60uCHU/MFxVZIrm8D3LeCiJDcleV7X9Q2ZU+nNVgpwHPAk4KPNuk93VdSwSnLkiOV5Sa4FLk5y\nQ5LndFjaUPJ8Dtzbq+qOZvk44F+qanPgCOCkxz9MozXX47eBHwE/Bj4JXJ7ktKnS2GCXjD4kuaSq\n9mqW/x34NnAKsBA4rKpe1GV9wybJlcBu9ELyzcC2VXV/kunApVW1W6cFDpkkFwJvATYF/jfw8qr6\nQZK9gH/qGa0nAAAaL0lEQVStqn07LXCIJLmqqnZtlle97pvHS6tqj+6qGz6j3jvPBv6tqr6eZB/g\n2Kr6424rHC6ez8FKck1Vrfym86Kq2nvEtsuqavfuqhsuSS4A3lhV1zTX4zur6o1J/gvw51X1qo5L\nXGu2MI/dLlV1clU9UlVfBfoZSkWPVs106o+sfNz8/whek+Mxvaour6ofASuq6gcAVXUJvT9K1L8r\nkhzaLP8kyQKA5pukB7sra0rYtqq+DlBVF+K1ubY8n2vvjKYFdAfgq0nek+RpzXvAjV0XN2Q2qapr\nYNX1OL9Z/iQwJUYZcZSM/sxOcjy94VK2SjK9qlZ+eE7vsK5hdXaS7wMz6LXU/3vz1+mfAt/rtLLh\nNPKPjA+M2rbRZBYyBbwVOK756vsO4EdJbgJuon34Ka3eDs1Yt6H3PjqzqlZ2efG9c+w8nwNUVX+f\n5E3Al4AdgY2BRcB/AK/rsLRhdF2So4DvAK8ElgI03xxPiYYwu2T0IckbR606q6p+neSpwOFV9cEu\n6hpmTd/aqqoLkuwIvILeX/RnVNUj7UdrpCQvA7494oNz5fodgb+sqo91U9nwam78m0uvUWF5Vf2y\n45KGUpI/HbXq4qq6N8lTgFdV1Qld1DWsPJ9aVyXZHPggMA/4CXB0Vd3T9F/etaou6LTAATAwS1KL\nJE+uql91XYekyZPkqVV1W9d1aN0xJZrJJ1qSjyfxxqlJkOTyrmsYNkl2H7E8PcmRSc5K8k9JZnZZ\n27BxFILJk+TkrmsYNkm+kuR1o4fr0oT4VNcFDJMk05L8TZJ/TPLHo7Yd+XjHDRNbmPuQZAXwC2AW\n8GXgS1U11DPWdCnJKx9vE3BSVc2azHqG3ag75/8HsCW9IdBeDmxZVW/osr5h4igEg5VRYwSP3AT8\npKpmT2Y9wy7JzfSG7XohvdGavgScXVW/77QwrfeSnALMBC4EXg98t6re22x71IhDw8qb/vqzvKoW\nNHfK/xXw+STT6L1Zfamqru22vKHzZeAL/GF0jJFmTHItU0FGLL8I2LuqHkzyPXp9yTQ+jxqFIImj\nEIzdysaGkddoNY//qJOKhtvtVfWqpo/9QuC/ACcn+U96n0XndlvecEvydOBZ9KZ5/mnX9QyZfVYO\nw5fk34BPJPkK8Foe/fofWgbm/hRAE4z/EfjH5mvw1wLnAE/vsLZhdBlwTFVdMXpDkj/roJ5ht1mS\nV9DrYrXJyhFcqqqS+BXS2DgKwWBdD7yoqh4zRFcz+ojGZuVn0W+Az9Gb/GlL4NXAYsDAPAZJzgNe\nXVV3JHk9cBS9kZo+lOTkqvrXbiscKqtGZKqqh4BFSf5feqNmTIkuRAbm/jzmr6Oquoxe8Bs9jJfW\n7D3Abx5n2ysms5Ap4rvAy5rlHyZ5SlX9shnF5Y6W4/RYC0c93gCgGYXgxMkvZ+gdC2zB6se0dfSW\nsbt39IqqupPerHTOTDd2s0bM9Hc48LyqurO59+MCwMDcvyVJDqiqb6xcUVUfSXILU+S90z7MfUiy\naVU95o1KkiQNpySXAn9RVTc3rc0HVtUDTZfLy6pqSky4ocGwhbkPhuXBS7I/8JfAdsDDwLXAKVW1\nrNPChlTTmkxV3ZZkFvAC4JqqurLbyoZfkmuraueu65hqHLZrfJr+y7Oq6rpR63dvvvlU//4WODfJ\nmcCVwHeSfBN4Pr0bpzUGU/1zyGHl1lKSS7quYdgk+f+AN9D7yutB4Lrm3/9K8uouaxtGSf6G3p3z\nFyR5O/CfwEHAV5K8pdPihkySe5L8pvn/niT3ADuuXN91fVOMw3aNUZLXAFcDZya5MsneIzaf1k1V\nw6uqzgf+GLiV3mfRxcADwLuq6pgOSxs668PnkF0yNOmSXF5V85vlDekNP7Nvki2A71fVbt1WOFya\nsaufA2xCb0SCpzd/4W8BnFdVe3Ra4BBJcjywOfD+lbP7Jfl5Vc3ttjIJkiyl123g1maow88CH6iq\nrya5tKr27LhErafWh88hW5jVhUdGjM+6DTANoKp+zRQZfmaSPVhV9zc3/1y38mvu5nz6F/EYVNXh\nwHHAl5IcnmQDPIcD1UzlrvGZVlW3Qm+oQ2B/4Mgkh+N1OmZJDkuyVbO8Y5LvJfl1kh8nmd91fUNm\nyn8O2Ye5D0l+BXyF3rjL3ymb5dfWPwGXNrOo7QK8HaDp8+S4wWNXSaY3w8kdtHJlkhn4R/GYVdXF\nzfCGh9EbgcSxwcdpNZMUBTih+WaJqvrK5Fc11O5JsuPK/stNS/N+wH8A3qA2dm+vqn9rlo8H/qVp\nrd+P3qgjzvDbvyn/OWSXjD4kuYbe8DKvBeYAZ9AbJP6CLusaZk0L8w7Asqq6q+t6hlmS7YFbmrEv\nR67fFti1qr7dTWXDL8nWwJ5VdU7XtQyjJA8C3wRu5w/fHr2K3ntoVdWbu6ptGCV5FnDf6Jujk0wH\nXlNVX+imsuGU5Jqq2qVZvqiq9h6x7bKVE3FozdaHzyEDcx9GTZe7PXBw829z4PSq+mCX9UmaeI7q\nMHbNTWlHA2dU1YnNOvuEa52Q5L8D2wIfofeZfj/wVXpTj/9lVf1Fh+VpHTMlmsknwap+tVV1Y1V9\nrAnQLwV+111ZU4+jjgxWkpO7rmEKcVSHMaqqi4AXAxslOa+5Uc1Wmgnga33squrvgfPpdbd8L72Z\nfL8O7AS8rrvKppapcm3awtyHJB+vqvd2XYc0VkmeXVUXd12H1Hw1+y/Agqraoet6phpf61pXTZVr\n08AsSS2SvKyqzuq6DklSdxwlQ5POUUcGqxlx4C3AK+gN0wdwM/A14FPNXcvqg6M6DFYzxfBbgdnA\nN6rq/x+x7ciq+m+dFTfFJDm5qhZ1XYfWT+vDa90WZk06Rx0ZrCRfAu4CPgMsb1bPBt4IPLmq/qqr\n2oaNozoMVpJTgJnAhcDr6U1S9N5m26qbqdWfEePXP2YT8JOqmj2Z9UgrrQ+vdQOzJp2jjgxWkmur\nauexbtNjOarDYI0cmqtppf8EsBW9P5YvcGa6sUnyML1Z1EZO8FTN422raqNOCptimuEkf1VV3tTf\np/Xhte4oGWshycIkz+m6jiHkqCOD9askr25mpQMgyQZJ/gr4dYd1DR1HdRi4VQGuqh5qugwsBb4D\nbNpZVcPremC/qpo74t8OzR90v+y6uCnkc8DVSY7pupAhMuVf6wbmtfMcetOSfr3rQobMeatbWVVX\nV9WHJ7uYKeBget0Gfpnk2mYGxduAVzbbNAZV9UhVHUdvWKn3dV3PkFuS5ICRK6rqI8Cn6XXH0tgc\nC2zxONs+NpmFTGVV9Wf0Jtb6dNe1DJEp/1q3S4Y0hSTZEqCq7uy6FkkaNkmeXFW/6roOrXtsYV5L\nSV7cdQ3SSlV158iw7PU5OFNl8H0NtyRPTfLUZnlWklcmeWbXdQ2jJEeOWJ7XfDt3cZIbkjy3w9Km\nlKnyOWQL81pKcmNVbd91HdLqeH2OjaMQaF2W5G+AxfSux48CbwKuAJ4PfKyqnI1yDEbdgH428G9V\n9fXm3oVjq+qPu61wapgqn0OOw9yHJI83aUGALSezFmk0r8+BWsHjj0LwR51UJP3BYcAzgU3oXadP\nr6rbkmxB794QA/P4bVtVXweoqguTbNJ1QcNkffgcMjD35wXAIcC9o9YH2Gfyy5makiwEbquqH3dd\ny5Dx+hyc64EXVdWNozckuamDeqYkh+0atwer6n7g/iTXVdVtAFX16yR+XTx2OzRBL8DsJDOb8wsw\nvcO6htGU/xwyMPfnAuD+qvru6A3NJBwajOcA85NsWFUHdl3MEPH6HJyVoxA8JjDjKASD9DlgxyRn\nVpUjkfSvkkxvZu88aOXKJDPwnqTxWDjq8QYASZ4CnDj55Qy1Kf85ZB9mSdKkSxJgXlVd2XUtw6KZ\n6OmWqnpo1PptgV2r6tvdVCZNfQbmMVp5U5DDzgxOkrnAnsBPq+rqruuRNDGSPB14FnBVVf2063q0\nfkvyFeBM4GtVNborgfQofoXThyTbJzk9yQrgx8CFSW5v1s3ptrrhk+Q/RiwvpDcT0P8DnJXkTV3V\nJWmwmtkSt2qWXw+cAxwIfDnJuzotTup1A3wFcGOSf0/yiiROLz4OSd48Ynl2kv+T5K4kP0yyc5e1\nDYqBuT9fBr4KPLWqdqqqpwNbA/8BnN5pZcPpaSOWjwBeWFWHAn8M/G03JUmaALOq6o5m+XDgeVX1\nVnpB5b90V5YEwO1V9Sp6M9H9b3rX5M1JPp3kJZ1WNnwOG7H8cXq56cnAPzNF+oMbmPuzVVV9uaoe\nXrmiqh6uqtOZIsOlTLKR/YA2qqqfAzQfrI90U5L0+JJsnWTjrusYQg82/Wuhd/f8fc3y74Bp3ZQk\nrVIAVfWbqvpcVb0UeAa9b5IXd1rZcNulqk6uqkeq6qv0gvPQc5SM/lyc5BPAZ4CVQ0ttB7wRuLSz\nqobXs5L8ht5wMxsn2bqqbm2+CvNDdECSfAa4Hzihqq7oup4h56gO4/O3wLlJzgSuBL6T5Jv0Jtr4\ndKeVTSG+1sftMf2Wm5lST2r+qX+zkxxP73N9qxGjucAUGaLPm/760AS5t9AbgmZla8nNwFnApxxL\ndDCSbE7vTu8fdV3LVJBkb2B7YJ+qOqLreoadozqMT5LNgL8GdqbXSLOc3k1W3uA7IL7W1bUkbxy1\n6qxmfPCnAodX1Qe7qGuQDMyS1CLJkx0VR5qakjyJXl/760at372qLuuoLK2D7MPcpyR/nuTEJGc1\n/05MckDXdQ2jJL9KckqSFzWtdloLSXZIcmqS/5Zk0ySfTHJFkv/lKC5jk2TfJFcluTLJc5J8C7go\nyU1Jntd1fcMmye4jlqcnObJ5//ynJDO7rG0YeT4HK8lrgKuBM5vX/N4jNp/WTVVaVxmY+5DkWODd\nwHfpzfb1sWb58CTHdVnbkFoBLAU+AixPclyS53Zc0zA7DbiIXn+8C+h9ABwIfAM4tbuyhtK/AK8B\n3gqcDXy4qnak1x3rmC4LG1KnjVg+Gng68D+ATbCP6HicNmLZ87n2Pgg8u6r2AA4FPpfkFc02G3P0\nKHbJ6EOSa6vqMeMINq2j11bVTh2UNbSSXFJVezXL2wMHN/82B06fCn2dJlOSS6tqz2b5xqrafnXb\ntGajzuVVVbXriG2rrlv1Z9T5XArsXVUPNu+dP6mq3dufQSN5PgcryeVVNX/E462B/6R3g/+bfL1r\nJFuY+/PAqK9qVtobeGCyi5kCVv3lXlU3VtXHmjeml9Ibbkpj80iSnZtrdGaSBbBqVjVHHRmbke+J\nHxi1zQkNxm6zZjKIvwQ2WXnXfPVaamytGTvP52Ddk2THlQ+q6lZgP3rfKD2zq6KmkiQLkzyn6zoG\nwWHl+vMm4MQkT6R3hzf0hpW7u9mmsTlvdSubu+Y/PMm1TAV/R2/Q/UeAlwMfSPIs4Ek4OcRYHZVk\nZlXdX1UjZ6TcEfhsh3UNq+8CL2uWf5jkKVX1y+bO+TtajtPqeT4H6+2M6npRVfc09ye9ppuSppzn\nAPOTbFhVB3ZdzNqwS8YYNG9Kq4aVq6rbuqxHejzpTUf865GT7UiSpPGxS8YYVNVtVXVx88+wPAGS\n/EXXNUwFVXVHVT2c5MVd1zJMHIVg4iX5p65rmIp8rQ9WkpO7rmGYJNk+yYxmOUkOTfKvSd6eZEr0\nZrCFeS15I9BgJflwVf1D13VMFaNvAlS7UTek/g9gS3oz0r0c2LKq3tBlfcMmvZm/HrUKeD1N95aq\nOnzSi5qifK0PVpJnV9XFXdcxLJJcQW/inPuTfBTYEfgP4IUAVfXmLusbhCmR+rtkWB4sw/LYJTnr\n8TbRC3zq38j+jC/iD6MQfA/4SUc1DbNX0Ot3ey5/OLcHAwaRcfC1PnkMy2O2QVXd3yz/Gb33zkeA\nzyeZEu+dBuYxSvJkAGf+Gr8kLwPOrSpHGBmMFwCH0BuHeaQA+0x+OUNts2Yc1g0YNQpBEr+OG7t5\nwD8CBwDvq6pbkvxDVX2m47qGla/1SZLk5Kpa1HUdQ+SmJC+squ8AN9AbGOEXSabMH3IG5j40YwV/\njF6L0129VXkS8B1gcVXd0GF5w+jLwH1Jvg58CfimN6etlQuA+6vqu6M3JLmmg3qGmaMQDFBV3QO8\nJ8mzgS8kORvvnVkbvtYHaGUD2Oo20RvmVP17K/DZJB+iN4LY0mas8M2B93ZZ2KDYh7kPSX4EHAuc\nsTLYJZkGvBp4T1U5S90YJLmUXr+mV9H7enY34KvAl1b3QSBp+DWTa7wDeF5VHdJ1PVKSh4Ff8Oiu\nWNU83raqHHt9jJLsCuxMr0F2OXBR0zVj6BmY+5DkZ483m1/bNq3e6Bslm9a71wCvBWZX1XadFTfk\nkmwBPFxVv+m6lmHVXI9U1W1JZtH7Gvyaqrqy28okDVKSnwEvqqobV7PtJj+LNJJfjfXn4iSfSPKc\nJNs0/56T5BPApV0XN4RGDxR/W1UdX1XPA57fUU1Dq7keP5vkbnrdBq5IcmOSDyWZ3nV9wyTJ3wA/\nAi5I8nZ60+QeBHwlyVs6LW6KcdiuwfJ8jsuxwBaPs+1jk1nIVDZVrk1bmPuQZCPgLfSmy1w1cQlw\nFvCpqnI65zFIsl9Vnd91HVNFku8AH6mq85O8kl6L6JH0pnb+I29c6V+Sy+nNTLUJva9qn960NG8B\nnFdVe3Ra4BTisF2D5fnUumqqXJsGZk26JKk1XHj97KOeJD+pqmeNeHxxVT27Wb66qp7RXXXDZdQ4\nzKPP66VVtWd31UkaNLtgqV92yRinJJd0XcMQOy/Ju5rRR1ZJslGSFyb5DPDGjmobRiuSHJJk2yTv\nojekz8qbrHyNj02N6MZy0MqVzQxWnssBSHJt1zVMJZ7P8bML1uAkOSzJVs3y05N8L8ldSX6cZH7X\n9Q2CLczjZGvT+DXh483A64C59IbqmwFMozfBwSeqyr7hfWr+8DiG3pi3S4H3V9WtzfiX+1XVmZ0W\nOESac3lLVT00av22wK5V9e1uKhtOSe6hN+oA/OHehZnA/fSGt35SJ4UNKc/nYNkFa3CSXFlVz2yW\nzwZOqaqvJtkP+O9VtW+nBQ6AgXmckvy3qjqy6zqGXdOatxXw26q6q+t6JA1OMzX25vT+iPtls+7n\nVTW328qGk+dzsOyCNThJrqmqXZrli6pq7xHbLquq3burbjD8irEPzVfbjzI6LK9uH61ZVT1YVbca\nlidGkr/ouoapYqrc6T2Zqupw4DjgS0kOT7IBf2gh1Rh5PgfOLliDc0aS05LsAHw1yXuSPC3JocBj\nhu0bRl4Q/bHPrYbV3mveRX36n10XMIyau+P/rHn4XXrdrzROns+BegXNHxxVtXzE+i2B/9pJRUOq\nqv4eOJ/e7L3vBf4R+DqwE73ul0PPLhl9eJw+t5vQ+4PDPreS1IckWwN7VtU5XdcyFXg+144jNmks\nDMxjZJ9brWuSvAw4t6oe6LqWYZdkQ3pjrr8C2KZZfTPwNXpjrj/YVW1TTZIXV9W3uq5j2CR5EjCr\nqq4btX73qrqso7KGUpLzgTOBr42c7a+Ze+H59L45Pq+qTuukwCHSfAN/e1U90HRRfROwF/BT4JOj\nb6QeRgZmacgl+S1wH72vv74EfLOqHu62quGU5Ev0vkH6DLDyK9rZ9D44n1xVf9VVbVNNkhuravs1\n76mVkryG3ux0twPTgTdV1UXNtlU3sKk/jtg0OEmuAPapqvuTfBTYEfgP4IUAVfXmLusbBAOzNOSS\nXErvTelVwMHAbsBXgS9V1Xe7rG3YJLm2qnYe6zatXpKzHm8T8MKqesJk1jPskiwFDmyGjdwH+Czw\ngWb4Lkd1WAt+e7x2kvy0quY1yxcDe1fVI83jR41AMqw27LoASWutqurXwCeBTzYzV70GODrJ7Kra\nrtvyhsqvkrwaOHPEm/0GwKuBX3da2XB6AXAIcO+o9QH2mfxyht60qroVoKouTLI/8J9JtsPRMtZK\n093q1q7rGGI3JXlhVX2H3uRZ2wG/aOYDmBIMzNLwe9SQhlV1G3A8cHySp3VT0tA6GPgo8IkkKwPy\n5sB5zTaNzQXA/av7piPJNR3UM+zuSbLjyv7LTUvzfvS++n5mp5VpffdW4LNJPgTcDSxtvhHZnN6o\nGUPPLhnSkEuyX1Wd33UdU83KlpGqurPrWiSAJM+i9wfIz0atnw68pqq+0E1lUk+SXYGd6TXILgcu\nWvlt3bAzMEtDzqGRJoejOoyd1+ZgeT61rlofrk0nLpGGnxPrTI5PdV3AEPLaHCzPp9ZVU/7atIVZ\nGnIOjTQ4juowWF6bg+X51Lpqfbg2DczSFOLQSGunudHv8UZ1+HJVPWXyq5oavDYHy/OpddVUvTYd\nJUOaQhwaaa05qsME8docLM+n1lVT9dq0hVmSJElq4U1/ktRIkkHsI0maWgzMkvQHU/5Ob0nS2Nkl\nQ5Ia68Od3pKksTMwS9JqTNU7vSVJY2dgliRJklrYh1mSJElqYWCWJEmSWjhxiSStw5I8DFw+YtXL\nq+qGjsqRpPWSfZglaR2W5N6q2nQcx21YVQ9NRE36v+3doYqVQRgG4PdFDJsEw4JJ06JJWZsWQbwD\nwaJ4CWIUvAqRBaNXIBgVk20FWQ1bjaJdtOgYzq8cQX7YctxzfB74YfjmD198mRlmgP+NIxkAa6bt\nubav276dvitT/dpUf57kcKrdbrvf9qDtk7Yn/mnzAGtIYAY43ramsHvQ9tlU+5zkxhhjN8mtJI+W\n/t9Ncm+MsdP2wjR/dYxxKcn3LO6YBuAInGEGON6+TmF32ckkj9v+CsE7S3P7Y4wP0/h6kstJ3kwv\nem9lEbYBOAKBGWD93E/yKcnFLHYKvy3NfVkaN8nTMcaDFfYGsHEcyQBYP6eSfBxj/EhyJ4unu//m\nVZKbbbeTpO3ptmdX1CPAxhCYAdbPXpK7bd8lOZ8/V5V/G2McJnmY5EXb90leJjmzsi4BNoRr5QAA\nYIYVZgAAmCEwAwDADIEZAABmCMwAADBDYAYAgBkCMwAAzBCYAQBghsAMAAAzfgLkegd8QOq8DgAA\nAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x8b76839ef0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "t1 = X_train.groupby(['Fare'])['Fare'].count() / np.float(len(X_train))\n",
    "t2 = X_test.groupby(['Fare'])['Fare'].count() / np.float(len(X_test))\n",
    "t3 = submission.groupby(['Fare'])['Fare'].count() / np.float(len(submission))\n",
    "\n",
    "temp = pd.concat([t1,t2,t3], axis=1)\n",
    "temp.columns = ['train', 'test', 'submission']\n",
    "temp.plot.bar(figsize=(12,6))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# find quantiles and discretise train set\n",
    "X_train['Ticket_numerical'], bins = pd.qcut(x=X_train['Ticket_numerical'], q=8, retbins=True, precision=3, duplicates='raise')\n",
    "X_test['Ticket_numerical'] = pd.cut(x = X_test['Ticket_numerical'], bins=bins, include_lowest=True)\n",
    "submission['Ticket_numerical_temp'] = pd.cut(x = submission['Ticket_numerical'], bins=bins, include_lowest=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test.Ticket_numerical.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "submission.Ticket_numerical_temp.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Ticket_numerical</th>\n",
       "      <th>Ticket_numerical_temp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>186</th>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>302</th>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Ticket_numerical Ticket_numerical_temp\n",
       "186               2.0                   NaN\n",
       "302               2.0                   NaN"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "submission[submission.Ticket_numerical_temp.isnull()][['Ticket_numerical', 'Ticket_numerical_temp']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are 2 values of ticket in the submission dataset, that could not be placed within the intervals calculated for the training set. let's look at the intervals to understand why this could have been the case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(2.999, 2816.875], (14737.5, 24421.875], (364848.0, 3101317.0], (24421.875, 113053.0], (248701.0, 347077.25], (113053.0, 248701.0], (2816.875, 14737.5], (347077.25, 364848.0]]\n",
       "Categories (8, interval[float64]): [(2.999, 2816.875] < (2816.875, 14737.5] < (14737.5, 24421.875] < (24421.875, 113053.0] < (113053.0, 248701.0] < (248701.0, 347077.25] < (347077.25, 364848.0] < (364848.0, 3101317.0]]"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# inspect the ticket bins in training set\n",
    "X_train.Ticket_numerical.unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The lowest bin starts at 2.99, and the values for the tickets in the submission are 2, so slight below the interval. I will bottom code then that variable, and include those 2 values within the lowest interval. See below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Interval(2.9990000000000001, 2816.875, closed='right')"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.Ticket_numerical.unique()[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are 2 Ticket numerical parts in the submission dataset that are lower than any other in the training set. This I will include those in the lowest interval [2.99-2816]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "submission.loc[submission.Ticket_numerical_temp.isnull(), 'Ticket_numerical_temp'] = X_train.Ticket_numerical.unique()[0]\n",
    "submission.Ticket_numerical_temp.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Cabin_numerical</th>\n",
       "      <th>Cabin_categorical</th>\n",
       "      <th>Ticket_numerical</th>\n",
       "      <th>Ticket_categorical</th>\n",
       "      <th>Title</th>\n",
       "      <th>Family_size</th>\n",
       "      <th>is_mother</th>\n",
       "      <th>Age_NA</th>\n",
       "      <th>Ticket_numerical_NA</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>892</td>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>34.5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>(7.75, 7.918]</td>\n",
       "      <td>Q</td>\n",
       "      <td>158.222059</td>\n",
       "      <td>Missing</td>\n",
       "      <td>(248701.0, 347077.25]</td>\n",
       "      <td>NAN</td>\n",
       "      <td>Mr</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>893</td>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>47.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>(-0.001, 7.75]</td>\n",
       "      <td>S</td>\n",
       "      <td>158.222059</td>\n",
       "      <td>Missing</td>\n",
       "      <td>(347077.25, 364848.0]</td>\n",
       "      <td>NAN</td>\n",
       "      <td>Mrs</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>894</td>\n",
       "      <td>2</td>\n",
       "      <td>male</td>\n",
       "      <td>62.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>(7.918, 10.353]</td>\n",
       "      <td>Q</td>\n",
       "      <td>158.222059</td>\n",
       "      <td>Missing</td>\n",
       "      <td>(113053.0, 248701.0]</td>\n",
       "      <td>NAN</td>\n",
       "      <td>Mr</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>895</td>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>27.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>(7.918, 10.353]</td>\n",
       "      <td>S</td>\n",
       "      <td>158.222059</td>\n",
       "      <td>Missing</td>\n",
       "      <td>(248701.0, 347077.25]</td>\n",
       "      <td>NAN</td>\n",
       "      <td>Mr</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>896</td>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>(10.353, 14.456]</td>\n",
       "      <td>S</td>\n",
       "      <td>158.222059</td>\n",
       "      <td>Missing</td>\n",
       "      <td>(364848.0, 3101317.0]</td>\n",
       "      <td>NAN</td>\n",
       "      <td>Mrs</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Pclass     Sex   Age  SibSp  Parch              Fare Embarked  \\\n",
       "0          892       3    male  34.5      0      0     (7.75, 7.918]        Q   \n",
       "1          893       3  female  47.0      1      0    (-0.001, 7.75]        S   \n",
       "2          894       2    male  62.0      0      0   (7.918, 10.353]        Q   \n",
       "3          895       3    male  27.0      0      0   (7.918, 10.353]        S   \n",
       "4          896       3  female  22.0      1      1  (10.353, 14.456]        S   \n",
       "\n",
       "   Cabin_numerical Cabin_categorical       Ticket_numerical  \\\n",
       "0       158.222059           Missing  (248701.0, 347077.25]   \n",
       "1       158.222059           Missing  (347077.25, 364848.0]   \n",
       "2       158.222059           Missing   (113053.0, 248701.0]   \n",
       "3       158.222059           Missing  (248701.0, 347077.25]   \n",
       "4       158.222059           Missing  (364848.0, 3101317.0]   \n",
       "\n",
       "  Ticket_categorical Title  Family_size  is_mother  Age_NA  \\\n",
       "0                NAN    Mr            1          0       0   \n",
       "1                NAN   Mrs            2          0       0   \n",
       "2                NAN    Mr            1          0       0   \n",
       "3                NAN    Mr            1          0       0   \n",
       "4                NAN   Mrs            3          1       0   \n",
       "\n",
       "   Ticket_numerical_NA  \n",
       "0                    0  \n",
       "1                    0  \n",
       "2                    0  \n",
       "3                    0  \n",
       "4                    0  "
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "submission['Ticket_numerical'] = submission['Ticket_numerical_temp']\n",
    "submission.drop(labels=['Ticket_numerical_temp'], inplace=True, axis=1)\n",
    "submission.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Engineering rare labels in categorical variables (section 9)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sex male      0.65309\n",
      "female    0.34691\n",
      "Name: Sex, dtype: float64\n",
      "\n",
      "Embarked S    0.728933\n",
      "C    0.181180\n",
      "Q    0.089888\n",
      "Name: Embarked, dtype: float64\n",
      "\n",
      "Cabin_categorical Missing    0.771067\n",
      "C          0.061798\n",
      "B          0.050562\n",
      "E          0.040730\n",
      "D          0.036517\n",
      "A          0.021067\n",
      "F          0.012640\n",
      "G          0.004213\n",
      "T          0.001404\n",
      "Name: Cabin_categorical, dtype: float64\n",
      "\n",
      "Ticket_categorical NAN        0.728933\n",
      "PC         0.066011\n",
      "CA         0.050562\n",
      "A          0.035112\n",
      "STONO      0.023876\n",
      "SOTONOQ    0.016854\n",
      "SCPARIS    0.014045\n",
      "WC         0.014045\n",
      "SOC        0.007022\n",
      "C          0.005618\n",
      "LINE       0.005618\n",
      "PP         0.004213\n",
      "WEP        0.004213\n",
      "PPP        0.002809\n",
      "SOPP       0.002809\n",
      "FCC        0.002809\n",
      "SCAH       0.002809\n",
      "SOTONO     0.002809\n",
      "AS         0.001404\n",
      "SCOW       0.001404\n",
      "FA         0.001404\n",
      "FC         0.001404\n",
      "SCA        0.001404\n",
      "SP         0.001404\n",
      "SWPP       0.001404\n",
      "Name: Ticket_categorical, dtype: float64\n",
      "\n",
      "Title Mr        0.589888\n",
      "Miss      0.200843\n",
      "Mrs       0.140449\n",
      "Master    0.049157\n",
      "Other     0.019663\n",
      "Name: Title, dtype: float64\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# find unfrequent labels in categorical variables\n",
    "for var in categorical:\n",
    "    print(var, X_train[var].value_counts()/np.float(len(X_train)))\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Cabin contains the rare labels G and T: replace by most frequent category\n",
    "- Ticket contains a lot of infrequent labels: replace by rare\n",
    "\n",
    "Because the number of passengers in the rare cabins is so small, grouping them into a new category called rare, will be in itself rare, and may be prone to over-fitting. Thus, in cabin, I will replace rare labels by the most frequent category.\n",
    "In ticket_categorical, on the other hand, the number of infrequent labels is high, therefore grouping them into a new label makes sense."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def rare_imputation(variable, which='rare'):    \n",
    "    # find frequent labels\n",
    "    temp = X_train.groupby([variable])[variable].count()/np.float(len(X_train))\n",
    "    frequent_cat = [x for x in temp.loc[temp>0.01].index.values]\n",
    "    \n",
    "    # create new variables, with Rare labels imputed\n",
    "    if which=='frequent':\n",
    "        # find the most frequent category\n",
    "        mode_label = X_train.groupby(variable)[variable].count().sort_values().tail(1).index.values[0]\n",
    "        X_train[variable] = np.where(X_train[variable].isin(frequent_cat), X_train[variable], mode_label)\n",
    "        X_test[variable] = np.where(X_test[variable].isin(frequent_cat), X_test[variable], mode_label)\n",
    "        submission[variable] = np.where(submission[variable].isin(frequent_cat), submission[variable], mode_label)\n",
    "    \n",
    "    else:\n",
    "        X_train[variable] = np.where(X_train[variable].isin(frequent_cat), X_train[variable], 'Rare')\n",
    "        X_test[variable] = np.where(X_test[variable].isin(frequent_cat), X_test[variable], 'Rare')\n",
    "        submission[variable] = np.where(submission[variable].isin(frequent_cat), submission[variable], 'Rare')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "rare_imputation('Cabin_categorical', 'frequent')\n",
    "rare_imputation('Ticket_categorical', 'rare')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sex male      0.65309\n",
      "female    0.34691\n",
      "Name: Sex, dtype: float64\n",
      "\n",
      "Embarked S    0.728933\n",
      "C    0.181180\n",
      "Q    0.089888\n",
      "Name: Embarked, dtype: float64\n",
      "\n",
      "Cabin_categorical Missing    0.776685\n",
      "C          0.061798\n",
      "B          0.050562\n",
      "E          0.040730\n",
      "D          0.036517\n",
      "A          0.021067\n",
      "F          0.012640\n",
      "Name: Cabin_categorical, dtype: float64\n",
      "\n",
      "Ticket_categorical NAN        0.728933\n",
      "PC         0.066011\n",
      "Rare       0.050562\n",
      "CA         0.050562\n",
      "A          0.035112\n",
      "STONO      0.023876\n",
      "SOTONOQ    0.016854\n",
      "WC         0.014045\n",
      "SCPARIS    0.014045\n",
      "Name: Ticket_categorical, dtype: float64\n",
      "\n",
      "Title Mr        0.589888\n",
      "Miss      0.200843\n",
      "Mrs       0.140449\n",
      "Master    0.049157\n",
      "Other     0.019663\n",
      "Name: Title, dtype: float64\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# let's check that it worked\n",
    "for var in categorical:\n",
    "    print(var, X_train[var].value_counts()/np.float(len(X_train)))\n",
    "    print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sex male      0.636364\n",
      "female    0.363636\n",
      "Name: Sex, dtype: float64\n",
      "\n",
      "Embarked S    0.645933\n",
      "C    0.244019\n",
      "Q    0.110048\n",
      "Name: Embarked, dtype: float64\n",
      "\n",
      "Cabin_categorical Missing    0.784689\n",
      "C          0.083732\n",
      "B          0.043062\n",
      "D          0.031100\n",
      "E          0.021531\n",
      "F          0.019139\n",
      "A          0.016746\n",
      "Name: Cabin_categorical, dtype: float64\n",
      "\n",
      "Ticket_categorical NAN        0.708134\n",
      "PC         0.076555\n",
      "Rare       0.064593\n",
      "CA         0.064593\n",
      "A          0.026316\n",
      "SOTONOQ    0.021531\n",
      "SCPARIS    0.019139\n",
      "WC         0.011962\n",
      "STONO      0.007177\n",
      "Name: Ticket_categorical, dtype: float64\n",
      "\n",
      "Title Mr        0.574163\n",
      "Miss      0.186603\n",
      "Mrs       0.172249\n",
      "Master    0.050239\n",
      "Other     0.016746\n",
      "Name: Title, dtype: float64\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# let's check that it worked\n",
    "for var in categorical:\n",
    "    print(var, submission[var].value_counts()/np.float(len(submission)))\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Encode categorical variables (section 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Sex', 'Embarked', 'Cabin_categorical', 'Ticket_categorical', 'Title']"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "categorical"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Sex: one hot encoding\n",
    "- Remaining variables: replace by risk probability"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "for df in [X_train, X_test, submission]:\n",
    "    df['Sex']  = pd.get_dummies(df.Sex, drop_first=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1], dtype=uint64)"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.Sex.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 0], dtype=uint64)"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test.Sex.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 0], dtype=uint64)"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "submission.Sex.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def encode_categorical_variables(var, target):\n",
    "        # make label to risk dictionary\n",
    "        ordered_labels = X_train.groupby([var])[target].mean().to_dict()\n",
    "        \n",
    "        # encode variables\n",
    "        X_train[var] = X_train[var].map(ordered_labels)\n",
    "        X_test[var] = X_test[var].map(ordered_labels)\n",
    "        submission[var] = submission[var].map(ordered_labels)\n",
    "\n",
    "# enccode labels in categorical vars\n",
    "for var in categorical:\n",
    "    encode_categorical_variables(var, 'Survived')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# parse discretised variables to object before encoding\n",
    "for df in [X_train, X_test, submission]:\n",
    "    df.Fare = df.Fare.astype('O')\n",
    "    df.Ticket_numerical = df.Ticket_numerical.astype('O')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fare\n",
      "Ticket_numerical\n"
     ]
    }
   ],
   "source": [
    "# encode labels\n",
    "for var in ['Fare', 'Ticket_numerical']:\n",
    "    print(var)\n",
    "    encode_categorical_variables(var, 'Survived')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Cabin_numerical</th>\n",
       "      <th>Cabin_categorical</th>\n",
       "      <th>Ticket_numerical</th>\n",
       "      <th>Ticket_categorical</th>\n",
       "      <th>Title</th>\n",
       "      <th>Family_size</th>\n",
       "      <th>is_mother</th>\n",
       "      <th>Age_NA</th>\n",
       "      <th>Ticket_numerical_NA</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>140</th>\n",
       "      <td>141</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0.744939</td>\n",
       "      <td>27.0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.397849</td>\n",
       "      <td>0.527132</td>\n",
       "      <td>158.222059</td>\n",
       "      <td>0.307414</td>\n",
       "      <td>0.348315</td>\n",
       "      <td>0.394990</td>\n",
       "      <td>0.790000</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>439</th>\n",
       "      <td>440</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.191398</td>\n",
       "      <td>31.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.404494</td>\n",
       "      <td>0.342967</td>\n",
       "      <td>158.222059</td>\n",
       "      <td>0.307414</td>\n",
       "      <td>0.573034</td>\n",
       "      <td>0.305556</td>\n",
       "      <td>0.157143</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>817</th>\n",
       "      <td>818</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.191398</td>\n",
       "      <td>31.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.494118</td>\n",
       "      <td>0.527132</td>\n",
       "      <td>158.222059</td>\n",
       "      <td>0.307414</td>\n",
       "      <td>0.348315</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.157143</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>378</th>\n",
       "      <td>379</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0.191398</td>\n",
       "      <td>20.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.223214</td>\n",
       "      <td>0.527132</td>\n",
       "      <td>158.222059</td>\n",
       "      <td>0.307414</td>\n",
       "      <td>0.348315</td>\n",
       "      <td>0.394990</td>\n",
       "      <td>0.157143</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>491</th>\n",
       "      <td>492</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0.191398</td>\n",
       "      <td>21.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.223214</td>\n",
       "      <td>0.342967</td>\n",
       "      <td>158.222059</td>\n",
       "      <td>0.307414</td>\n",
       "      <td>0.306818</td>\n",
       "      <td>0.166667</td>\n",
       "      <td>0.157143</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Survived  Pclass       Sex   Age  SibSp  Parch      Fare  \\\n",
       "140          141         0       3  0.744939  27.0      0      2  0.397849   \n",
       "439          440         0       2  0.191398  31.0      0      0  0.404494   \n",
       "817          818         0       2  0.191398  31.0      1      1  0.494118   \n",
       "378          379         0       3  0.191398  20.0      0      0  0.223214   \n",
       "491          492         0       3  0.191398  21.0      0      0  0.223214   \n",
       "\n",
       "     Embarked  Cabin_numerical  Cabin_categorical  Ticket_numerical  \\\n",
       "140  0.527132       158.222059           0.307414          0.348315   \n",
       "439  0.342967       158.222059           0.307414          0.573034   \n",
       "817  0.527132       158.222059           0.307414          0.348315   \n",
       "378  0.527132       158.222059           0.307414          0.348315   \n",
       "491  0.342967       158.222059           0.307414          0.306818   \n",
       "\n",
       "     Ticket_categorical     Title  Family_size  is_mother  Age_NA  \\\n",
       "140            0.394990  0.790000            3          0       1   \n",
       "439            0.305556  0.157143            1          0       0   \n",
       "817            0.500000  0.157143            3          0       0   \n",
       "378            0.394990  0.157143            1          0       0   \n",
       "491            0.166667  0.157143            1          0       0   \n",
       "\n",
       "     Ticket_numerical_NA  \n",
       "140                    0  \n",
       "439                    0  \n",
       "817                    0  \n",
       "378                    0  \n",
       "491                    0  "
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#let's inspect the dataset\n",
    "X_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the labels how now been replaced by the survival probability.\n",
    "\n",
    "### Feature scaling (section 13)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Cabin_numerical</th>\n",
       "      <th>Cabin_categorical</th>\n",
       "      <th>Ticket_numerical</th>\n",
       "      <th>Ticket_categorical</th>\n",
       "      <th>Title</th>\n",
       "      <th>Family_size</th>\n",
       "      <th>is_mother</th>\n",
       "      <th>Age_NA</th>\n",
       "      <th>Ticket_numerical_NA</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "      <td>712.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>443.283708</td>\n",
       "      <td>0.383427</td>\n",
       "      <td>2.317416</td>\n",
       "      <td>0.383427</td>\n",
       "      <td>29.807584</td>\n",
       "      <td>0.484551</td>\n",
       "      <td>0.353933</td>\n",
       "      <td>0.383427</td>\n",
       "      <td>0.383427</td>\n",
       "      <td>133.956178</td>\n",
       "      <td>0.383427</td>\n",
       "      <td>0.383427</td>\n",
       "      <td>0.383427</td>\n",
       "      <td>0.383427</td>\n",
       "      <td>1.875000</td>\n",
       "      <td>0.067416</td>\n",
       "      <td>0.198034</td>\n",
       "      <td>0.005618</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>260.003783</td>\n",
       "      <td>0.486563</td>\n",
       "      <td>0.833767</td>\n",
       "      <td>0.263664</td>\n",
       "      <td>14.496489</td>\n",
       "      <td>0.907718</td>\n",
       "      <td>0.673811</td>\n",
       "      <td>0.144616</td>\n",
       "      <td>0.071238</td>\n",
       "      <td>48.398112</td>\n",
       "      <td>0.149408</td>\n",
       "      <td>0.118238</td>\n",
       "      <td>0.098434</td>\n",
       "      <td>0.276532</td>\n",
       "      <td>1.460148</td>\n",
       "      <td>0.250917</td>\n",
       "      <td>0.398798</td>\n",
       "      <td>0.074795</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.191398</td>\n",
       "      <td>0.670000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.212121</td>\n",
       "      <td>0.342967</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.307414</td>\n",
       "      <td>0.166667</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.157143</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>211.750000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.191398</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.232770</td>\n",
       "      <td>0.342967</td>\n",
       "      <td>158.222059</td>\n",
       "      <td>0.307414</td>\n",
       "      <td>0.321087</td>\n",
       "      <td>0.394990</td>\n",
       "      <td>0.157143</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>441.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>0.191398</td>\n",
       "      <td>28.750000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.397849</td>\n",
       "      <td>0.342967</td>\n",
       "      <td>158.222059</td>\n",
       "      <td>0.307414</td>\n",
       "      <td>0.370787</td>\n",
       "      <td>0.394990</td>\n",
       "      <td>0.157143</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>671.250000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>0.744939</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.444444</td>\n",
       "      <td>0.421875</td>\n",
       "      <td>158.222059</td>\n",
       "      <td>0.307414</td>\n",
       "      <td>0.474719</td>\n",
       "      <td>0.394990</td>\n",
       "      <td>0.706294</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>890.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>0.744939</td>\n",
       "      <td>73.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.659091</td>\n",
       "      <td>0.527132</td>\n",
       "      <td>158.222059</td>\n",
       "      <td>0.758621</td>\n",
       "      <td>0.573034</td>\n",
       "      <td>0.617021</td>\n",
       "      <td>0.790000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       PassengerId    Survived      Pclass         Sex         Age  \\\n",
       "count   712.000000  712.000000  712.000000  712.000000  712.000000   \n",
       "mean    443.283708    0.383427    2.317416    0.383427   29.807584   \n",
       "std     260.003783    0.486563    0.833767    0.263664   14.496489   \n",
       "min       1.000000    0.000000    1.000000    0.191398    0.670000   \n",
       "25%     211.750000    0.000000    2.000000    0.191398   21.000000   \n",
       "50%     441.000000    0.000000    3.000000    0.191398   28.750000   \n",
       "75%     671.250000    1.000000    3.000000    0.744939   38.000000   \n",
       "max     890.000000    1.000000    3.000000    0.744939   73.000000   \n",
       "\n",
       "            SibSp       Parch        Fare    Embarked  Cabin_numerical  \\\n",
       "count  712.000000  712.000000  712.000000  712.000000       712.000000   \n",
       "mean     0.484551    0.353933    0.383427    0.383427       133.956178   \n",
       "std      0.907718    0.673811    0.144616    0.071238        48.398112   \n",
       "min      0.000000    0.000000    0.212121    0.342967         2.000000   \n",
       "25%      0.000000    0.000000    0.232770    0.342967       158.222059   \n",
       "50%      0.000000    0.000000    0.397849    0.342967       158.222059   \n",
       "75%      1.000000    0.000000    0.444444    0.421875       158.222059   \n",
       "max      4.000000    2.000000    0.659091    0.527132       158.222059   \n",
       "\n",
       "       Cabin_categorical  Ticket_numerical  Ticket_categorical       Title  \\\n",
       "count         712.000000        712.000000          712.000000  712.000000   \n",
       "mean            0.383427          0.383427            0.383427    0.383427   \n",
       "std             0.149408          0.118238            0.098434    0.276532   \n",
       "min             0.307414          0.166667            0.080000    0.157143   \n",
       "25%             0.307414          0.321087            0.394990    0.157143   \n",
       "50%             0.307414          0.370787            0.394990    0.157143   \n",
       "75%             0.307414          0.474719            0.394990    0.706294   \n",
       "max             0.758621          0.573034            0.617021    0.790000   \n",
       "\n",
       "       Family_size   is_mother      Age_NA  Ticket_numerical_NA  \n",
       "count   712.000000  712.000000  712.000000           712.000000  \n",
       "mean      1.875000    0.067416    0.198034             0.005618  \n",
       "std       1.460148    0.250917    0.398798             0.074795  \n",
       "min       1.000000    0.000000    0.000000             0.000000  \n",
       "25%       1.000000    0.000000    0.000000             0.000000  \n",
       "50%       1.000000    0.000000    0.000000             0.000000  \n",
       "75%       2.000000    0.000000    0.000000             0.000000  \n",
       "max       7.000000    1.000000    1.000000             1.000000  "
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "variables_that_need_scaling = ['Pclass', 'Age', 'Sibsp', 'Parch', 'Cabin_numerical', 'Family_size']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Pclass',\n",
       " 'Sex',\n",
       " 'Age',\n",
       " 'SibSp',\n",
       " 'Parch',\n",
       " 'Fare',\n",
       " 'Embarked',\n",
       " 'Cabin_numerical',\n",
       " 'Cabin_categorical',\n",
       " 'Ticket_numerical',\n",
       " 'Ticket_categorical',\n",
       " 'Title',\n",
       " 'Family_size',\n",
       " 'is_mother',\n",
       " 'Age_NA',\n",
       " 'Ticket_numerical_NA']"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "training_vars = [var for var in X_train.columns if var not in ['PassengerId', 'Survived']]\n",
    "training_vars"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MinMaxScaler(copy=True, feature_range=(0, 1))"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# fit scaler\n",
    "scaler = MinMaxScaler() # create an instance\n",
    "scaler.fit(X_train[training_vars]) #  fit  the scaler to the train set and then transform it"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The scaler is now ready, we can use it in a machine learning algorithm when required. See below.\n",
    "\n",
    "### Machine Learning algorithm building\n",
    "\n",
    "#### xgboost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xgb train roc-auc: 0.9388929218086394\n",
      "xgb test roc-auc: 0.9077733860342556\n"
     ]
    }
   ],
   "source": [
    "xgb_model = xgb.XGBClassifier()\n",
    "\n",
    "eval_set = [(X_test[training_vars], y_test)]\n",
    "xgb_model.fit(X_train[training_vars], y_train, eval_metric=\"auc\", eval_set=eval_set, verbose=False)\n",
    "\n",
    "pred = xgb_model.predict_proba(X_train[training_vars])\n",
    "print('xgb train roc-auc: {}'.format(roc_auc_score(y_train, pred[:,1])))\n",
    "pred = xgb_model.predict_proba(X_test[training_vars])\n",
    "print('xgb test roc-auc: {}'.format(roc_auc_score(y_test, pred[:,1])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Random Forests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RF train roc-auc: 0.9982310779577295\n",
      "RF test roc-auc: 0.8945981554677207\n"
     ]
    }
   ],
   "source": [
    "rf_model = RandomForestClassifier()\n",
    "rf_model.fit(X_train[training_vars], y_train)\n",
    "\n",
    "pred = rf_model.predict_proba(X_train[training_vars])\n",
    "print('RF train roc-auc: {}'.format(roc_auc_score(y_train, pred[:,1])))\n",
    "pred = rf_model.predict_proba(X_test[training_vars])\n",
    "print('RF test roc-auc: {}'.format(roc_auc_score(y_test, pred[:,1])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Adaboost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Adaboost train roc-auc: 0.8973441137450249\n",
      "Adaboost test roc-auc: 0.8706192358366273\n"
     ]
    }
   ],
   "source": [
    "ada_model = AdaBoostClassifier()\n",
    "ada_model.fit(X_train[training_vars], y_train)\n",
    "\n",
    "pred = ada_model.predict_proba(X_train[training_vars])\n",
    "print('Adaboost train roc-auc: {}'.format(roc_auc_score(y_train, pred[:,1])))\n",
    "pred = ada_model.predict_proba(X_test[training_vars])\n",
    "print('Adaboost test roc-auc: {}'.format(roc_auc_score(y_test, pred[:,1])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Logistic regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Logit train roc-auc: 0.8724915934483133\n",
      "Logit test roc-auc: 0.8268774703557312\n"
     ]
    }
   ],
   "source": [
    "logit_model = LogisticRegression()\n",
    "logit_model.fit(scaler.transform(X_train[training_vars]), y_train)\n",
    "\n",
    "pred = logit_model.predict_proba(scaler.transform(X_train[training_vars]))\n",
    "print('Logit train roc-auc: {}'.format(roc_auc_score(y_train, pred[:,1])))\n",
    "pred = ada_model.predict_proba(scaler.transform(X_test[training_vars]))\n",
    "print('Logit test roc-auc: {}'.format(roc_auc_score(y_test, pred[:,1])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Select threshold for maximum accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ensemble test roc-auc: 0.9106719367588931\n"
     ]
    }
   ],
   "source": [
    "pred_ls = []\n",
    "for model in [xgb_model, rf_model, ada_model, logit_model]:\n",
    "    pred_ls.append(pd.Series(model.predict_proba(X_test[training_vars])[:,1]))\n",
    "\n",
    "final_pred = pd.concat(pred_ls, axis=1).mean(axis=1)\n",
    "print('Ensemble test roc-auc: {}'.format(roc_auc_score(y_test,final_pred)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.62887334,  0.57844404,  0.57404189,  0.56136034,  0.5611278 ,\n",
       "        0.52896941,  0.52819212,  0.51329908,  0.51176485,  0.46151215,\n",
       "        0.42590368,  0.4157962 ,  0.40317914,  0.38411698,  0.37637827,\n",
       "        0.3716263 ,  0.36255445,  0.35742418,  0.35366244,  0.34411429,\n",
       "        0.33705063,  0.33621705,  0.32975272,  0.3228687 ,  0.32075407,\n",
       "        0.32020933,  0.30779934,  0.30114494,  0.28647033,  0.27682098,\n",
       "        0.25926278,  0.24921405,  0.23840703,  0.23403145,  0.22316067,\n",
       "        0.22203072,  0.21961142,  0.21952678,  0.18889223,  0.1883331 ,\n",
       "        0.18169372,  0.1797722 ,  0.17787358,  0.15210567,  0.15075582,\n",
       "        0.14506151,  0.14492661,  0.14479344,  0.14448713,  0.14272373,\n",
       "        0.14182324,  0.14054127,  0.13657478,  0.13573485,  0.1332098 ,\n",
       "        0.13034344,  0.12988231])"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tpr, tpr, thresholds = metrics.roc_curve(y_test, final_pred)\n",
    "thresholds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>thresholds</th>\n",
       "      <th>accuracy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.461512</td>\n",
       "      <td>0.843575</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>0.371626</td>\n",
       "      <td>0.843575</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>0.376378</td>\n",
       "      <td>0.843575</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0.384117</td>\n",
       "      <td>0.837989</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>0.337051</td>\n",
       "      <td>0.837989</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    thresholds  accuracy\n",
       "9     0.461512  0.843575\n",
       "15    0.371626  0.843575\n",
       "14    0.376378  0.843575\n",
       "13    0.384117  0.837989\n",
       "20    0.337051  0.837989"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "accuracy_ls = []\n",
    "for thres in thresholds:\n",
    "    y_pred = np.where(final_pred>thres,1,0)\n",
    "    accuracy_ls.append(metrics.accuracy_score(y_test, y_pred, normalize=True))\n",
    "    \n",
    "accuracy_ls = pd.concat([pd.Series(thresholds), pd.Series(accuracy_ls)],\n",
    "                        axis=1)\n",
    "accuracy_ls.columns = ['thresholds', 'accuracy']\n",
    "accuracy_ls.sort_values(by='accuracy', ascending=False, inplace=True)\n",
    "accuracy_ls.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Submission to Kaggle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "pred_ls = []\n",
    "for model in [xgb_model, rf_model, ada_model, logit_model]:\n",
    "    pred_ls.append(pd.Series(model.predict_proba(submission[training_vars])[:,1]))\n",
    "\n",
    "final_pred = pd.concat(pred_ls, axis=1).mean(axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "final_pred = pd.Series(np.where(final_pred>0.40,1,0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>892</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>893</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>894</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>895</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>896</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived\n",
       "0          892         0\n",
       "1          893         0\n",
       "2          894         0\n",
       "3          895         0\n",
       "4          896         0"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "temp = pd.concat([submission.PassengerId, final_pred], axis=1)\n",
    "temp.columns = ['PassengerId', 'Survived']\n",
    "temp.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "temp.to_csv('submit_titanic.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature importance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x8b76bbeb70>"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtMAAAHHCAYAAAB9W9QhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XmYZWV19/3vj0Y0oghqi4RZ02JwAEmL+uhjRGMCGG30\nCQoqEqMBoijEkIiaRIOJEuMQNAaCEQNOqK+iHcUBiRMqsRskICqxgxBAJkdwZFrvH3sXnK6u6jq9\nqa69T9f3c13nqrPHWqeqzqm1733f605VIUmSJGnDbdZ3AJIkSdKkMpmWJEmSOjKZliRJkjoymZYk\nSZI6MpmWJEmSOjKZliRJkjoymZYkSZI6MpmWJEmSOjKZliRJkjravO8ANsR973vf2mWXXfoOQ5Ik\nSZu488477/tVtXSu/SYqmd5ll11YvXp132FIkiRpE5fk8nH2s5uHJEmS1JHJtCRJktSRybQkSZLU\nkcm0JEmS1JHJtCRJktSRybQkSZLUkcm0JEmS1JHJtCRJktSRybQkSZLUkcm0JEmS1JHJtCRJktSR\nybQkSZLUkcm0JEmS1JHJtCRJktTR5uPslGRf4ARgCfCvVXX8tO3PAV4OBLgR+JOq+q/1HZvk3sAH\ngF2Ay4BnVtWP7vxLgl2O/cR8nGYtlx3/lHk/pyRJkibbnC3TSZYAbwf2A3YHDk6y+7Tdvgv8dlU9\nDHgtcPIYxx4LnF1Vy4Cz22VJkiRpYozTzWNvYE1VXVpVNwGnAytGd6iqr4y0Kp8L7DDGsSuAU9vn\npwIHdH8ZkiRJ0sIbJ5neHrhiZPnKdt1sXgB8coxjt62qq9vn1wDbznSyJIclWZ1k9fXXXz9GuJIk\nSdLCmNcBiEn2oUmmX74hx1VVATXLtpOranlVLV+6dOk8RClJkiTNj3GS6auAHUeWd2jXrSXJw4F/\nBVZU1Q/GOPbaJNu1x24HXLdhoUuSJEn9GieZXgUsS7Jrki2Ag4CVozsk2Qn4CHBIVf33mMeuBA5t\nnx8KfKz7y5AkSZIW3pyl8arqliRHAp+mKW93SlVdnOSIdvtJwF8D9wH+OQnALW3XjBmPbU99PPDB\nJC8ALgeeOc+vTZIkSdqoxqozXVVnAmdOW3fSyPMXAi8c99h2/Q+AJ21IsJIkSdKQOAOiJEmS1JHJ\ntCRJktSRybQkSZLUkcm0JEmS1NFYAxC1cexy7Cfm/ZyXHf+UeT+nJEmSZmbLtCRJktSRybQkSZLU\nkcm0JEmS1JHJtCRJktSRybQkSZLUkcm0JEmS1JHJtCRJktSRybQkSZLUkcm0JEmS1JHJtCRJktSR\nybQkSZLUkcm0JEmS1JHJtCRJktSRybQkSZLUkcm0JEmS1JHJtCRJktSRybQkSZLUkcm0JEmS1JHJ\ntCRJktSRybQkSZLUkcm0JEmS1JHJtCRJktSRybQkSZLUkcm0JEmS1NFYyXSSfZNckmRNkmNn2P7g\nJF9N8qskx4ys3y3JBSOPG5Ic3W57TZKrRrbtP38vS5IkSdr4Np9rhyRLgLcDTwauBFYlWVlV3xzZ\n7YfAS4EDRo+tqkuAPUfOcxVwxsgub6mqN96pVyBJkiT1ZJyW6b2BNVV1aVXdBJwOrBjdoaquq6pV\nwM3rOc+TgP+pqss7RytJkiQNyDjJ9PbAFSPLV7brNtRBwPunrXtJkguTnJJkmw7nlCRJknqzIAMQ\nk2wBPA340MjqE4EH0HQDuRp40yzHHpZkdZLV119//UaPVZIkSRrXOMn0VcCOI8s7tOs2xH7A+VV1\n7dSKqrq2qm6tqtuAd9B0J1lHVZ1cVcuravnSpUs38NtKkiRJG884yfQqYFmSXdsW5oOAlRv4fQ5m\nWhePJNuNLD4d+MYGnlOSJEnq1ZzVPKrqliRHAp8GlgCnVNXFSY5ot5+U5P7AamAr4La2/N3uVXVD\nki1pKoEcPu3Ub0iyJ1DAZTNslyRJkgZtzmQaoKrOBM6ctu6kkefX0HT/mOnYnwH3mWH9IRsUqSRJ\nkjQwzoAoSZIkdWQyLUmSJHVkMi1JkiR1ZDItSZIkdWQyLUmSJHVkMi1JkiR1ZDItSZIkdWQyLUmS\nJHVkMi1JkiR1ZDItSZIkdWQyLUmSJHVkMi1JkiR1ZDItSZIkdWQyLUmSJHVkMi1JkiR1ZDItSZIk\ndWQyLUmSJHVkMi1JkiR1ZDItSZIkdWQyLUmSJHVkMi1JkiR1ZDItSZIkdWQyLUmSJHVkMi1JkiR1\nZDItSZIkdWQyLUmSJHVkMi1JkiR1ZDItSZIkdWQyLUmSJHVkMi1JkiR1NFYynWTfJJckWZPk2Bm2\nPzjJV5P8Kskx07ZdluSiJBckWT2y/t5JzkrynfbrNnf+5UiSJEkLZ85kOskS4O3AfsDuwMFJdp+2\n2w+BlwJvnOU0+1TVnlW1fGTdscDZVbUMOLtdliRJkibGOC3TewNrqurSqroJOB1YMbpDVV1XVauA\nmzfge68ATm2fnwocsAHHSpIkSb0bJ5neHrhiZPnKdt24CvhskvOSHDayftuqurp9fg2w7UwHJzks\nyeokq6+//voN+LaSJEnSxrUQAxAfV1V70nQTeXGSx0/foaqKJuleR1WdXFXLq2r50qVLN3KokiRJ\n0vjGSaavAnYcWd6hXTeWqrqq/XodcAZNtxGAa5NsB9B+vW7cc0qSJElDME4yvQpYlmTXJFsABwEr\nxzl5ki2T3HPqOfC7wDfazSuBQ9vnhwIf25DAJUmSpL5tPtcOVXVLkiOBTwNLgFOq6uIkR7TbT0py\nf2A1sBVwW5KjaSp/3Bc4I8nU93pfVX2qPfXxwAeTvAC4HHjm/L40SZIkaeOaM5kGqKozgTOnrTtp\n5Pk1NN0/prsB2GOWc/4AeNLYkUqSJEkD4wyIkiRJUkcm05IkSVJHJtOSJElSRybTkiRJUkcm05Ik\nSVJHJtOSJElSRybTkiRJUkcm05IkSVJHJtOSJElSRybTkiRJUkcm05IkSVJHJtOSJElSRybTkiRJ\nUkcm05IkSVJHJtOSJElSRybTkiRJUkcm05IkSVJHJtOSJElSRybTkiRJUkcm05IkSVJHJtOSJElS\nRybTkiRJUkcm05IkSVJHJtOSJElSRybTkiRJUkcm05IkSVJHJtOSJElSRybTkiRJUkcm05IkSVJH\nYyXTSfZNckmSNUmOnWH7g5N8Ncmvkhwzsn7HJJ9L8s0kFyc5amTba5JcleSC9rH//LwkSZIkaWFs\nPtcOSZYAbweeDFwJrEqysqq+ObLbD4GXAgdMO/wW4M+q6vwk9wTOS3LWyLFvqao33ulXIUmSJPVg\nnJbpvYE1VXVpVd0EnA6sGN2hqq6rqlXAzdPWX11V57fPbwS+BWw/L5FLkiRJPRsnmd4euGJk+Uo6\nJMRJdgEeAfznyOqXJLkwySlJtpnluMOSrE6y+vrrr9/QbytJkiRtNAsyADHJPYAPA0dX1Q3t6hOB\nBwB7AlcDb5rp2Ko6uaqWV9XypUuXLkS4kiRJ0ljGSaavAnYcWd6hXTeWJHehSaTfW1UfmVpfVddW\n1a1VdRvwDpruJJIkSdLEGCeZXgUsS7Jrki2Ag4CV45w8SYB3At+qqjdP27bdyOLTgW+MF7IkSZI0\nDHNW86iqW5IcCXwaWAKcUlUXJzmi3X5SkvsDq4GtgNuSHA3sDjwcOAS4KMkF7SlfWVVnAm9IsidQ\nwGXA4fP70iRJkqSNa85kGqBNfs+ctu6kkefX0HT/mO4cILOc85Dxw5QkSZKGxxkQJUmSpI5MpiVJ\nkqSOTKYlSZKkjkymJUmSpI5MpiVJkqSOTKYlSZKkjkymJUmSpI5MpiVJkqSOTKYlSZKkjkymJUmS\npI5MpiVJkqSOTKYlSZKkjkymJUmSpI5MpiVJkqSOTKYlSZKkjkymJUmSpI5MpiVJkqSOTKYlSZKk\njkymJUmSpI5MpiVJkqSOTKYlSZKkjjbvOwAN3y7HfmLez3nZ8U+Z93NKkiQtNFumJUmSpI5MpiVJ\nkqSOTKYlSZKkjkymJUmSpI5MpiVJkqSOTKYlSZKkjkymJUmSpI7GSqaT7JvkkiRrkhw7w/YHJ/lq\nkl8lOWacY5PcO8lZSb7Tft3mzr8cSZIkaeHMmUwnWQK8HdgP2B04OMnu03b7IfBS4I0bcOyxwNlV\ntQw4u12WJEmSJsY4LdN7A2uq6tKqugk4HVgxukNVXVdVq4CbN+DYFcCp7fNTgQM6vgZJkiSpF+Mk\n09sDV4wsX9muG8f6jt22qq5un18DbDvmOSVJkqRBGMQAxKoqoGbaluSwJKuTrL7++usXODJJkiRp\nduMk01cBO44s79CuG8f6jr02yXYA7dfrZjpBVZ1cVcuravnSpUvH/LaSJEnSxjdOMr0KWJZk1yRb\nAAcBK8c8//qOXQkc2j4/FPjY+GFLkiRJ/dt8rh2q6pYkRwKfBpYAp1TVxUmOaLeflOT+wGpgK+C2\nJEcDu1fVDTMd2576eOCDSV4AXA48c75fnCRJkrQxzZlMA1TVmcCZ09adNPL8GpouHGMd267/AfCk\nDQlWkiRJGpJBDECUJEmSJpHJtCRJktSRybQkSZLUkcm0JEmS1JHJtCRJktSRybQkSZLUkcm0JEmS\n1JHJtCRJktSRybQkSZLUkcm0JEmS1JHJtCRJktSRybQkSZLUkcm0JEmS1JHJtCRJktSRybQkSZLU\nkcm0JEmS1JHJtCRJktSRybQkSZLUkcm0JEmS1JHJtCRJktSRybQkSZLUkcm0JEmS1JHJtCRJktSR\nybQkSZLUkcm0JEmS1JHJtCRJktSRybQkSZLUkcm0JEmS1JHJtCRJktSRybQkSZLU0VjJdJJ9k1yS\nZE2SY2fYniRvbbdfmGSvdv1uSS4YedyQ5Oh222uSXDWybf/5fWmSJEnSxrX5XDskWQK8HXgycCWw\nKsnKqvrmyG77Acvax6OAE4FHVdUlwJ4j57kKOGPkuLdU1Rvn44VIkiRJC22clum9gTVVdWlV3QSc\nDqyYts8K4LRqnAtsnWS7afs8Cfifqrr8TkctSZIkDcA4yfT2wBUjy1e26zZ0n4OA909b95K2W8gp\nSbaZ6ZsnOSzJ6iSrr7/++jHClSRJkhbGggxATLIF8DTgQyOrTwQeQNMN5GrgTTMdW1UnV9Xyqlq+\ndOnSjR6rJEmSNK5xkumrgB1Hlndo123IPvsB51fVtVMrquraqrq1qm4D3kHTnUSSJEmaGOMk06uA\nZUl2bVuYDwJWTttnJfC8tqrHo4GfVNXVI9sPZloXj2l9qp8OfGODo5ckSZJ6NGc1j6q6JcmRwKeB\nJcApVXVxkiPa7ScBZwL7A2uAnwPPnzo+yZY0lUAOn3bqNyTZEyjgshm2S5IkSYM2ZzINUFVn0iTM\no+tOGnlewItnOfZnwH1mWH/IBkUqSZIkDcxYybQ0CXY59hPzfs7Ljn/KvJ9TkiRtOpxOXJIkSerI\nZFqSJEnqyGRakiRJ6shkWpIkSerIAYjSAnOgpCRJmw5bpiVJkqSOTKYlSZKkjkymJUmSpI5MpiVJ\nkqSOTKYlSZKkjkymJUmSpI5MpiVJkqSOTKYlSZKkjkymJUmSpI5MpiVJkqSOTKYlSZKkjkymJUmS\npI5MpiVJkqSONu87AEnDtMuxn5j3c152/FPm/ZySJPXJlmlJkiSpI1umJU20+W5Bt/VckrQhbJmW\nJEmSOjKZliRJkjoymZYkSZI6MpmWJEmSOjKZliRJkjoymZYkSZI6MpmWJEmSOhormU6yb5JLkqxJ\ncuwM25Pkre32C5PsNbLtsiQXJbkgyeqR9fdOclaS77Rft5mflyRJkiQtjDmT6SRLgLcD+wG7Awcn\n2X3abvsBy9rHYcCJ07bvU1V7VtXykXXHAmdX1TLg7HZZkiRJmhjjtEzvDaypqkur6ibgdGDFtH1W\nAKdV41xg6yTbzXHeFcCp7fNTgQM2IG5JkiSpd+Mk09sDV4wsX9muG3efAj6b5Lwkh43ss21VXd0+\nvwbYduyoJUmSpAHYfAG+x+Oq6qok9wPOSvLtqvri6A5VVUlqpoPbBPwwgJ122mnjRytJkiSNaZyW\n6auAHUeWd2jXjbVPVU19vQ44g6bbCMC1U11B2q/XzfTNq+rkqlpeVcuXLl06RriSJEnSwhinZXoV\nsCzJrjQJ8kHAs6ftsxI4MsnpwKOAn1TV1Um2BDarqhvb578LHDdyzKHA8e3Xj93pVyNJA7TLsZ+Y\n93NedvxT5v2ckqQNN2cyXVW3JDkS+DSwBDilqi5OckS7/STgTGB/YA3wc+D57eHbAmckmfpe76uq\nT7Xbjgc+mOQFwOXAM+ftVUmSJEkLYKw+01V1Jk3CPLrupJHnBbx4huMuBfaY5Zw/AJ60IcFKkiRJ\nQ7IQAxAlSRPA7iiStOGcTlySJEnqyGRakiRJ6shkWpIkSerIZFqSJEnqyAGIkqSJ4kBJSUNiy7Qk\nSZLUkcm0JEmS1JHJtCRJktSRybQkSZLUkcm0JEmS1JHJtCRJktSRybQkSZLUkcm0JEmS1JHJtCRJ\nktSRybQkSZLUkcm0JEmS1JHJtCRJktSRybQkSZLU0eZ9ByBJ0qZol2M/Me/nvOz4p8z7OSXdObZM\nS5IkSR2ZTEuSJEkdmUxLkiRJHZlMS5IkSR2ZTEuSJEkdmUxLkiRJHZlMS5IkSR2ZTEuSJEkdmUxL\nkiRJHY2VTCfZN8klSdYkOXaG7Uny1nb7hUn2atfvmORzSb6Z5OIkR40c85okVyW5oH3sP38vS5Ik\nSdr45pxOPMkS4O3Ak4ErgVVJVlbVN0d22w9Y1j4eBZzYfr0F+LOqOj/JPYHzkpw1cuxbquqN8/dy\nJEmSpIUzTsv03sCaqrq0qm4CTgdWTNtnBXBaNc4Ftk6yXVVdXVXnA1TVjcC3gO3nMX5JkiSpN+Mk\n09sDV4wsX8m6CfGc+yTZBXgE8J8jq1/Sdgs5Jck2Y8YsSZIkDcKCDEBMcg/gw8DRVXVDu/pE4AHA\nnsDVwJtmOfawJKuTrL7++usXIlxJkiRpLHP2mQauAnYcWd6hXTfWPknuQpNIv7eqPjK1Q1VdO/U8\nyTuAj8/0zavqZOBkgOXLl9cY8UqSpDHtcuwn5v2clx3/lHk/pzRU47RMrwKWJdk1yRbAQcDKafus\nBJ7XVvV4NPCTqro6SYB3At+qqjePHpBku5HFpwPf6PwqJEmSpB7M2TJdVbckORL4NLAEOKWqLk5y\nRLv9JOBMYH9gDfBz4Pnt4Y8FDgEuSnJBu+6VVXUm8IYkewIFXAYcPm+vSpIkSVoA43TzoE1+z5y2\n7qSR5wW8eIbjzgEyyzkP2aBIJUmSpIEZK5mWJEnqk327NVROJy5JkiR1ZDItSZIkdWQyLUmSJHVk\nMi1JkiR1ZDItSZIkdWQyLUmSJHVkMi1JkiR1ZDItSZIkdWQyLUmSJHXkDIiSJEnzwFkaFydbpiVJ\nkqSOTKYlSZKkjkymJUmSpI5MpiVJkqSOTKYlSZKkjkymJUmSpI5MpiVJkqSOTKYlSZKkjkymJUmS\npI5MpiVJkqSOTKYlSZKkjkymJUmSpI5MpiVJkqSOTKYlSZKkjjbvOwBJkiQtnF2O/cS8n/Oy458y\n7+ecFLZMS5IkSR2ZTEuSJEkdmUxLkiRJHdlnWpIkSYMzKX27x2qZTrJvkkuSrEly7Azbk+St7fYL\nk+w117FJ7p3krCTfab9uMz8vSZIkSVoYcybTSZYAbwf2A3YHDk6y+7Td9gOWtY/DgBPHOPZY4Oyq\nWgac3S5LkiRJE2Oclum9gTVVdWlV3QScDqyYts8K4LRqnAtsnWS7OY5dAZzaPj8VOOBOvhZJkiRp\nQaWq1r9D8gfAvlX1wnb5EOBRVXXkyD4fB46vqnPa5bOBlwO7zHZskh9X1dbt+gA/mlqe9v0Po2nt\nBtgNuOROvN6Z3Bf4/jyfc2MwzvkzCTGCcc4345xfxjl/JiFGMM75Zpzza2PEuXNVLZ1rp0EMQKyq\nSjJjVl9VJwMnb6zvnWR1VS3fWOefL8Y5fyYhRjDO+Wac88s4588kxAjGOd+Mc371Gec43TyuAnYc\nWd6hXTfOPus79tq2Kwjt1+vGD1uSJEnq3zjJ9CpgWZJdk2wBHASsnLbPSuB5bVWPRwM/qaqr5zh2\nJXBo+/xQ4GN38rVIkiRJC2rObh5VdUuSI4FPA0uAU6rq4iRHtNtPAs4E9gfWAD8Hnr++Y9tTHw98\nMMkLgMuBZ87rKxvfRutCMs+Mc/5MQoxgnPPNOOeXcc6fSYgRjHO+Gef86i3OOQcgSpIkSZqZ04lL\nkiRJHZlMS5IkSR2ZTEuSJGniJXlkH993EHWmF1qSXwN2qqr5ngBGA5Nkr/Vtr6rzFyoWSRtPkn8H\nZh0EVFVPW8BwpImVZDPg0VX1lb5jGUeS3YGD28ePgQWvNb3oBiAmeSrwRmCLqto1yZ7AcUP7oE2y\nLfA64Nerar/2j+UxVfXOnkNbS5LXAn9TVbe0y1sBJ1TV8/uNrJHkc+vZXFX1xAULZgztbKDPAR5Q\nVccl2Qm4f1V9refQAEhy7/Vtr6ofLlQs40jyQODKqvpVkicADwdOq6of9xvZ2pI8CDgR2LaqHprk\n4cDTqupvew4NgCRvY/2J6ksXMJwZJfnt9W2vqi8sVCzjSnJ/YG+an+2qqrqm55BuN+kXJ0l2BA6q\nqn/oO5bpkmwP7MxIg2ZVfbG/iNaV5OtV9Yi+45hNkl24I4G+mebnubyqLuslnkWYTJ8HPBH4/NQf\nSpKLquph/Ua2tiSfBN4FvKqq9kiyOfD1Acb5euB3acohbgv8E/C2qvqnXgObUElOBG4DnlhVv5lk\nG+AzVdXLravpknyX5h9sZthcVfWABQ5pvZJcQNNKsQtNCc+PAQ+pqv37jGu6JF8A/hz4l5HPpW9U\n1UP7jayR5ND1ba+qUxcqlk1FkhcCfw38B8376bdpGnZO6TWw1sjFyTOA+wPvaZcPBq6tqj/tJbD1\nSLIUOJAmxl8HzqiqY/qNam1J/h54FvBN4NZ2dQ3t4iTJG4GvAh+pgSWKSb4KbAWcDpxeVd9J8t2q\n2rWvmBZjN4+bq+onTQPg7Qb1h9K6b1V9MMkr4Paa3bfOddBCq6pXJPks8J/Aj4DHV9WansOaUZKH\nArsDd5taV1Wn9RfRjB5VVXsl+TpAVf2onfBoEPr8sOrotva983Sai7y3Tf1sB+buVfW1aZ9Lt/QV\nzHSTlCwnWQa8nnXf64O60KO5eHpEVf0AIMl9gK8Ag0imp1ryk7xp2hTN/55kdU9hrSPJPWkS/mcD\nDwI+AuxaVTv0GtjsDgB2q6pf9R3IHA4HXgbcmuQXNBd8VVVb9RsWANcC29M04C0FvkPPedxiTKYv\nTvJsYEn7oftSmg+woflZ++FaAFMzS/Yb0rqSPB54K3Ac8DDgbUleUFXf6zeytSV5NfAEmn+wZwL7\nAecAQ0umb06yhDt+70tpWqoHp201X8baCcugblXS/DwPppll9anturv0GM9svt92SZn6vf8BcHW/\nIa2r/Xt8OesmqkPqLvUu4NXAW4B9aO6aDXGw/Q+AG0eWb2zXDc2WSR5QVZcCJNkV2LLnmEZdB3wN\n+EvgnKqq9uJ5qC6l+QwadDJdVffsO4bZVNUBSe5FcxH1mjaX2zrJ3n11iVyM3TzuDryKpmtCaGZn\nfG1V/bLXwKZpB869DXgo8A2aq68/qKoLew1smiRfA/6wqr7ZLj8DeF1VPbjfyNaW5CJgD5quMnu0\nfdLfU1VP7jm0tSR5Ds0twL2AU4E/AP6yqj7Ua2DTtLeojwJ2AC4AHg18dWBJ1dTAlCNoYnt/mwg8\ns6r+vufQ1pLkATSzd/0fmjs83wWe21f/v9kk+QzwAeAYmp/rocD1VfXyXgMbkeS8qvqt0e57U+v6\njm1UktNoGiA+RnMRtQK4sH1QVW/uL7o7JNmX5m/zUpr/mTsDh1fVp3sNrJXkaOAgmgT//TR/n2cN\n7U7EyLiD7Wn+F53NSEI9hHEHo0bG7+xaVa9t+6BvN5TxO6OS3I/m/+ZBNMUldlzwGBZbMj1J2n7S\nu9F8gF1SVTf3HNI6kiypqlunrbvP1K3LoUjytarau+0zvw9NK9C3hpb0AyR5MPAkmt/72VX1rZ5D\nWkd7cfJI4Nyq2rON+XVV9YyeQ5tV25K+49AuSEcl2RLYrKpunHPnHowkqhdW1cPbdauG0qcfIMlX\ngMcB/x9Nf+SrgOOrardeA5umvVs2q6r6m4WKZS5J7gpMfVZ+e4hdFNoL0oNo+ksvo7k7cUZV/Xev\ngbUmbdzB0MfvzCbJzlV1+UJ/30XTzWPSRia3LbyjHpTkJ8BFVXVdHzHN4r5JXgdsX1X7TlUdAQZV\ndQRYnWRr4B3AecBPaQZXDEbbvePiNsH/dt/xzOGXVfXLJCS5a1V9O8mgkhWAJJ8HnkbzWXcecF2S\nL1fVy3oNbJp2PMQ/AK+YGuyT5PyqWm9pxx5MXdBfneQpwPeA9VZ46cFRwN1puvC9lubieb2JTB9G\nk+U2Ufnx0AZ6we13c18G7FxVf5xkWZLdqurjfcc2qu2G8jrgde34mGfTdOn7jV4Da00ly+0F8y+n\nGqHaz/279hnbLAY7fifJu5g9nyvgBQsYDrCIkmmacniT5AU0SelUabcn0CQDuyY5rqre3Vdg0/wb\nbdWRdvm/aW6zDSqZrqoXtU9PSvIpYKuhtVBW1a1JLkmyU1X9b9/xzOHK9uLko8BZSX4ELHhrwBju\nVVU3tN1STquqVycZ1O+9dTFNv97PJHlWNSUGZ6qY0re/bfsq/hlNN7StgEFVdaiqVe3Tn9L0lx6U\nJH8NfLC9AL0r8ElgT+CWJM+uqs/2G+E63kXzv+cx7fJVwIeAQSXTo6rqG8Ar28fQnA38Ds3fJ8Cv\nAZ+h6eI1JEMevzPT396ONJ9FSxY4FmARJdMjI5OPqqoTRrclOQoYWg3SzYHfrKpr4fa606cBjwK+\nCAwlmZ6IqiPtgJT/qKqfVNVlSbZOckBVfbTv2KbZhmaQ7NeAn02tHNqdk6qaGuDzmjS1vO8FfKrH\nkGazeZLtgGdyxwXfEN1SVX+R5FnAl5I8jwFWGRppjfwJTYvv4CQ5Cziw2lribavv6VX1e/1Gdrtn\n0bSYQ9OQZ7FeAAAgAElEQVRivhnNmJgH0YyTGFoy/cCqelY7kJeq+nnbn3YQcke5zttXjSxXVT1w\n4aNar7tV1VQiTVX9tG39H5q3AmcA90vyd7Tjd/oNqVFVH5563nbveSXweOB4emrIWzTJ9IhDgROm\nrfvDGdb1bcepRLp1Xbvuh0mG1Hd6IqqOAK+uqjOmFqrqx22fxaEl03/VdwDjaH/PF1fVjVX1hTST\n9TyCpkTikBxHM8j4nKpa1X7wfqfnmGYSgKr6QJKLgfcBO/Ub0rqSnAocNS1RfVNV/VG/ka3lvjUy\nKU97e/p+fQY0zU0j3Tl+D3h/e8v/W+04maG5Kc2swVOf8Q9kWJUops92txnNxfMxwBDLYP4syV7V\nzr6b5LeAX/Qc0zqq6r3tGKOp8TsHDGn8TjtO5y9p/u/8A3BEtZPH9WGIb9yNor2qfjZNN4mVI5vu\nCQxq1rbW55N8nOZ2GsD/a9dtSTNd5lC8DFgJPDDJl2mrjvQb0oxmKo01uL//GuAsbbM4kabiyJSf\nzrCud20VlA+NLF9K814amhdOPamqbyT5vzTVHYbm4TMkqkObJe220a5SSXZmWK38v2r79F5L07o/\nOqnIEFsoX01z12nHJO8FHkvTADUIdUed7s2AQ2jqd18APGWqytTAHAV8KMn3aJLU+9PcrRii7wA3\n0P6vHEoXxCQfAn4LeBNN145bga2mbphUDzPxDi6Z2Ii+QlO39b40v4ApN9KWIhqYF9PUUHxcu7ya\nZqrhnzGA26tJHglcUVXnp5kp63CaJOUzwJW9Bjez1UneDLy9XX4xTT/AQWlbfN8G/CawBU3/r5/V\nMArlj8roYKmqum2IrWpJ7kYz/uAhrF0XeRAtqUmeWFX/AezcJn2jfjrTMT3bLMk2VfUjuH16+aH9\n3l8FnJNmVskA/xc4rN+Q1nIUTaWRpcBbquq7AEn2Z4AtqVV1VpLzacpfhubOxPd7Dut2Se4C/BFN\nUnUOTQvqUCcO24zmc/3BNJW6YLiVul5CcyF1LU2yOtV95uF9xtV6JE0sx9CM34A7xpgUsOBlES2N\nN2Bti8+zaaZH/S7w4RrINN3th+vvtN1OHk8zredLaAbS/GZVDap1um3R/yuagR8AZwF/216cDEaa\nmcUOomlNXQ48D3hQVb2i18CmSfIR4PM0rdEALwL2qaoDegtqBm0Lxrdp3kfH0dRN/VZVHdVrYK0k\nf9MOinzXDJtrKEn/lLYv9ytp/j5Dcxfq7wY0IBqAJPelSf6gKd84mORv0rQD3v96ZHkz4N1V9Zwe\nw7pdkitpZgv9R2CdVtOq+siCB7UeSb5eVUO7m7OOJGtoKnoMqszthkjykKq6eEG+12JJppOcU1WP\nS3IjMwxWGErLX5IH0dTJPBj4Pu0ECVU1vdWqV0n+q6r2aJ+/nWbihte0yxdU1Z59xjepkqyuquVZ\nu47v4D582z6obwWeSPN+Ohs4emBlG2//2U39PNtWrC9V1aPnPFgzSlP+cmpynv8Yyq30JA9uK2TM\n2NVoqo/qULRjTV5Nc/exaFpVjxta8tJe6P13Vb2+rT7yQZrJr17Tb2SNJP/GesqkDfCC9I00ZVk/\nUgNOwNqB5U/usx/ynZUFLC86tNtzG9OWMOwpMlvfBr4E/P7Uraokgyo91VqSZPP2jfYk1r6NOpi/\nqyT/WFVHZ5Y640OrkgH8PE0tzwuSvIGma9LgpkJuk+aD+o5jDFO3T3/c9lO9BhjSYDTg9opC76Lp\ndvYOmr7nx1bVZ3oNrJVkq7bE4L1pfobvG9l27z76KM7gZTSfQ2+aYVtxxwXAUJxOU5lpqg//c2ga\nT35n1iP68UfAe9uKTfsAn6yqt/Qc0+2q6g/H2S/JoTWMiVEOp/lbvSXJLxleg95UDf5LacZpfYK1\nZ2ocxMycY1qwqjODSXoWwGCvAKd5Bk2S8rk09ZBPZ5j1Zt8PfCHJ92lGIn8JIMlvMKxqHlO3nyel\nzvghNMnzkTR9AHdkQAPmkvxFVb0hd0yNu5Ya2JS4wMltxYm/ohkoew/gr9d/SC/+qKpOSPJ7wH1o\n/g7eTTMGYQjeB/w+zTiDmcqQ9T51c1Ud1nZB+Muq+nLf8Yxhu6p67cjy36YpjTgI01r4TwD+Bfgy\nzef+XkNr6R/DUTSlB3s1AQ16U/H9b/vYon3A5ORRUxYs3sXUzeNKYNYrqqFdbbV9fFfQdPd4Ik2N\n6TOG0lIFtw+W245mitGfteseBNxjSB+0aQrPnzaUPn4zGcoo6bkkeWpV/XtmmRp3IC0/E2ekG8oJ\nwOer6oyhde9JM1R+x6H/nQ7t5zabdkD012i6TUDT/3zvqjpm9qMWTnubfzZVVUNr6V+vIf1dtBf4\ny1h7UPQX+4toXUkObKshrXfdkC1kN4/FlExfTTNYasZW3hqZ2nVo2jfegcCzqupJfccziZKcAzyx\nqm7qO5aZjL7pk3y4qgbTGj1de3Hy90P5pz+TkVuVMxrgxfO7gO2BXYE9aKq4fL6qfqvXwKZJclFV\nPazvONZn6H1SR8bthKb74dQkV0uAnw7ldj/cPtjwwKr6QN+x3FkLmVjNEccLaVrJd6Ap4fdo4KtD\nuziZ6ec1lJ/huJKcu1DjYxZTN4+rq+q4voPooi1DdXL7UDeXAl9OU2N8dGbBoSRVoxd5vd8yX59q\npj1/bN9xzGHot1KnewFNJZxLq5lh7j6MTIW9kKPS53B+kkfWHVN2D9FUn9Rbk/yCgfVJnYDb/Ler\npuTln9P05Z50Q+kueRRNabdzq2qfNJOPvK7nmG6XZD9gf2D7JG8d2bQVTdWU3s02yHjK1J3xhRxo\nvpiS6aG8kdSP/2kfmzHMRKtmeT5UF7QXJh9i7YuTQZShGvKdpplU1W3A+SPLPwBGqzq8m2FMiPMo\n4DlJLqf5vU8lqkOoPQsMP1mdtKojwGeTHEOTUI++14cw6HRDDKUf/S+r6pdJSHLX9m9ht7kPWzDf\no5nX4mmsPRfDjTTjeIZgpkHGU3oZbLyYunkMZcS5epTk7lX1877jmC7JrdyRnPwaMBXjoFrVpkxQ\nXeRJmP56TkPp75l1J5YBoKouX+hY1ifJ04DHt4ufr6qP9xnPqCQnt4MlR/skj06ANLTb/d+dYXVV\n1aDuoCXZlqaF99erar+2hONjquqdPYe2liRn0Nx1Opom6fsRcJeq2r/XwKZpy4gGeFC7apCTywzF\nokmmtbgleQzwTprBkTsl2QM4vKpe1HNo2ohmSkKHkphuiCH1VUzyOGBZVb0ryVKa99RMCVcvkhxP\ncxv9ve2qg4HVNZCJj5LsDfxvVV3TLh9KU7HnMuA1Nvp0k+STNOUlX1VVe6SZkfXrQ+7jn2b24HsB\nnxraeJ42ttNo/i5DU1nq0AEOlHwosDtrD+Y8bcHjMJnWYpDkP2lGy6+cSqSSfKOqHtpvZJMpyQ40\n055P9Z3+Ek0L8KCmkk/yX8ATau3pr78w5H+wMxlKMp3k1TQzc+5WVQ9K8uvAh6pqMH3ok1wI7Nl2\nnZkaMPv1oXRFyYTNHgvDSVjWJ8mqqnrk6MVyBjSBWJK7AUcAvwFcBLyzBjwhSpLzgGdX1SXt8oOA\n9w9pUHT7efQEmr/NM4H9gHP6eA8tpj7TWuSq6oqmutftbp1tX83pXTS1hw9sl5/brntybxHN7E3A\nuUmmyo8dCPxdj/F0NZRWq6cDj6Dt311V30syxD7KWwNTLbz36jOQGSwZaX1+FnByVX0Y+HCSC3qM\na0azJSw0rZZD8rN24G7B7aVbhzTnwak0k0h9ieZnuDvNYMShustUIg1QVf/ddv0Ykj+gqX709ap6\nftvV5z19BGIyrcXiiiT/B6j2A+Eo4Fs9xzTJllbVaL/pf0tydG/RzKKqTkuymjsGpDyjBjL99agk\nH6HphvTJqRbVUQs5Kn0ON1VVJZlKWLbsO6AZvB74etsnOTR9p4/tN6S1TMTssSMGk7DM4WU0EzM9\nMMmXgaU0sQ/F7lN3xJK8k6bG+JCtTvKv3PG7fg7NwMQh+UVbceaWJFsB19F0R1lwQ3zjShvDETSz\neG0PXEUzs9yLe41osv0gyXNpZsKEpl/qD9az/4Ka4ZbqSUO+pQr8M82gpLcm+RDwrtFWoQH5YJJ/\nAbZO8sc0U02/o+eY1lJV70/yeZp+0wAvn+qfPBCTMnvslMEkLOtTVee3/Xx3o7mIGtqAudtjqapb\npt0lHaI/ofkfOTWr7ZdoPqeGZHWSrWk+g84DfkpTY37B2Wda0gZrqzq8DXgMzW3VrwAvrYHMjpfk\nA6x9S/Wyqhpcy/l0Se5Fc2HyKuAKmn8S7xlSUpDkycDv0iQsn66qs3oOaS2zlJz7CXD5UC6oMiGz\nxwIk+WfglcBBwJ/RJCwXVNXz13vgAkvyjBlW/wS4qKquW+h4phup2ARrV20aZMWmSZNkF2Crqrqw\nl+9vMq3FIMmuNIN8dmHkjkxVPa2vmLTxZGSmvnZU/9eGMIBvfdr+ns8FDqGp9fpe4HHAw6rqCT2G\nto62hXL0fTSYChRJzqWpyX0hTaLyUOBimr7Tf1JVn+kxvInWd8KyPkk+QXNxP1Vy8Ak0rZW7AsdV\n1bt7Cm0iJfl94LXAzjTv9cEl/UmeDvxHVf2kXd6aZsD5Rxc6Frt5aLH4KE2f1H8H1umTqg0zbWas\nKT+hKUH2sYWOZwYTdUu1rT27G83kLE+tqqvbTR9o+3wPQpLDgb8BfknzPgrNnYkh1Rz+HvCCameM\nbOsNHwf8BfARmi5e2gBtq+/jaH7X59BcqAzN5jTVUK6F2+tOn0Yz0dAXad5bGt8/As+gadkfaqvr\nq6vqjKmFqvpxO2DWZFraSH5ZVTMlgOrmbsCDaWZAhKZO7neBPZLsM4AuFXskuaF9HuDX2uXBta60\n3lpVn5tpQ1UtX+hg1uMY4KFV9f2+A1mPB9XI1OtV9c00sw5eOvSLqiFqu3n8BneMjzg8ye9U1dDG\nnOw4lUi3rmvX/TDJYLpJTZArgG8MOJGGZkbj6XrJa02mtVic0F6xfgb41dTKofVPnCAPBx5bVbcC\nJDmRpn/y42gG/PWqqpb0HcM4Rvt5ztTnswYyPfuI/+GO2TmH6uL27/H0dvlZwDeT3JWROxYa2xNp\nWnynKricStNtZmg+n+TjrH2B//m24syP+wtrYv0FcGaSL7D2/8w39xfSOlYneTPw9nb5xaw9BfqC\nMZnWYvEwmr6oT+SObh7FHSXTtGG2Ae7BHdUHtgTuXVW3JvnV7IdpmqeuZ1vRdEsYklcAX2knQRr9\nB/vS2Q9ZcH8IvIhmumaAL9O0qN8M7NNTTJNsDbATMDVl/I7tuqF5MU23hMe1y6uBbdsBnv7eN9zf\n0Qw2vRuwRc+xzOYlwF8BH2iXz6KnKl0m01osDgQeMLQpWyfYG4AL2hJkU7V8X9e2An22z8AmydAq\nIozhX4D/oLn7MMixB1X1i7ZrwsdnKC/40z5imkRJ/p3mgu6ewLeSfK1dfhQDrJHc1j+/FHg0zef9\nd4EP9xvVRPv1oc8Q3F4oDaKGvNU8tCgk+Shw2BBKJG0qkmwH7N0urqqq7/UZzyRK8tyqek+Sl820\nfWC3VBmdqnmokjwN+Adgi6raNcmeNNUcrNyzAdqazbOqqi8sVCzr05YUPLh9fJ+mlfKYqtq518Am\nXJI3AJ8dYvWbJP9YVUePXPCtpY/3ui3TWiy2Br6dZBVr3572H2wHaUZyPYmmtf+4JDsl2buqBtdi\nNXBTMwgOcUrumXwyyWE0VXFG30eDKY0HvJrmIu/zAFV1QVsaUxtgerI8vRzigHybZrzG71fVGoAk\nf9pvSJuEPwGOabvt3cywBm9PVWZ5Y69RjLBlWovCbK0sQ2ldmTTtAK/bgCdW1W8m2YZmAopHznGo\nJliS786wuqpqMKXxkpxbVY8ebUVPcmFVPbzv2CZRe/F0HNPKIQ7ld57kAJoJZR4LfIpm4Om/VpUX\nUBtRkoeMVs3p4fsvAU6rquf0FcOoIV5lSvPOpHnePaqq9krydYCq+lGSoQ5SGbxJmVRoQhKUi5M8\nG1iSZBnNdMhf6TmmSfbnDLgcYjtBx0fb8RoraAae3q+94D9jiN0UNhHvppkcqRftYPedk2wxhLFQ\nJtNaFJLcyB19q7YA7gL8bCC3rCbRzW3LwFS5rKUMdEDahJiISYWSPG+m9VV12kLHsh4voZmO/VfA\n+4BP08zkpm4moRzi1GC09wHva++UHQi8HCfp2ViGULT9UuDLSVZyx1TtvYw1MZnWolBVt/dJbfv7\nrqAZ9a1u3gqcQdMC9HfAH9CUKFI3kzKp0Gg3nrvR9Js/n2amuaF4SlW9iiahBiDJgdxRf1gbZhLK\nIa6lqn4EnNw+tHEMoY/w/7SPzeh53Il9prVoTUJlgiFL8mCaZCrA2VX1rZ5Dmlhtt4RlTNikQkm2\nBk6vqn37jmVKkvOraq+51mk8bUm8c5hWDrGqTu0tKPVuSO+pJHevql7vntgyrUVh2uxymwHLaQbU\nqIMk766qQ2hG0k9fpw03qZMK/QwYRD/qJPsB+wPbJxlt5d8KuKWfqDYJd6mqGUs3alHrvZ9yksfQ\ndI+7B7BTkj2Aw6vqRQsdi8m0FovRmeZuAS6j6eqhbh4yutD2n/6tnmLZFEzEpELT6rpuBuwOfLC/\niNbyPZpZ757G2lMK3whYKq27SSiHqHmW5LHABVX1syTPpRlseEJVXQ5QVUPoJvmPwO8BKwGq6r+S\nPL6PQOzmIWlsSV4BvBL4NZpBSVODUG4CTq6qV/QV2ySblEmFppWYvAW4vKqu7CuemSS5S1Xd3Hcc\nm4pJKIeo+ZfkQmAP4OHAvwH/CjyzqtY7mc9CSvKfVfWoaWUw/6uq9ljoWGyZ1qLQVpv4Y9YtPfZH\nfcU0iarq9cDrk7zexHleTcSkQhNSYnKXJK+naTW/29RKk79uJqQcoubfLe0U7SuAf6qqdyZ5Qd9B\nTXNFkv8DVJK7AEcBvYzdMZnWYvExmlmyPgvc2nMsE6+qXtGWn1rG2gnLF/uLaqK9uu8AxtGOPfh7\n4H40dyWGNCvalHfR/DzfAuwDPJ+mS4o2QJK/qKo3tM8PrKoPjWx7XVW9sr/otABubO9EPhd4fJLN\naErKDskRwAnA9sBVNAO4X9xHIHbz0KKQ5IKq2rPvODYVSV5I0wqwA3ABTZnBr1bV0AfM6U5IsgZ4\n6pArtyQ5r6p+K8lFVfWw0XV9xzZJRqs1TK/cMKRKDto4ktwfeDawqqq+lGQn4AkDqyk/GLZMa7H4\neJL9q+rMvgPZRBxFU3P43Krapy2T97qeY5pYSR4NvA34TZpJhZYwzEmFrh1yIt36VduK9p0kR9K0\nWN2j55gmUWZ5PtOyNjFVdQ3w5pHl/2VY9eQHNXOsybQWi6OAVyb5FXAzw7w9PUl+WVW/TEKSu1bV\nt5Ps1ndQE+yfgINoJhZZDjwPeFCvEc1sdZIP0MzYONq3+yP9hbSOo4C700wj/lqarh6H9hrRZKpZ\nns+0rE1EknOq6nHTZg2GYf7PHMzMsXbzkIAkD6mqi/uOY1IkOYOmL+rRNLWQf0RTj3b/XgObUElW\nV9XyJBdW1cPbdYObVCjJu2ZYXQ7k3fQkuZWmjni4o3oP7fLdqmpo/We1yExV8+g7DjCZlgD7AN4Z\nbbm0ewGfGnqd5KFK8kXgd2jKT10DXA38YR8lnu6MJK9oK770GcNZwIFV9eN2eRuaWRp/r8+4JM2v\nIc0cazcPqWEfwA3Q9vG9uKpurKovJNkKeATwnz2HNqkOoak4cSTNBCM7Av+v14i6ORDoNZkG7juV\nSANU1Y+S3K/PgCRtFIOZOdZkWmp4i2bDnEgzI9aUn86wTnNIslNV/e/UrGI0U9z/TZ8x3UlDuCi9\nbernCpBkZ3x/S5uiwcwca+1NSV2kRvqIVdVteHHexUenniT5cJ+BzJMhJK2vAs5J8u4k7wG+CDjB\nkLTp+QbNhFe985+f1Oj9ynbCXJrkpTSt0QAvAi7tMZ5JNdqSuynM0Nd7y3RVfSrJXjS1zwGOrqrv\nT213sLG0yRjMzLEm01oUkpxdVU+abV1VPXrmIzWLI4C3An9J0xp5NnBYrxFNpvWVH5tEH5p7l42v\nTZ4/Psvmd2N3JGlTMJiZY63moU1akrvR1Jz9HPAE7mg524qm+sSDewptkzaEqg6TYI7yY0Or6UqS\npcAfs+4kCRNTGm+IJQclTTZbprWpO5ymFvKvA6Plcm6gmShDG8cQqjoMXlUt6TuGDfQx4EvAZ4Fb\ne46lK1uQpE3AtIlltgDuQk8zx5pMa5NWVScAJyR5SVW9re94FpHe+85qo7h7Vb287yAkqaruOfU8\nSYAV3DFWYkFZzUOLxSlJ/jLJyQBJliX5/b6D2oTZ+rdp+niSSZ/l0sHG0iamGh8FepmcyT7TWhSS\nfAA4D3heVT00yd2Br1TVnj2HtkmyX+qmqb2tuiXNyPmbGWDf7rkGG0vaNCR5xsjiZsBy4Ler6jEL\nHYvdPLRYPLCqnpXkYICq+nl7W0gbxyCqOmh+jd5WHZqRwcb3bacQHx1svH1vgUnaWJ468vwW4DKa\nrh4LzmRai8VNSX6NtvtBkgcyUpdSG2auqg5V9bp+ItPGkOTBVfXttn7zOqrq/JnWLzAHG0uLSFU9\nv+8YptjNQ4tCkifT1ETeHfgM8FjgD6vq833GNamSfIWmqsN5jFR1qKpNYRY/TZPk5Ko6LMnnZthc\nVfXEBQ9qFg42lhaHIZXqNJnWopHkPjQjfQOcOzormjZMkgvsb64hSrIl8KfATu0FwDJgt6qabRIX\nSRNoSI06JtNaFJIcV1V/PbK8GfDuqnpOj2FNrCR/SzOA88y+Y9HCafslvwh4HE2XqS8BJ1XVL3sN\nbISDjaXFYUiNOpbG02KxY5JXACS5K3AG8J1+Q5poR9GUSftFkhuS3Jjkhr6D0kZ3GvAQ4G00/ZAf\nQjM995A8sKreQFNthKr6OdY9lzZFgynVacu0FoW2csd7gYuAfYBPVtVb+o1KmixJvllVu8+1rk/t\nrd8nAV+uqr3awcbvr6q9ew5N0jwaUqlOq3lokzat+sAJwL8AXwa+kGSvgVQhmBgTUtVBG8/5SR5d\nVecCJHkUsLrnmKZ7NfApmrtR76UdbNxrRJLm3VylOpM8pKouXohYbJnWJm2W6gNTBlWFYBJMUlUH\nzZ8kF9H0kb4LsBvwv+3yzsC3h9QyDQ42lgRJzq+qGRt+5v17mUxLktYnyc7r215Vly9ULHNxsLEk\nWNiZeB2AqEUhyeuSbD2yvE1bkUIdJLlbkpcl+UiSDyc5uq30oE1QVV0++gB+QdMyPfUYEgcbS4IF\n/GyyZVqLwkxXqAt5C2hTk+SDwI3Ae9pVzwa2rqr/v737CbGqDsM4/jwOJeUf1EIXERTWpnRRtIhy\nY9LCKKFFRNgiM5eR2EpwEQlGgRLUpiTFxWzauEgRM9PCwoVZFBXUogiKkPxDUaAtnhb3THMdm0kv\nc8/vnnO+H5jFOWdgnsUM886Z931/T5RLhWGzvU7STvVOGTyjXpvHt0nuLhqsD8PGAKR6f8czgIiu\nGLM9N8lFSaqOFp9bOFOTrZjSJ3vM9jfF0qAu29XrRf4gyT22V0t6unAmSQwbA7jCpbq+EMU0umJc\n0lHbe6vrDZL2FczTdE3Y6oDZ93eSs7bn2J6T5Jjt10uHquyccn1e0l3V/UhiOBZoEdtHk6yZ7l6S\n++vKQjGNTkjyqu0v1ds/K0nbkxwumamJpmx1+NT2ZVsdSmZDLS7Yni/pY0njts9I+rNwJklSktWl\nMwAYvmo+50ZJN9terMlDmRZKuqVIJnqmAVytJm11wOyxfYekZZK+UG/4cI6k9er9EXUwyWcF413G\n9g5JryW5UF0vlvRikm1lkwGYDbZfkLRZvdmNX/oe/S5pd5I3a89EMY02s30iyarqpKT+b/ZiJyW1\nie2lkv7d4pHkp4JxMCS2D0jamuSrKfdXStqR5LEyya7EsDHQDbafT/JG6RwSq/HQfvOk3klJSRb2\nfSygkB6c7XW2v5f0g6SPJP0o6VDRUBimZVMLaUmq7t1Wf5wZjVUr8SQxbAy02B7b22y/LUm277T9\naIkgFNNoO/71MhwTWx2+S3K7er3oJ8tGwhAtmuHZDbWluDoTw8YbbW+UdEQMGwNttEe9jR0PVNc/\nSypyfgQDiGi7pba3TPcwya46w7TIKG91wOw7ZXtTkt39N20/J2lk+qUlho2BDlme5EnbT0lSkr+q\nPfO1o5hG241Jmq/JaV/MjpHd6oCh2Cxpv+31miye75N0vaTHi6WaRpJDou0IaLtLVRtXJMn2ckkX\nSwRhABGtxuDR7GrSVgfMvuqQlhXV5ddJPiyZpx/DxkC32H5Y0jb19sm/L+lBSc8kOV57FopptNl/\nTfZjcE3a6oBu4Wcd6B7bN6k3v2NJJ5P8ViIHA4houzX//ym4Bk3a6oBu4c0Q0CG2X05yNsnBJAck\nnbM9XiILPdNotSTnSmdomSZtdUC3MGwMdMuttrcmeaVah/mupM9LBOHNNIBrccr2pqk3R3GrAzpn\nYth4wTQfANrlWUkrbW+V9J6k40leKhGEnmkAV832Mkn71dvtecVWhyS/lsqGbmPYGOgG2/0/59dJ\nekvSJ5LekaQkp2vPRDEN4FqN8lYHdBMDiEA32D42w+Mkeai2MBWKaQBA49lewowEgBLomQYANB6F\nNNAttnfYXtR3vdh2kePEKaYBAADQNGuTXJi4SHJe0iMlglBMAwAAoGnGqpV4kqTqaPG5M3z+0LBn\nGgAAAE0zLumo7b3V9QZJ+0oEYQARAAAAjWN7rSZPOj6S5HCRHBTTAAAAwGBo8wAAAEAj2D6RZJXt\nPyT1vxG2enumF9adiWIaAAAATTFPkpIsKB1kAts8AAAA0BQj15/Mm2kAAAA0xVLbW6Z7mGRXnWEk\nimkAAAA0x5ik+er1SI8EtnkAAACgEWyfTnJv6Rz96JkGAABAU4zMG+kJvJkGAABAI9hekuRc6Rz9\nKJ9IwAcAAAAwSURBVKYBAACAAdHmAQAAAAyIYhoAAAAYEMU0AAAAMCCKaQAAAGBAFNMAAADAgP4B\n+HbXsxyHFcEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x8b76bad630>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "importance = pd.Series(rf_model.feature_importances_)\n",
    "importance.index = training_vars\n",
    "importance.sort_values(inplace=True, ascending=False)\n",
    "importance.plot.bar(figsize=(12,6))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x8b76cf6668>"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAswAAAHHCAYAAACr/g5EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcJXV1///Xm2FRUUQFN3YVUFxQHBEjXyMaE3HDGIkg\n7gsSN4gxCahRgwkao0Y0RiQqETfUn2JQUcQFFZHIgAREUQmigAugoLiwen5/VF3mTk93dc/Qc6tu\nz+v5ePRjuqpu9Zzp6b516lPncz6pKiRJkiTNboO+A5AkSZKGzIRZkiRJ6mDCLEmSJHUwYZYkSZI6\nmDBLkiRJHUyYJUmSpA4mzJIkSVIHE2ZJkiSpgwmzJEmS1GHDvgOYzRZbbFHbb79932FIkiRpCTvz\nzDOvqKot53vdIBPm7bffnhUrVvQdhiRJkpawJD9ayOssyZAkSZI6mDBLkiRJHUyYJUmSpA4mzJIk\nSVIHE2ZJkiSpw4IS5iSPTvK9JBckOXSW4/skOSfJ2UlWJNlzoedKkiRJQzZvwpxkGfAOYG9gF2D/\nJLvMeNkXgV2r6v7Ac4B3r8G5kiRJ0mAtZIR5d+CCqrqwqq4DjgP2GX9BVf2mqqrd3BSohZ4rSZIk\nDdlCEuatgIvHti9p960iyZ8nOR/4DM0o84LPlSRJkoZq0Sb9VdXxVXVP4InA69b0/CQHtvXPKy6/\n/PLFCkuSJEm6WRaSMF8KbDO2vXW7b1ZV9VXgbkm2WJNzq+roqlpeVcu33HLeJb0lSZKkiVhIwnwG\nsGOSHZJsDOwHnDD+giT3SJL2892ATYBfLORcSZIkacg2nO8FVXVDkhcDJwHLgPdW1XlJDmqPHwX8\nBfCMJNcDvwee0k4CnPXcdfRvkSRJkhZdVja3GI7ly5fXihUr+g5DkiRJS1iSM6tq+Xyvm3eEeci2\nP/Qzi/41L3rDYxf9a0qSJGl6uTS2JEmS1MGEWZIkSepgwixJkiR1MGGWJEmSOpgwS5IkSR1MmCVJ\nkqQOJsySJElSBxNmSZIkqYMJsyRJktTBhFmSJEnqYMIsSZIkdTBhliRJkjqYMEuSJEkdTJglSZKk\nDibMkiRJUgcTZkmSJKmDCbMkSZLUwYRZkiRJ6mDCLEmSJHUwYZYkSZI6mDBLkiRJHUyYJUmSpA4m\nzJIkSVIHE2ZJkiSpgwmzJEmS1MGEWZIkSepgwixJkiR1MGGWJEmSOpgwS5IkSR1MmCVJkqQOJsyS\nJElSBxNmSZIkqYMJsyRJktTBhFmSJEnqYMIsSZIkdTBhliRJkjqYMEuSJEkdTJglSZKkDibMkiRJ\nUgcTZkmSJKmDCbMkSZLUwYRZkiRJ6mDCLEmSJHVYUMKc5NFJvpfkgiSHznL8gCTnJDk3yWlJdh07\ndlG7/+wkKxYzeEmSJGld23C+FyRZBrwDeBRwCXBGkhOq6jtjL/sh8MdVdWWSvYGjgQePHd+rqq5Y\nxLglSZKkiVjICPPuwAVVdWFVXQccB+wz/oKqOq2qrmw3Twe2XtwwJUmSpH4sJGHeCrh4bPuSdt9c\nngt8dmy7gC8kOTPJgXOdlOTAJCuSrLj88ssXEJYkSZK07s1bkrEmkuxFkzDvObZ7z6q6NMkdgZOT\nnF9VX515blUdTVPKwfLly2sx45IkSZLW1kJGmC8Fthnb3rrdt4ok9wPeDexTVb8Y7a+qS9s/LwOO\npynxkCRJkqbCQhLmM4Adk+yQZGNgP+CE8Rck2Rb4BPD0qvr+2P5Nk9xm9Dnwp8C3Fyt4SZIkaV2b\ntySjqm5I8mLgJGAZ8N6qOi/JQe3xo4BXA3cA/iMJwA1VtRy4E3B8u29D4ENV9bl18i+RJEmS1oEF\n1TBX1YnAiTP2HTX2+fOA581y3oXArjP3S5IkSdPClf4kSZKkDibMkiRJUgcTZkmSJKmDCbMkSZLU\nwYRZkiRJ6mDCLEmSJHUwYZYkSZI6mDBLkiRJHUyYJUmSpA4mzJIkSVIHE2ZJkiSpgwmzJEmS1MGE\nWZIkSepgwixJkiR1MGGWJEmSOpgwS5IkSR1MmCVJkqQOJsySJElSBxNmSZIkqYMJsyRJktTBhFmS\nJEnqYMIsSZIkdTBhliRJkjqYMEuSJEkdTJglSZKkDibMkiRJUgcTZkmSJKmDCbMkSZLUwYRZkiRJ\n6mDCLEmSJHUwYZYkSZI6mDBLkiRJHUyYJUmSpA4mzJIkSVIHE2ZJkiSpgwmzJEmS1MGEWZIkSepg\nwixJkiR1MGGWJEmSOpgwS5IkSR1MmCVJkqQOJsySJElSBxNmSZIkqcOCEuYkj07yvSQXJDl0luMH\nJDknyblJTkuy60LPlSRJkoZs3oQ5yTLgHcDewC7A/kl2mfGyHwJ/XFX3BV4HHL0G50qSJEmDtZAR\n5t2BC6rqwqq6DjgO2Gf8BVV1WlVd2W6eDmy90HMlSZKkIVtIwrwVcPHY9iXtvrk8F/jsWp4rSZIk\nDcqGi/nFkuxFkzDvuRbnHggcCLDtttsuZliSJEnSWlvICPOlwDZj21u3+1aR5H7Au4F9quoXa3Iu\nQFUdXVXLq2r5lltuuZDYJUmSpHVuIQnzGcCOSXZIsjGwH3DC+AuSbAt8Anh6VX1/Tc6VJEmShmze\nkoyquiHJi4GTgGXAe6vqvCQHtcePAl4N3AH4jyQAN7SjxbOeu47+LZIkSdKiW1ANc1WdCJw4Y99R\nY58/D3jeQs+VJEmSpoUr/UmSJEkdTJglSZKkDibMkiRJUgcTZkmSJKmDCbMkSZLUwYRZkiRJ6mDC\nLEmSJHUwYZYkSZI6mDBLkiRJHUyYJUmSpA4mzJIkSVIHE2ZJkiSpgwmzJEmS1MGEWZIkSepgwixJ\nkiR1MGGWJEmSOpgwS5IkSR1MmCVJkqQOJsySJElSBxNmSZIkqYMJsyRJktTBhFmSJEnqYMIsSZIk\ndTBhliRJkjqYMEuSJEkdTJglSZKkDibMkiRJUgcTZkmSJKmDCbMkSZLUwYRZkiRJ6mDCLEmSJHUw\nYZYkSZI6mDBLkiRJHUyYJUmSpA4mzJIkSVIHE2ZJkiSpgwmzJEmS1MGEWZIkSepgwixJkiR1MGGW\nJEmSOpgwS5IkSR1MmCVJkqQOJsySJElSBxNmSZIkqcOCEuYkj07yvSQXJDl0luP3TPKNJNcmefmM\nYxclOTfJ2UlWLFbgkiRJ0iRsON8LkiwD3gE8CrgEOCPJCVX1nbGX/RJ4KfDEOb7MXlV1xc0NVpIk\nSZq0hYww7w5cUFUXVtV1wHHAPuMvqKrLquoM4Pp1EKMkSZLUm4UkzFsBF49tX9LuW6gCvpDkzCQH\nzvWiJAcmWZFkxeWXX74GX16SJEladyYx6W/Pqro/sDfwoiQPm+1FVXV0VS2vquVbbrnlBMKSJEmS\n5reQhPlSYJux7a3bfQtSVZe2f14GHE9T4iFJkiRNhYUkzGcAOybZIcnGwH7ACQv54kk2TXKb0efA\nnwLfXttgJUmSpEmbt0tGVd2Q5MXAScAy4L1VdV6Sg9rjRyW5M7AC2Az4Q5JDgF2ALYDjk4z+rg9V\n1efWzT9FkiRJWnzzJswAVXUicOKMfUeNff4zmlKNmX4N7HpzApQkSZL65Ep/kiRJUgcTZkmSJKmD\nCbMkSZLUwYRZkiRJ6mDCLEmSJHUwYZYkSZI6mDBLkiRJHUyYJUmSpA4mzJIkSVIHE2ZJkiSpgwmz\nJEmS1MGEWZIkSepgwixJkiR1MGGWJEmSOpgwS5IkSR1MmCVJkqQOJsySJElSBxNmSZIkqYMJsyRJ\nktTBhFmSJEnqYMIsSZIkdTBhliRJkjqYMEuSJEkdTJglSZKkDibMkiRJUgcTZkmSJKmDCbMkSZLU\nwYRZkiRJ6mDCLEmSJHUwYZYkSZI6mDBLkiRJHUyYJUmSpA4mzJIkSVIHE2ZJkiSpgwmzJEmS1MGE\nWZIkSepgwixJkiR1MGGWJEmSOpgwS5IkSR1MmCVJkqQOJsySJElSBxNmSZIkqcOGfQewPtj+0M8s\n+te86A2PXfSvKUmSpNU5wixJkiR1WFDCnOTRSb6X5IIkh85y/J5JvpHk2iQvX5NzJUmSpCGbN2FO\nsgx4B7A3sAuwf5JdZrzsl8BLgTetxbmSJEnSYC1khHl34IKqurCqrgOOA/YZf0FVXVZVZwDXr+m5\nkiRJ0pAtJGHeCrh4bPuSdt9CLPjcJAcmWZFkxeWXX77ALy9JkiStW4PpklFVRwNHAyxfvrx6Dme9\nZDcPSZKk1S1khPlSYJux7a3bfQtxc86VJEmSereQhPkMYMckOyTZGNgPOGGBX//mnCtJkiT1bt6S\njKq6IcmLgZOAZcB7q+q8JAe1x49KcmdgBbAZ8IckhwC7VNWvZzt3Xf1jJEmSpMW2oBrmqjoROHHG\nvqPGPv8ZTbnFgs6VJEmSpoUr/UmSJEkdTJglSZKkDibMkiRJUgcTZkmSJKmDCbMkSZLUwYRZkiRJ\n6mDCLEmSJHUwYZYkSZI6mDBLkiRJHUyYJUmSpA4mzJIkSVIHE2ZJkiSpgwmzJEmS1MGEWZIkSepg\nwixJkiR1MGGWJEmSOmzYdwDSmtr+0M8s+te86A2PXfSvKUmSlgZHmCVJkqQOJsySJElSB0sypHXA\nshFJkpYOR5glSZKkDibMkiRJUgcTZkmSJKmDCbMkSZLUwYRZkiRJ6mCXDGk9ZjcPSZLm5wizJEmS\n1MERZkmD50i4JKlPjjBLkiRJHUyYJUmSpA4mzJIkSVIHE2ZJkiSpgwmzJEmS1MGEWZIkSepgwixJ\nkiR1MGGWJEmSOpgwS5IkSR1MmCVJkqQOJsySJElSBxNmSZIkqYMJsyRJktTBhFmSJEnqYMIsSZIk\ndVhQwpzk0Um+l+SCJIfOcjxJ3tYePyfJbmPHLkpybpKzk6xYzOAlSZKkdW3D+V6QZBnwDuBRwCXA\nGUlOqKrvjL1sb2DH9uPBwDvbP0f2qqorFi1qSZIkaUIWMsK8O3BBVV1YVdcBxwH7zHjNPsCx1Tgd\n2DzJXRY5VkmSJGni5h1hBrYCLh7bvoRVR4/nes1WwE+BAr6Q5EbgXVV19NqHK0nDtf2hn1n0r3nR\nGx676F9TkrRmFpIw31x7VtWlSe4InJzk/Kr66swXJTkQOBBg2223nUBYkiRJ0vwWkjBfCmwztr11\nu29Br6mq0Z+XJTmepsRjtYS5HXk+GmD58uW1wPglSWvIkXBJWjMLqWE+A9gxyQ5JNgb2A06Y8ZoT\ngGe03TL2AH5VVT9NsmmS2wAk2RT4U+Dbixi/JEmStE7NO8JcVTckeTFwErAMeG9VnZfkoPb4UcCJ\nwGOAC4DfAc9uT78TcHyS0d/1oar63KL/KyRJkqR1ZEE1zFV1Ik1SPL7vqLHPC3jRLOddCOx6M2OU\nJEmSeuNKf5IkSVIHE2ZJkiSpgwmzJEmS1MGEWZIkSepgwixJkiR1MGGWJEmSOpgwS5IkSR1MmCVJ\nkqQOJsySJElSBxNmSZIkqcOClsaWJGnStj/0M4v+NS96w2MX/WtKWvocYZYkSZI6mDBLkiRJHUyY\nJUmSpA4mzJIkSVIHE2ZJkiSpgwmzJEmS1MGEWZIkSepgwixJkiR1MGGWJEmSOpgwS5IkSR1MmCVJ\nkqQOG/YdgCRJ02z7Qz+zqF/vojc8dlG/nqSbzxFmSZIkqYMJsyRJktTBhFmSJEnqYMIsSZIkdTBh\nliRJkjrYJUOSpCVusTt5gN08tH5xhFmSJEnqYMIsSZIkdbAkQ5IkDYKlIxoqR5glSZKkDibMkiRJ\nUgcTZkmSJKmDCbMkSZLUwYRZkiRJ6mDCLEmSJHWwrZwkSdIasP3d+scRZkmSJKmDCbMkSZLUwYRZ\nkiRJ6mANsyRJ0hJkrfXicYRZkiRJ6rCghDnJo5N8L8kFSQ6d5XiSvK09fk6S3RZ6riRJkjRk8ybM\nSZYB7wD2BnYB9k+yy4yX7Q3s2H4cCLxzDc6VJEmSBmshI8y7AxdU1YVVdR1wHLDPjNfsAxxbjdOB\nzZPcZYHnSpIkSYOVqup+QfJk4NFV9bx2++nAg6vqxWOv+TTwhqo6td3+IvD3wPbznTv2NQ6kGZ0G\n2Bn43s37p61mC+CKRf6ai20aYgTjXGzGubimIc5piBGMc7EZ5+IyzsUzDTHCuolzu6racr4XDaZL\nRlUdDRy9rr5+khVVtXxdff3FMA0xgnEuNuNcXNMQ5zTECMa52IxzcRnn4pmGGKHfOBeSMF8KbDO2\nvXW7byGv2WgB50qSJEmDtZAa5jOAHZPskGRjYD/ghBmvOQF4RtstYw/gV1X10wWeK0mSJA3WvCPM\nVXVDkhcDJwHLgPdW1XlJDmqPHwWcCDwGuAD4HfDsrnPXyb9kfuus3GMRTUOMYJyLzTgX1zTEOQ0x\ngnEuNuNcXMa5eKYhRugxznkn/UmSJEnrM1f6kyRJkjqYMEuSJEkdTJglSZI0NZI8aNJ/52D6MGuY\nkuzWdbyqzppULAuR5JbAtlW12AvfaGCSfAqYcxJGVT1hguFI0lRJsgGwR1Wd1ncsC5FkF2D/9uMq\nYKL9mJfspL8kdwKOAO5aVXu33+iHVNV7eg4NgCRvp/ti/9IJhjOnJF/uOFxV9YiJBTOPJI8H3gRs\nXFU7JLk/cPgQE6ckOwHvBO5UVfdJcj/gCVX1Tz2HtookAQ4A7lZVhyfZFrhzVX2z59BI8sddx6vq\nK5OKZSGS3B24pKquTfJw4H7AsVV1Vb+RNZLcvut4Vf1yUrEsVJI7A7vTvJeeUVU/6zmkm0zjDV2S\nrYDtGBtMq6qv9hdRtyTbAPtV1b/2Hcu4JK8D/rGqbmi3NwOOrKpn9xvZ6pJ8q6oe0Hccc0myPSuT\n5Otpfj6XV9VFE49lCSfMnwWOAV5ZVbsm2RD4VlXdt+fQAEjyzK7jVfW+ScWyVCQ5E3gEcMroDSDJ\nuUP5Px+X5CvA3wLvGov121V1n34jW1WSdwJ/AB5RVfdKcjvg81U18cdh0y7J2TQjItvTtOL8b+De\nVfWYPuMaSfJDmgQvsxyuqrrbhEPqlOR5wKuBL9HE/Mc0N8jv7TWw1tgN3ZOAOwMfaLf3B35eVX/d\nS2BzSPIvwFOA7wA3trtraIl9ki2BfWm+j3cFjq+ql/cb1aqSvB74U5oWu3cC/h14e1X9e6+BzSLJ\nm4BvAJ+ogSWESb4BbAYcBxxXVT9I8sOq2qGPeJZyScYWVfXRJIfBTT2hb5zvpEmZxoQ4yX2AXYBb\njPZV1bH9RbSa66vqV82g6E0G9QYw5lZV9c0Zsd7QVzAdHlxVuyX5FkBVXdkuQjQYSXYEXs/qP5uD\nSvCAP7TvQ39Oc/F8++j7OgR9XYRuhr8FHlBVvwBIcgfgNGAQCfPoCUeSN89YyvdTSVb0FFaXJwI7\nV9W1fQcyU5Lb0Nx4PBXYCfgEsENVbd1rYHOoqsOSfAH4H+BK4GFVdUHPYc3lBcDLgBuT/J7m5rOq\narN+wwLg58BWNDcdWwI/oMdr+lJOmH/bvoEWwGgFwn5DWl17t/z3rH6xH0ypA0CS1wAPp4nzRGBv\n4FRgSAnzeUmeCixrk6iX0lxAh+iK9hH96OfzycBP+w1pVtcnWcbKOLekGXEekmOA1wD/BuxFM6oz\nxAnN1yfZH3gm8Ph230Y9xjOn9knCjqz6njS0R/O/AK4e27663Tc0mya5W1VdCJBkB2DTnmOazYU0\nP4+DS5iBy4BvAq8CTq2qam88BynJw4C3AYcD9wXenuS5VfWTfiNbXVXdpu8Y5lJVT0xyW5qbpde2\n1/XNk+zeR1ngUi7J2A14O3Af4Ns0dydPrqpzeg1shiSfBz4CvBw4iOZienlV/X2vgc2Q5FxgV5qy\nll3bGvEPVNWjeg7tJkluBbyS5lFYaFaYfF1VXdNrYLNIcjeaFYv+iGYE4ofA0/qoy+qS5ACax7S7\nAe8Dngy8qqo+1mtgY5KcWVUPHC+/Ge3rO7Zx7TyKg4BvVNWH28TpL6vqX3oObRVtqcPBwNbA2cAe\nNDEP7Sb+WJpk5L9pbuj2Ac5pP6iqt/QX3UpJHk3zu34hzfvSdsALquqkXgNrjc2n2YrmPf6LjCXN\nQ5hPk+QQYD+aG40P01wzTx7gUyQAknwTeFZVfafdfhJwRFXds9/IVjc2T2WHqnpdWxd+lyHMU5kp\nyR1prkf70Uzu32aif/9STZgB2rrlnWnepL5XVdf3HNJqxi7251TV/dp9ZwytRjTJN6tq97ZOeC+a\n0ZzvDvENYJok2RTYoKqunvfFPUlyT+CRNL9HX6yq7/Yc0iqSnAbsCfx/NPWslwJvqKqdew2sQzuC\nu83QbuDhppvjBwGnV9X92///I6rqST2Htor2qdecquofJxXLfJJsAozeK88fUtnDNM2naQca9qOp\nX96R5snS8VX1/V4DmyHJsqq6cca+O4zKh4ZkWuepJNmuqn40yb9zyZZktHd043ZK8ivg3Kq6rI+Y\n5jBK4n+a5LHAT4DO2eo9WZFkc+A/gTOB39BMFOjdlM5GvxH4V+Cw0USLJGdVVWcbv0lqSzHOa2+K\nzu87ng4HA7eiKcF5Hc0NXWcS0IckpwBPoHnfPRO4LMnXq+plvQa2umuq6pokJNmkqs5PMribj/GE\nuL3IXzW0SUtw05OvlwHbVdXzk+yYZOeq+nTfscHKhLi9eb9mlOi1v/+b9BnbTG1ZyxHAEe2cmqfS\nlAjeo9fAVrdFkiOArarq0e3TpYcAg+jSNcNg56kkOYa5r+0FPHeC4SzdhJnmG/kQYNQW7eE0F6kd\nkhxeVe/vK7AZ/qmt0fkbmhKSzYBBzZ4GqKoXtp8eleRzwGYDGh17U98BrIXzaOpsP5/kKdW07Jqt\nO0FvqurGJN9Lsm1V/bjveOZSVWe0n/6Gpn55qG5bVb9uSx6OrarXJBnK79C4S9qb408CJye5Epjo\nSE6XJK8GPtom8psAnwXuD9yQ5KlV9YV+I1zNMTTXnoe025cCHwMGkTCP+SLwJzS/RwC3BD5PUzY2\nOFX1beAV7cfQ/Bdtl652+/s0ZSRDTJiHPE9ltt+RbWhypGUTjmVJJ8wbAveqqp/DTX2ZjwUeDHwV\nGETCPDbK8CuakbFBaidYfKmqflVVFyXZPMkTq+qTfcc2Nhv94Ko6cvxYkoOBQfXjbd1QVX+X5CnA\n15I8g2F29LgdzWTKbwK/He0c0qh9kpOBfavtZ9yONh5XVX/Wb2Sr2TDJXYC/ZOWFdHCqajSZ6rVp\n+rDfFvhcjyHN9BSaJwnQPEnYgGaOyk40dfZDS5jvXlVPaSd8UlW/a+tGh+YWVTVKlqmq37Sj473L\nypaHN+0a266quvvko+o06C5dM7wNOB64Y5J/pp2n0m9Ijar6+OjzthznFcDDgDfQw83HUk6Ytxkl\ny63L2n2/TDKYWuYk7wMOnnGxf3NVPaffyFbzmqo6frRRVVe1NYS9J8xjngkcOWPfs2bZNwQBqKqP\nJDkP+BCwbb8hzeof+g5gAbaoscU/2keKd+wzoDkcTjMR9dSqOqO9APyg55hWk6aj0HlVdXVVfSXN\nogsPoGmRNQTXjZVe/Bnw4baM4LvtvJWhuS7NCqSjEby7M8xOFL9Nslu1q7cmeSDw+55jGpm5otsG\nNDeeLwcG05pxzFR06QKoqg+2c5NG81SeOKR5Ku0cilfRvAf9K3BQtQvCTNoQ31wWyylJPk3z6Avg\nL9p9m9IsqTgU95vlYj/EVXdma9M1iJ+fduTmqTTlNieMHboNMLjVyVrPG31SVd9O8v9oZvkPSg1s\ntbw5/GG8bCTJdgxwtL7tLPKxse0Lad6XhuadNF1RRn4zy74+XdvWr/6c5qnc+KIVgxgRneE1NCP0\n2yT5IPBQmhv5oTkY+FiSn9AkTnemGc3vXa3stb0B8HSaHtxnA48ddaIYmJcBJwB3T/J12i5d/YbU\n6QfAr2mv6UMpw0vyMeCBwJtpyjBuBDYbPaCpCa8+OoiEZx15EU3vvj3b7RU0yxD/lmGVPmyQ5HZV\ndSXctDztEP9fViR5C/COdvtFNHV5Q3AaTQ/jLWh+sUaupm0xNRRJHlFVXwK2axO7cb+Z7Zw+tSMj\nbwfuBWxMUzf22xpGU/uRVwKnplk9McD/Aw7sN6TVJbkFzdyKe7Nqf+OhPU3K+OS5qvrDwEZuD6bp\niLIl8G9V9UOAJI9hgKONVXVykrNo2vOF5oniFT2HtYo2Ed2YppPHaILnYDpLJdkIeA5N0nQqzSjo\n4BYCSfIg4OKqOivNSo8voLkp/jxwSa/BzSHJS2hu6n5Ok5COyl3u12dcrQfRxPJymnlesHKuTwET\nbSu41NvKPYBm5HFfmj63H6+BLU3Z1q6+gmbkKTR3of88oEmJwE0zqP+BZlIIwMnAP7U3IFqgJP/Y\nTvY6ZpbDNbTkKc2KZPvR/HwuB54B7FRVh/Ua2AxJtqBJSKBphzaohARuGi05n+Y96XCa3qffraqD\new1shiSfAE6hGVUGeCGwV1U9sbegplg7yfzVY9sbAO+vqgN6DGs1Sb5VVUN8ukmSS2hWQn0rsNrI\nZ1V9YuJBzaK9MfqTtvTzYTRLOr+EZlLqvapqcKPMSS6g6ZQxuJZ3C5Xk3lV13jr/e5ZawpxkJ5oe\njfsDV9AuClJVM0fzBqNtOTNaFOBLA33ENFhJTq2qPZNczSwTQwY2GjpVkqyoquVZtU/4IC6sSe7Z\ndkqYtVRgVIs5FKPv2+h72Y6afa2q9pj35Alq67/fRvOeVDTdEw6pYbXjHC2F/Rqap4hFM/J4+NAu\n/O3N8fer6vVtV4+P0iwA9dp+I1tVkjfRtAr9xNDa8yX5Lzraiw1loCHJ/1bVru3n76BZhOy17fbZ\nVXX/PuObTTux91F91QUvhkyoJeuQHrMtlvOBrwGPGz2ySTK4Nm1JNmtbTN0e+BnNpK/RsdtPujZn\nLkneWlWHZI5exwPplrApDHuJz5na7h3H0JSN/CdNfeihVfX5XgNb3e/S9OQ8O8kbaUpfhrLs9Mto\nSi/ePMuxYuVN6FCMHm9f1dbg/gwY3OTENjHer+84FuA4mo5HozrwA2gGSP5kzjP68Rzgg23HhL2A\nz1bVv/Uc02xeQPM7dUOSaxjQgENVPWshr0vyzOp3oZVlSTZsk89Hsmpp2KDyrSSj/u8X0szv+gyr\nrvA4iJVK44YzAAAaUElEQVQyF2giXWcG9R+4SJ5E82b/5TT9go9jYP1tWx8CHkdTBzxbu5yhLPk5\nKg0Zcq/jQY2GLNBzqurIJH8G3IFmIsv7aWrdhuTpNAnyi2nqB7dhIBPVqurA9vH2q6rq633HswBH\nt11w/oFmQtCtgVd3nzI5Sf6uqt6YlUslr6IGsETyDHepqteNbf9TmjaNgzDjyceRwLuArwNfGe9G\nMRTTNODQ4WCa1oJ9+TDN/+8VNB1GvgaQ5B4Mr0vG6P/7x+3Hxu0HTN81dSLxLrmSjJG25nYfmtKM\nR9D0YD5+SCN4aaZ6bjOE2ahd0jQ1P3ZoNXcjbX3bnHfDQ7xTHnssfyRwSlUdP5RSBxjOLOmFGNL3\nbZoleXxVfSpzLJXc88jdatpJyN+kKXGAZv7H7lX18rnPmpz2UfdcqqqG9gRk1NZ0R1adlPrV/iJa\nM0N4L2gnSt+FZnnp37b7dgJuPbSbJIAk+7YdfDr3DdmkSjKWbMI8rn0T2Bd4SlU9su94xiU5t6ru\n23cc80lyKs1a89f1HctMSX5KM0Fp1icJNbaE7lC0dY1bATsAu9J0nzilqh7Ya2Ct8TegJB+vqkGM\nKs9myLWXsMqjz1kN6YauvTn+l6EknbMZm6sQmnKs0YIQy4DfDKGEYKR9ArJvVX2k71jmk2YFyoOB\nrWlatu0BfGOIif1cJpU4LSWzfc+m7fuY5PRJzAVZiiUZq2lbth3dfgzNWUkeVCuX9x2qC4Gvp+lz\nPL7i2xAu9j+tqsP7DmINPZdm5vSF1az8dQfGlnWe1KzfDuM3H0MpD5rLqPbyxiS/Z0C1l62pedRd\nzXLoD+07ji7TVDrQtuT7W5ra6qE7mKaN1+lVtVeaBSOO6DmmNTXE8stBSrI38BhgqyRvGzu0GU1H\nkt7NNaF7ZDRiP6mJ0+tFwjxwDwYOSPIjmkR0dLEfQg/Ecf/XfmzA8BKAqXuTrKo/AGeNbf8CGJ/d\n/376XSii5vh8cIaeQA3xCcc8zm5vjD/GqjfHQ2ndNVXdUYAvJHk5TdI8/v0cxMTuMddU1TVJSLJJ\n+z3eef7TBmUa5jIMxU9o1qd4AquuqXA1zXyVIZhtQvfIxCd2rxclGUOW1RevAKCqfjTpWBYiya2q\n6nd9xzFuSF1FFkvftXhJbmTlDdwtgdH/+dBGbwFI8gTgYe3mKVX16T7jmU2S99EsWnFVu3074M1D\naYk1koH3CE9ydDvhc7xGeHyhlUGVECT54Sy7q6oG9eQmyfE0T7kOoUlErgQ2qqrH9BrYmCR3ohn1\nvmtV7d22ZH1IVb2n59CmVtveMsBO7a7BLFgzNCbMA5BkT2DHqjomyZY0kwNme5PtTZKHAO+hiW3b\nJLsCL6iqF/Yc2pI0bTVkfUryBppHyR9sd+0PrKjhLa6y2k1Q3zdG0yjJ7sCPq+pn7fYzaTq3XAS8\ndqndPPchzSp1twU+N6R5K0k+S9OO85VVtWuaFSi/NQ3zgIaq/b8+lub3JzSdkJ45tMmebSvOXVh1\nQuqxE43BhLlfSV5Ds4LazlW1U5K7Ah+rqkHVESb5H5pZ6CeMLvBJvl1V9+k3sqXJhHnhkpwD3L8t\ncxlNXPvW0Mqakvwv8PB2TgVtD/avDO1in2RrmuXQR+9BX6MZGR/E0r6ZztXUer/YzyXNku0HAfcA\nzgXeUwNdxCLJGVX1oPEbzQx0QZBpkeRM4KlV9b12eyfgw0OZgA435UkPp/kdOhHYGzh10r/r1jD3\n78+BB9DWs1bVT5IMsiazqi5uOuHd5Ma5XqubbTCjOlNic2A0snjbPgPp8Gbg9CSjNmj7Av/cYzxz\nOYamT/y+7fbT2n2P6i2iVS0bG0V+CnB0VX0c+HiSs3uMa1ZzXexpRvWG4H00i+p8jSa2XWgmAA7R\nb9sJ0gU3tXAbWn/jabPRKFkGqKrvt2UaQ/Jkmm5S36qqZ7elOR+YdBAmzP27rqoqyegNYNO+A5rD\nxUn+CKj2l+lg4Ls9xzS1knyCpsTls6OR0XGTmvW7RLwe+FZb0xqaWuZD+w1pdVV1bJIVrJyo8qSq\n+k6fMc1hy6oar2P+rySH9BbN6qZmNbXWIC72HXYZPeVI8h6a3tZD9TKaRX/unuTrwJY031+tvRVJ\n3s3Kn8kDaCYDDsnv244zNyTZDLiMpnRkoob45rK++WiSdwGbJ3k+zTKq/9lzTLM5iGa1qq2AS2lW\npHtRrxFNt/+gmWDztiQfA44Zv8vXwlXVh5OcQlPHDPD3o/rWIZjlkfdRQ33k3fpFkqfRrFoGTU34\nLzpeP2nTtJoaDORi3+GmCV5VdcOMp4iDUlVntTW3O9PcHDtB7eb7K5pr+Wglz6/RXJ+GZEWSzWly\nozOB39D03p8oa5gHIMmjgD+leQM4qapO7jkkTUiS29IkJK8ELqZ5Q/iAF4GFm6O92K+AHw0hMU3y\nEVZ95H1RVQ1pxHYVbeeetwMPoXn0fRrw0hrQyo+ZotXUkvwH8ApgP+BvaC72Z1fVsztPnJCxjjiw\nalecwXXESfKkWXb/Cji3qi6bdDyavCTbA5tV1TkT/7tNmIehHXm4acR/aDO9k+xAM7Fme1aN8wl9\nxTTt2lq8pwFPp+mJ+UFgT+C+VfXwHkObKklOp+lZfQ7NRf4+wHk0tcx/VVWf7zG8VVbzbGf1f9MJ\nneunPi/2S0GSz9DcyI1aCj6cZsRxB+Dwqnp/T6FNrSSPA14HbEdzbR/ijdKfA1+qql+125vTTKD+\n5CTjsCSjZ0leAPwjcA3wB9ofVoa3utonaWpuP0UTp26GtufpzjQLlDy+qn7aHvpIW+eqhfsJ8Nxq\nV0Zse7MeDvwd8Ama8qE+Tc0jb4AZq36N/IqmVd9/TzqepaAdGd2T5r39VJqbO625DWk6ofwcburL\nfCzNAmBfpXk/1Zp5K/AkmlH6oY6gvqaqjh9tVNVV7WRaE+b1zMuB+1TVFX0HMo9rqmq2C6nWztuq\n6suzHaiq5ZMOZsrtVGPLiFfVd9rV4C4cSHK6a5Jft58HuGW7PbiRnNYtgHvSrPQHTY/jH9L8O/Ya\ncjnJELUlGfdgZU34C5L8SVU5B2TNbTNKlluXtft+mcQytrVzMfDtASfL0KwwPNPE81cT5v79HytX\nURuyI9s7us8D1452Dq1ecOjGa/Bmq8ergSw/PGXOS/JOmn680LQa+06STRgb3e1LVS3rO4Y1dD/g\noVV1I0D7vf0azQjpuX0GNqUeQTMqOuqE9D6akiGtuVOSfJpVb+ZOabtLXdVfWFPt74ATk3yFVa/t\nb+kvpNWsSPIW4B3t9otYdTnviTBh7t9hwGntwiDjP6wvnfuUXtyXptb2EawsyZj4Wu5LwOM7jhVN\nCYHWzLOAF9Is6QvwdZonN9cDe/UU0zS7HXBrVnac2BS4fVXdmOTauU/THC4AtgV+1G5v0+7TmnsR\nTfnAnu32CuBO7cRPf9fXzj/TTES9BbBxz7HM5SXAPwAfabdPpocuXSbM/XsX8CWakZsh1wbvC9xt\nSMukTqOhzIxfSqrq9+1j70/P0prvN33ENOXeCJzdtuob9bU+oh3F+0KfgU2TJJ+iuQm+DfDdJN9s\ntx/MsHsdD1a7ZsGFwB4016QfAh/vN6qpd9ehr9jb3hD13lvfLhk9G1/ic8iSfBI40NY9N0+Sp1XV\nB5K8bLbjA3sMNhWSPAH4V2Djqtohyf1pZszbwWUtJbkLsHu7eUZV/aTPeKZR2y94TlX1lUnFMu3a\nloH7tx9X0Iw0vryqtus1sCUgyRuBL/TdTWg2Sd5aVYeM3XyuYtLv8Y4w9++zSQ6k6T4xXpIxqLZy\nNEsPn5/kDFaN06RkzYxWchzk8udT6jU0yd0pAFV1dtsGUWshzUzJR9I8UTo8ybZJdq8qR0XXwMyE\neGbrUK2R82nq6B9XVRcAJPnrfkNaMv4KeHlbbnU9w5qMPOp68qZeo2g5wtyzJD+cZXdV1aDays01\nWuIoifqW5PSq2mP8aU2Sc6rqfn3HNo3aSX5/AB5RVfdKcjuaBUIeNM+pmkU7IHI4M1qHDu09fsiS\nPJFm4ZeHAp+jmeD77qryxngdS3Lv8S5EPfz9y4Bjq+qAvmIY8W63Z9PyC29ivLhcCGZRnZfkqcCy\nJDvSLPF6Ws8xTbMHV9VuSb4FUFVXJhnqZKBp8LdMR+vQwWoXqPhkW0e/D80E3zu2N3fHD7GcYAl5\nP83CUL1oJxtvl2TjvudQmTD3LMkzZttfVcdOOpYuSa5mZQ3RxsBGwG8H8thmGrkQzOJ5Cc3S4tcC\nHwJOolm5Smvn+nZUZ9QGbUv8Gb05pqV16OC1k78+BHyoffKxL/D39L840VI2hGb2FwJfT3ICK5dx\nn/icHxPm/o0/5rwFTe3gWTSrFw1GVd1Uc9vWOO5DM1NZa8eFYBbPY6vqlTRJMwBJ9mVlr1atmbcB\nx9OM4P0z8GSalk5aO9PSOnSqVNWVwNHth9adIdTt/l/7sQE9zv+xhnlg2jXSj6uqR/cdy3ympcPH\nELUlBDviQjA3W5Kzqmq3+fZp4ZLck+bmPcAXq+q7PYc0tdp2cqcyo3VoVb2vt6CkBRrSe2mSW1VV\nb09rHGEent8Cg6trnrEq3QbAcppJLFo7LgRzMyXZG3gMsFWS8dH6zYAb+olq+iV5f1U9naYzwcx9\nWnMbVdWsbSSlKdD72gtJHkJTwnhrYNskuwIvqKoXTjIOE+aezegvuAGwC/DR/iKa0/gKdTcAF9GU\nZWjtuBDMzfcTmpW+nsCqy6ReDdhyau3de3yjrWd+YE+xLAXT0jpU66EkDwXOrqrfJnkazQS/I6vq\nRwBVNYTSy7cCfwacAFBV/5vkYZMOwpKMns1o13YD8KOquqSveDQZLgSzeJJsVFXX9x3HtEtyGPAK\n4JY0k9RGk32uA46uqsP6im2aTUvrUK2fkpwD7ArcD/gv4N3AX1ZV58I7k5Tkf6rqwTNah/5vVe06\nyTgcYe7ZtLRra2fKP5/V26A9p6+YppwLwSye7ZO8nubpzC1GO01I1kxVvR54fZLXmxwvnmlpHar1\n1g3tkuP7AP9eVe9J8ty+g5rh4iR/BFSSjYCDgYnPqzBh7llbG/wvwB1pRnSGtMrOuP+mWWnpC8CN\nPceyFLym7wCWkGNovp//BuwFPJumvElroaoOa1t27ciqNyBf7S+q6ZPk76rqje3n+1bVx8aOHVFV\nr+gvOukmV7dPl54GPCzJBjRtY4fkIOBIYCvgUprJ8i+adBCWZPQsyQXA44c+Cz3J2VV1/77jkGZK\ncmZVPTDJuVV13/F9fcc2jZI8j2YEZ2vgbJr2kd+oKiekroHx7gIzOw0MqfOA1m9J7gw8FTijqr6W\nZFvg4UNbC2IIHGHu38+Hniy3Pp3kMVV1Yt+BLAVJ9gDeDtyLZiGYZbgQzNq6th0V+UGSF9OMQNy6\n55im2cE0/eFPr6q92hZzR/Qc0zTKHJ/Pti31oqp+BrxlbPvHDGwdiKGsjGvC3L8VST5Cs/LbeC3r\nJ/oLaVYHA69Ici1wPcMtHZkW/w7sR7O4xnLgGcBOvUY0vQ4GbkWzJPbraMoyntlrRNPtmqq6JglJ\nNqmq85Ps3HdQU6jm+Hy2bWmikpxaVXvOWMUXhnltH8TKuJZk9CzJMbPsrmmbTJfk3lV1Xt9xTIsk\nK6pqeZJzqup+7T4XglHvkhxPUwd+CE1f8Ctpegk/ptfApkySG2n66oeVnUdot29RVUOrE5UGadQl\no/c4TJiHLclh7ez1QbMmb80k+SrwJzQtfH4G/BR41qTb5CwFSU4G9q2qq9rt29Gslvln/UY2/dq2\nl7cFPmfPcEl9GMrKuJZkDN++wOATZqzJW1NPp+nk8GKaRTa2Af6i14im1xajZBmgqq5Mcsc+A5pm\nbX39eVV1dVV9JclmwAOA/+k5NEnrp0GsjGvCPHzTkoj6qGIBkmxbVT8eraJEs7z4P/YZ0xLwh9H3\nFSDJdvjzeHO8k2a1r5HfzLJPkiZlECvj2qt0+LzwLy2fHH2S5ON9BrKEvBI4Ncn7k3wA+Crgwhtr\nLzVWq1dVf8DBFUn9+TbNYl+98k1w+KZlhNn6xoUZ//90JbpFUFWfS7IbTb9ggEOq6orRcSekrrEL\nk7yUZlQZ4IXAhT3GI2n9NoiVcU2Yh+9j879k3Uvyxap65Fz7qmqP2c/UDF2tprSW2gT503Mcfj+W\nE6yJg4C3Aa+i+Rn9InBgrxFJWp8NYmVcu2T0LMmWwPNZvSH3INrKJbkFTY/bLwMPZ+UI6WY0M+fv\n2VNoU2meVlND6325JNiub3FNS+ceSVpMjjD377+BrwFfAG7sOZbZvICmH+tdgfEWLr+mWXxDa6Cq\nlvUdw3rIUYHFNS2deyQtATMWV9kY2IgeVsY1Ye7frarq7/sOYi5VdSRwZJKXVNXb+45HUu+mZV6F\npCWgqm4z+jxJgH1YOWdlYuyS0b9PJ5mGFbTem+RVSY4GSLJjksf1HZS0AE5IXVyO2EvqRTU+CUx8\nYSprmHvWPmrYlGbm5/UMtJY1yUeAM4FnVNV9ktwKOK2q7t9zaFrPzTchVYvLmnBJk5TkSWObGwDL\ngT+uqodMMg5LMno2/qhh4O5eVU9Jsj9AVf2ufTQi9WJsQuoW7XLY4xNSt+otsKVvEJ17JK03Hj/2\n+Q3ARTRlGRNlwtyTJPesqvPb/rGrmfQa6QtwXZJb0j6OTXJ3xvohSj1wQuo6MF/nnqo6op/IJK2P\nqurZfccAlmT0JsnRVXVgki/PcriqaqJrpM8nyaNo+rLuAnweeCjwrKo6pc+4JCekLq4kp9F07jmT\nsc49VeXKlJImbijtd02YtWBJ7kAzMzXA6eOrqUl9SbIp8NfAtu1N6I7AzlU110Im6pDkbOcmSBqK\nodzEmzD3rK3DfCGwJ025w9eAo6rqml4DmyHJ4VX16rHtDYD3V9UBPYYlOSF1kSX5J5rv34l9xyJJ\nQ7mJt61c/44F7g28nabu8t40S/kOzTZJDgNIsglwPPCDfkOSgGZC6htpusxQVb/DXsE3x8E07S5/\nn+TXSa5O8uu+g5K03hpE+11HmHuW5DtVtct8+/rWdsT4IHAusBfw2ar6t36jkm56XPdI4OtVtVs7\nIfXDVbV7z6FJkm6mobTftUtG/85KskdVnQ6Q5MHAip5jusmMLh5HAu8Cvg58JcluA+zmofXPa4DP\n0TwF+SDthNReI5pCU9i5R9J6YL72u0nuXVXnres4HGHuSZJzaWqWNwJ2Bn7cbm8HnD+UEeY5uniM\nDK6bh9ZPTki9+aatc48kASQ5q6pmvdFf1L/HhLkfSbbrOl5VP5pULNI0c0KqJK2/JrX6qJP+elJV\nPxr/AH5PM8I8+hiUJEck2Xxs+3btbHqpb05IXURJbpHkZUk+keTjSQ5pu/lI0hBNJGdyhLlnSZ4A\nvJlmtbLLaEoyvltV9+41sBlmu4Ob1GMQqYsTUhdXko8CVwMfaHc9Fdi8qvbtLypJmt2kchEn/fXv\ndTS1l1+oqgck2Qt4Ws8xzWZZkk2q6lqAdpnsTXqOSesxJ6SuM/eZMYfiy0m+01s0ktTtukn8JSbM\n/bu+qn6RZIMkG1TVl5O8te+gZvFB4ItJjmm3nw28r8d4pDfP2L6SZun2N9M8onOS2toZdOceSeuX\nJF+sqkfOta+q9phEHCbM/bsqya2BrwIfTHIZ8NueY1pNVf1LknNo+t0CvK6qTuozJq3fqmqvvmNY\nSmZ07jktySqde/qMTdL6p507cStgiyS3Y+WCVJsBW008HmuY+5HkHsCdgLNpJvxtABxAc3H6TFWd\n2WN40tRIcgTwxqq6qt2+HfA3VfWqfiObLnbukTQkSQ4GDqGZ4/WTsUO/Bv6zqv59ovGYMPcjyaeB\nw6rq3Bn77wscUVWP7yeyVSU5tar2bFfaGf9h6WWlHWkmJ6SuG0nuCNzUHaOqftxjOJLWU0leUlVv\n7zsO28r1504zk2WAdt/2kw9nTptCs9JOVW029nEbk2UNxLK2nRzghNSbK8kTkvwA+CHwFeAi4LO9\nBiVpffbeJK9KcjRAkh2TPG7SQZgw92fzjmO3nFgU8/MRhIZuNCH1uUmeC5yME1JvjlHnnu9X1Q40\n8xZO7zckSeux99J0wvijdvtSYOLrQDjprz8rkjy/qv5zfGeS5wFDql++Y5KXzXWwqt4yyWCkmZyQ\nuuimpXOPpPXD3avqKUn2B6iq37X99yfKhLk/hwDHJzmAlQnycmBj4M97i2p1y4Bbs3J2qjQ4VfVZ\nLBtYLFPRuUfSeuO6ttSuAJLcHbh20kE46a9n7UIl92k3z6uqL/UZz0xOntJQOSF1cdm5R9IQJXkU\n8CqaPvufBx4KPKuqTploHCbM6jJbBwJpCPzZXFzT0rlH0vonyR1o5lYEOL2qrph0DE7603weOf9L\npF54t7+4pqVzj6T1SJLDq+oXVfWZqvo08MskH5x0HNYwq1NV/bLvGKQ5OCF1cU1L5x5J65dtkhxW\nVa9vW4h+FPjWpINwhFnStBpNSL3NHB9aMyuSPH/mzgF27pG0fnkOcN8khwGfAk6pqtdOOghrmCVN\nJSekLq4kdwKOp+l3ulrnnqr6WV+xSVr/JBl/f98IeBfwdeA9AFV11kTjMWGWNI2c9LduDL1zj6T1\nQ5IvdxyuqnrExILBhFnSlEpye2vsJUmTYA2zpKlksixJS1+SI5JsPrZ9uyQTXxrbhFmSJElDtXdV\nXTXaqKorgcdMOggTZkmSJA3VsradHADtMtmbdLx+nbAPsyRJkobqg8AXkxzTbj8beN+kg3DSnyRJ\nkgYryd6sXHn45Ko6aeIxmDBLkiRJc7MkQ5IkSYOS5NSq2jPJ1cD46G5o+jBvNsl4TJglSZI0NJsC\nVNVt+g4E7JIhSZKk4RlUzbAjzJIkSRqaOyZ52VwHq+otkwzGhFmSJElDswy4NU3Ncu/skiFJkqRB\nSXJWVe3Wdxwj1jBLkiRpaAYxsjziCLMkSZIGJcntq+qXfccxYsIsSZIkdbAkQ5IkSepgwixJkiR1\nMGGWJEmSOpgwS5IkSR1MmCVJkqQO/z9OZ7ryOs6bMQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x8b76d01668>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "importance = pd.Series(xgb_model.feature_importances_)\n",
    "importance.index = training_vars\n",
    "importance.sort_values(inplace=True, ascending=False)\n",
    "importance.plot.bar(figsize=(12,6))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x8b76f6ff28>"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsYAAAHHCAYAAAC83J6NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYbFV97//3h8FZROU4IVMUUHFEVFSuAYw3ggPGK3Ge\nFefANSaKMaKYoDHRKOgVSRTFWX+KcQAVBxQ0qAckIKKRICqOODA5g9/fH2vXOXX6VA9Ad+1dfd6v\n5+nH3ruqi699uqs/e+21vitVhSRJkrSp26zvAiRJkqQhMBhLkiRJGIwlSZIkwGAsSZIkAQZjSZIk\nCTAYS5IkScASgnGS6yT5SpL/SnJOkpdPeE6SHJnkvCRnJdl9ZcqVJEmSVsYWS3jO74B9q+ryJFsC\npyY5sapOG3vOfsDO3ce9gDd1/ytJkiTNhEVHjKu5vDvcsvuYuyvIAcBx3XNPA7ZOcsvlLVWSJEla\nOUsZMSbJ5sDpwG2BN1bVl+c8ZVvg+2PHF3bnfjTfa26zzTa14447XqViJUmSpKvq9NNP/1lVrVns\neUsKxlV1JXDXJFsDxye5Y1V9/aoWleQg4CCA7bffnrVr117Vl5AkSZKukiTfXcrzrlJXiqq6GPgc\n8MA5D/0A2G7s+Nbdublff0xV7VFVe6xZs2holyRJkqZmKV0p1nQjxSS5LvAA4JtznvYR4Aldd4o9\ngUuqat5pFJIkSdLQLGUqxS2Bt3fzjDcD3l9VH0vyTICqOho4AdgfOA/4NfDkFapXkiRJWhGLBuOq\nOgu424TzR499XsBzlrc0SZIkaXrc+U6SJEnCYCxJkiQBBmNJkiQJMBhLkiRJgMFYkiRJAgzGkiRJ\nEmAwliRJkgCDsSRJkgQYjCVJkiTAYCxJkiQBS9gSegh2fNHHl/01L3jVg5b9NSVJkjS7HDGWJEmS\nMBhLkiRJgMFYkiRJAgzGkiRJEmAwliRJkgCDsSRJkgQYjCVJkiTAYCxJkiQBBmNJkiQJMBhLkiRJ\ngMFYkiRJAgzGkiRJEmAwliRJkgCDsSRJkgQYjCVJkiTAYCxJkiQBBmNJkiQJMBhLkiRJgMFYkiRJ\nAgzGkiRJEmAwliRJkgCDsSRJkgQYjCVJkiTAYCxJkiQBBmNJkiQJMBhLkiRJgMFYkiRJAgzGkiRJ\nEmAwliRJkgCDsSRJkgQYjCVJkiTAYCxJkiQBBmNJkiQJMBhLkiRJgMFYkiRJApYQjJNsl+RzSb6R\n5JwkB094zt5JLklyZvfx0pUpV5IkSVoZWyzhOVcAf11VZyS5IXB6kpOq6htznndKVT14+UuUJEmS\nVt6iI8ZV9aOqOqP7/DLgXGDblS5MkiRJmqarNMc4yY7A3YAvT3j4PknOSnJikt3m+fqDkqxNsvai\niy66ysVKkiRJK2XJwTjJDYAPAodU1aVzHj4D2L6q7gwcBXx40mtU1TFVtUdV7bFmzZqrW7MkSZK0\n7JYUjJNsSQvF76qqD819vKourarLu89PALZMss2yVipJkiStoKV0pQjwFuDcqnrtPM+5Rfc8ktyz\ne92fL2ehkiRJ0kpaSleK+wKPB85OcmZ37sXA9gBVdTTwCOBZSa4AfgM8qqpqBeqVJEmSVsSiwbiq\nTgWyyHPeALxhuYqSJEmSps2d7yRJkiQMxpIkSRJgMJYkSZIAg7EkSZIEGIwlSZIkwGAsSZIkAQZj\nSZIkCTAYS5IkSYDBWJIkSQIMxpIkSRJgMJYkSZIAg7EkSZIEGIwlSZIkwGAsSZIkAQZjSZIkCTAY\nS5IkSYDBWJIkSQIMxpIkSRJgMJYkSZIAg7EkSZIEGIwlSZIkwGAsSZIkAQZjSZIkCTAYS5IkSYDB\nWJIkSQIMxpIkSRJgMJYkSZIAg7EkSZIEGIwlSZIkwGAsSZIkAQZjSZIkCTAYS5IkSYDBWJIkSQIM\nxpIkSRJgMJYkSZIAg7EkSZIEGIwlSZIkwGAsSZIkAQZjSZIkCTAYS5IkSYDBWJIkSQIMxpIkSRJg\nMJYkSZIAg7EkSZIEGIwlSZIkYAnBOMl2ST6X5BtJzkly8ITnJMmRSc5LclaS3VemXEmSJGllbLGE\n51wB/HVVnZHkhsDpSU6qqm+MPWc/YOfu417Am7r/lSRJkmbCoiPGVfWjqjqj+/wy4Fxg2zlPOwA4\nrprTgK2T3HLZq5UkSZJWyFWaY5xkR+BuwJfnPLQt8P2x4wvZODyT5KAka5Osveiii65apZIkSdIK\nWnIwTnID4IPAIVV16dX5j1XVMVW1R1XtsWbNmqvzEpIkSdKKWFIwTrIlLRS/q6o+NOEpPwC2Gzu+\ndXdOkiRJmglL6UoR4C3AuVX12nme9hHgCV13ij2BS6rqR8tYpyRJkrSiltKV4r7A44Gzk5zZnXsx\nsD1AVR0NnADsD5wH/Bp48vKXKkmSJK2cRYNxVZ0KZJHnFPCc5SpKkiRJmjZ3vpMkSZIwGEuSJEmA\nwViSJEkCDMaSJEkSYDCWJEmSAIOxJEmSBBiMJUmSJMBgLEmSJAEGY0mSJAkwGEuSJEmAwViSJEkC\nDMaSJEkSYDCWJEmSAIOxJEmSBBiMJUmSJMBgLEmSJAEGY0mSJAkwGEuSJEmAwViSJEkCDMaSJEkS\nYDCWJEmSAIOxJEmSBBiMJUmSJMBgLEmSJAEGY0mSJAkwGEuSJEmAwViSJEkCDMaSJEkSYDCWJEmS\nAIOxJEmSBMAWfRewmuz4oo8v6+td8KoHLevrSZIkaX6OGEuSJEkYjCVJkiTAYCxJkiQBBmNJkiQJ\nMBhLkiRJgMFYkiRJAgzGkiRJEmAwliRJkgCDsSRJkgQYjCVJkiTAYCxJkiQBBmNJkiQJMBhLkiRJ\ngMFYkiRJApYQjJO8NclPk3x9nsf3TnJJkjO7j5cuf5mSJEnSytpiCc95G/AG4LgFnnNKVT14WSqS\nJEmSerDoiHFVfQH4xRRqkSRJknqzXHOM75PkrCQnJtltviclOSjJ2iRrL7roomX6T0uSJEnX3HIE\n4zOA7avqzsBRwIfne2JVHVNVe1TVHmvWrFmG/7QkSZK0PK5xMK6qS6vq8u7zE4Atk2xzjSuTJEmS\npugaB+Mkt0iS7vN7dq/582v6upIkSdI0LdqVIsl7gL2BbZJcCBwGbAlQVUcDjwCeleQK4DfAo6qq\nVqxiSZIkaQUsGoyr6tGLPP4GWjs3SZIkaWa5850kSZKEwViSJEkCDMaSJEkSYDCWJEmSAIOxJEmS\nBBiMJUmSJMBgLEmSJAEGY0mSJAkwGEuSJEmAwViSJEkCDMaSJEkSYDCWJEmSAIOxJEmSBBiMJUmS\nJMBgLEmSJAEGY0mSJAkwGEuSJEmAwViSJEkCDMaSJEkSYDCWJEmSAIOxJEmSBBiMJUmSJMBgLEmS\nJAEGY0mSJAkwGEuSJEmAwViSJEkCDMaSJEkSYDCWJEmSAIOxJEmSBBiMJUmSJMBgLEmSJAEGY0mS\nJAkwGEuSJEmAwViSJEkCDMaSJEkSYDCWJEmSAIOxJEmSBBiMJUmSJMBgLEmSJAEGY0mSJAkwGEuS\nJEmAwViSJEkCYIu+C9B07fiijy/7a17wqgct+2tKkiRNm8FYg2SAlyRJ0+ZUCkmSJIklBOMkb03y\n0yRfn+fxJDkyyXlJzkqy+/KXKUmSJK2spYwYvw144AKP7wfs3H0cBLzpmpclSZIkTdeiwbiqvgD8\nYoGnHAAcV81pwNZJbrlcBUqSJEnTsBxzjLcFvj92fGF3TpIkSZoZU118l+SgJGuTrL3oooum+Z+W\nJEmSFrQcwfgHwHZjx7fuzm2kqo6pqj2qao81a9Ysw39akiRJWh7LEYw/Ajyh606xJ3BJVf1oGV5X\nkiRJmppFN/hI8h5gb2CbJBcChwFbAlTV0cAJwP7AecCvgSevVLGSJEnSSlk0GFfVoxd5vIDnLFtF\nkiRJUg/c+U6SJEnCYCxJkiQBBmNJkiQJMBhLkiRJgMFYkiRJAgzGkiRJEmAwliRJkgCDsSRJkgQY\njCVJkiTAYCxJkiQBBmNJkiQJMBhLkiRJgMFYkiRJAgzGkiRJEmAwliRJkgCDsSRJkgQYjCVJkiQA\ntui7AGmW7fiijy/7a17wqgct+2tKkqTFOWIsSZIkYTCWJEmSAIOxJEmSBBiMJUmSJMBgLEmSJAEG\nY0mSJAkwGEuSJEmAfYylTYL9liVJWpwjxpIkSRIGY0mSJAkwGEuSJEmAwViSJEkCDMaSJEkSYDCW\nJEmSAIOxJEmSBBiMJUmSJMBgLEmSJAEGY0mSJAkwGEuSJEmAwViSJEkCDMaSJEkSAFv0XYAkjez4\noo8v+2te8KoHLftrSpJWJ0eMJUmSJBwxlqSrzJFtSVqdHDGWJEmSMBhLkiRJgMFYkiRJAgzGkiRJ\nErDEYJzkgUm+leS8JC+a8PjeSS5Jcmb38dLlL1WSJElaOYt2pUiyOfBG4AHAhcBXk3ykqr4x56mn\nVNWDV6BGSZIkacUtZcT4nsB5VXV+Vf0eeC9wwMqWJUmSJE3XUoLxtsD3x44v7M7NdZ8kZyU5Mclu\nk14oyUFJ1iZZe9FFF12NciVJkqSVsVyL784Atq+qOwNHAR+e9KSqOqaq9qiqPdasWbNM/2lJkiTp\nmltKMP4BsN3Y8a27c+tU1aVVdXn3+QnAlkm2WbYqJUmSpBW2lGD8VWDnJDsluRbwKOAj409Icosk\n6T6/Z/e6P1/uYiVJkqSVsmhXiqq6IslzgU8CmwNvrapzkjyze/xo4BHAs5JcAfwGeFRV1QrWLUmS\nJC2rRYMxrJseccKcc0ePff4G4A3LW5okSZI0Pe58J0mSJGEwliRJkgCDsSRJkgQscY6xJGm27Pii\njy/7a17wqgct+2vOSp2SNg2OGEuSJEk4YixJ0qIc2ZY2DY4YS5IkSRiMJUmSJMCpFJIkrRqzMuVj\nVurUpscRY0mSJAmDsSRJkgQYjCVJkiTAYCxJkiQBBmNJkiQJMBhLkiRJgO3aJEmSJrKt3KbHEWNJ\nkiQJg7EkSZIEGIwlSZIkwGAsSZIkAS6+kyRJmmnLvUhwU14g6IixJEmShMFYkiRJAgzGkiRJEmAw\nliRJkgCDsSRJkgQYjCVJkiTAYCxJkiQBBmNJkiQJMBhLkiRJgMFYkiRJAgzGkiRJEmAwliRJkgCD\nsSRJkgQYjCVJkiTAYCxJkiQBBmNJkiQJMBhLkiRJgMFYkiRJAgzGkiRJEmAwliRJkgCDsSRJkgQY\njCVJkiQAtui7AEmSJK1uO77o48v+mhe86kHL/pqOGEuSJEksMRgneWCSbyU5L8mLJjyeJEd2j5+V\nZPflL1WSJElaOYsG4ySbA28E9gPuADw6yR3mPG0/YOfu4yDgTctcpyRJkrSiljJifE/gvKo6v6p+\nD7wXOGDOcw4AjqvmNGDrJLdc5lolSZKkFZOqWvgJySOAB1bV07rjxwP3qqrnjj3nY8CrqurU7vgz\nwAurau2c1zqINqIMsCvwreX6P9LZBvjZMr/mSrDO5WWdy2cWagTrXG7Wubxmoc5ZqBGsc7ltynXu\nUFVrFnvSVLtSVNUxwDEr9fpJ1lbVHiv1+svFOpeXdS6fWagRrHO5WefymoU6Z6FGsM7lZp2LW8pU\nih8A240d37o7d1WfI0mSJA3WUoLxV4Gdk+yU5FrAo4CPzHnOR4AndN0p9gQuqaofLXOtkiRJ0opZ\ndCpFVV2R5LnAJ4HNgbdW1TlJntk9fjRwArA/cB7wa+DJK1fyglZsmsYys87lZZ3LZxZqBOtcbta5\nvGahzlmoEaxzuVnnIhZdfCdJkiRtCtz5TpIkScJgLEmSJAEGY0nSDElyj75rkLR6TbWP8UpIcl1g\n+6pa7s1CJA1Ako8C8y6GqKqHTrGciZLsvtDjVXXGtGpZjZLcAXh093ExMPg+rEOUZDNgz6r6Ut+1\nSEM104vvkjwE+BfgWlW1U5K7AocP4Q/luCQ3B44AblVV+3Vv8veuqrf0XNpGkuwCvAm4eVXdMcmd\ngYdW1T/0XNoGktwGuLCqfpdkb+DOtG3JL+63sg0luQVtW/UCvlpVP+65pHWS3GShx6vqF9OqZSFJ\n/nShx6vq89OqZT5JPrfAw1VV+06tmEUkOYqFLzT+aorlzCvJjqwPw38AdgD2qKoL+qtqY7Nw4TYu\nydeq6m5917GQJAEeC/xJVR2eZHvgFlX1lZ5L20CSVwAvr6oruuOtgNdXVV+duZYkyXbAo6rqn/uu\nZa4k29J+19cN3FbVF6Zaw4wH49OBfYGTR7/oSc6uqjv1W9mGkpwIHAv8XVXdJckWwNeGVidAks8D\nfwO8eex7+vWqumO/lW0oyZm0UaMdae0C/wPYrar277OucUmeBrwU+CwQ4E9pF25v7bWwTpLv0P6g\nZ8LDVVV/MuWSNAVJnrjQ41X19mnVMp8k/wlsBbwXeG9VfTvJd6pqp55L28jYhdvDgVsA7+yOHw38\npKr+by+FzSPJvwD/CXyoBhoAkrwJ+COwb1XdPsmNgU9V1aCm0SR5JfC/aS1qbw68ATiqqt7Qa2ET\nJFkDHEj7ubwVcHxVvaDfqjaU5J+ARwLfAK7sTte0Ly5nfSrFH6rqknZxuc4Qf9G3qar3JzkU1vWG\nvnKxL+rJ9arqK3O+p1f0VcwC/th9H/+C9kZ0VJKv9V3UHH8D3K2qfg6Q5KbAl4BBBOMhhoyFJNkZ\neCVwB+A6o/NDC/BJ7sjGNR7XX0UbGkLwXYKfANvSwsYa4NsM87193R2LJK+Zs4XtR5Os7amshTwD\neD5wZZLf0C6Mq6q26resDdyrqnYfvadX1S+7DcYGpaoOTfJp4MvAL4H7VdV5PZe1TpIb0i7YHgPs\nAnwI2Kmqbt1rYfN7GLBrVf2uzyJmPRifk+QxwObdH82/ogWPoflVF4oKYLQ7YL8lzetn3TSFUa2P\nAIa4i+EfkjwaeCLwkO7clj3WM8nPgcvGji/rzg1ONyKzMxuGuanevlqCY4HDgH8F9qGN0gxqAXGS\nw4C9acH4BGA/4FRgMMF4pBtBeiEbh/jep31U1cOS3Ij2R/1l3fv71knuObTb6WOun+RPqup8gCQ7\nAdfvuaaNVNUN+65hCf6QZHPW/x1aQxtBHpQk9wOOBA4H7gQcleSpVfXDfitb56fAV4CXAKdWVXWD\nSUN1Pu3veK/BeNanUlwP+DvarYzQdud7RVX9ttfC5ugW5hwF3BH4Om0E5BFVdVavhU2Q5E9oO87c\nh3YF/B3gcQOc13cH4JnAf1bVe7o/Qn9ZVf/Uc2nrJDmO9mb5H7Q3+AOAs7oPquq1/VW3Xjfl42Dg\n1sCZwJ6072vvAWlcktOr6u7j06VG5/qubSTJ2cBdaFOl7tKtL3hnVT2g59I2kuRTwPuAF9B+l54I\nXFRVL+y1sAmS3Ix2i/VRtMXW2/Vc0kaSPJD23nk+7e/RDsAzquqTvRY2x9j83Z2q6hXdfNNbDumC\nI8ljaf/euwNvBx4BvKSqPtBrYXMk+QrwpKr6Rnf8cOCIqrpdv5U1SQ6h/c5cH3gP7ff9pAHeZRut\ne9iW9v75GcbC8bTXPcx0MJ4l3bziXWlvmN+qqj/0XNKCklwf2KyqLlv0yT3rRju3G9qFRjd6OK+q\nevm0allIF+buAZxWVXdNcjvam/vDey5tA0m+BOwF/H+0eds/AF5VVbv2WtiYJF+pqnt26x/2od0l\nOHcofyjHjV1onFVVd+7OfXVo8zjnSrJDVX237zomSXJtYPRv/c2+bwlPMkPzd28H3J/2N/MzVXVu\nzyVtJMnmVXXlnHM3HU2fG4puwOtRtPnFO9PuvB1fVf/da2Gdoa17mMmpFDO4CnhuwNglySXA2VX1\n0z5qmk839/mfgUNHCzOSnFFVC7ajmrYkJwMPpf0Mnw78NMkXq+r5vRY2Zjz4dn98Lh7oYpffVtVv\nk5Dk2lX1zSSDCZtjDgauR5sy9Qpa8FzwDbUHa5NsDfwb7efyctpCpyEaXZz/KMmDgB8CC3YqmZYk\nxzL/e3wBT51iOUvS3cF8PrBDVT09yc5Jdq2qj/Vd2xyDnr/bTaE4p7uY/Gbf9SximyRHANtW1QO7\nO5n3BgbVcaqb3nMEcES3BuIxtKlet+21sM4o+HYDcr8dXWx0PwvXnnY9MxmMaS3aZslTab8so5ZO\ne9P+aO6U5PCqekdfhU1wDm3e5qeSPLJay65JXQv6dqOqurSbBnBcVR2WZBAjxkleCry/C5jXBk4E\n7gpckeQxVfXpfivcyIVdmPswcFKSXwKDG5Grqq92n15Om188OFX17O7To5N8AthqaHcyxvxDN4/3\nr2lTvbYChtJBYVKY3I5W3+ZTrmWpjqW9r9+7O/4B8AEm/3/p06Dn71bVlUm+lWT7qvpe3/Us4m10\nHae64/+mTVcYVDAeV1VfB17cfQzNZ4A/o73HA1wX+BRtaufUzGQwHlsFfHBVvX78sSQHA733NZ1j\nC+D2VfUTWNfX+DjgXsAXgCEF4yuq6m+TPBI4JckTGOZq8C2S3BL4S9a/KQ3FI2kjmtBGNDejzSvf\nhTZfblDBuKpGizFeltaP90bAJ3osaaIkJwEHVteruhuFf29V/Xm/la3XLWz5bFVdUlUXJNk6ycOq\n6sN91zbX2EjmJbTR98Goqg+OPu9uA78YuB/wKoYbOm5TVY/sFgVTVb/u5vMOzZHA8cDNkvwj3fzd\nfkvayI1pi+u/AvxqdHJod4MZeMeprG/Jue7U2HFV1W2mX9WCrlNVo1BMVV3e3YmZqpkMxmOeCLx+\nzrknTTjXt+1Gobjz0+7cL5IMba5xAKrqfUnOAd4NbN9vSRMdTltseWpVfbX74/ntnmsa+f3YlIk/\nB97T3Ro6t5trPihpXVLOqarLqurzaU3q70ZrQTQk29TYBi7dLeCb9VnQBIdV1fGjg6q6uJtrPrhg\nnOTtwMFzLjReU1VP6beypptj+hLaz+I/A8+sbiOFgfp92k6so5HY29Dz6vpJqupd3Rz40fzdhw1w\n/u7f913AEg2949TcHSI3ow0mvQAYWntTaN/P3avbKTTJ3YHfTLuIwf2RXoruivwxtKkIHxl76IbA\nIHbrmuPkJB+j3VYD+D/duevTtjcdkqeNPqmqryf5X7RuCoPSrU7+wNjx+bTv6xD8rpvH9RPaSNx4\nE/WpX/0uwZtoq79HLp9wbgj+OH57NckODO9uxqT2cUN9n73zhAuNQeyIluQDwN2B19CmT1wJbDUa\ngK2B7Mo4x2G0Oy3bJXkXcF/aQM0QfRu4lO5nc2jTFmoAu1ku0fOBjwC3SfJFuo5T/Za0Xq3vob8Z\n8Hhab/0zgQeNOmkMzMHAB5L8kHbRdgvaHdipGuob9mK+ROutuw3tjXPkMrpWWAPzHFo/zr2647W0\nLZd/xUBuYSbZt6o+C+zQBY5xl0/6mj4luQ5t7vZubNiDdQijXQfTOiesAf61qr4DkGR/hnmVnvFF\ngVX1xyGObNOmzJyatjtjgP8FHNRvSRtZm+S1wBu74+fQ5p0O0WZJblxVvwRGW4QP5d/9HrSLnhfQ\n5kDD+rUOBQyq3RRAVZ2U5Axau8PQRuN/1nNZG0nyPFqI/wntgmN0e/3OfdY1rht5PQq4PXAt2rzy\nX9VANiFJcg/g+1V1RtrOh8+gDcx8Criw1+LGJNkSeArt4vJU2t2BwWxAMq4L79eidXUZLf7upYOX\n7dqmpBuJeQxtS8bvAB+sAW0bmeTl3QK2Yyc8XAMJnOt0I0rfpH1PD6f15Ty3qg7utbAZlORDwMm0\nUWKAZwP7VNXDeitqHkm2oQUPaO3lBhU8urtAf09bQAJwEvAP3UXwoHTrB15Mu/MS2kjXPw5sMfCC\nkuxWVef0XQdAt5D6pWPHmwHvqKrH9ljWRpKcR+tMMaiWYuPSdgx8FO1ncw/gCcAuVXVor4V1ugug\nP+umQ96PtnX582iLrG9fVYMYNU5yIW3n2tcBG90RqKoPTb2oBST5WlX1ftdqJoNxklOraq8klzFh\nYvmArip3ofUNfDTwM7pm+lU1d0RWV9HoFyhdD9buyviUqtpz0S+ekm7u2WG0OwVFu2I/fGh/kLp5\nukcC+9Lq/AxwyFBaCSa5XdfhY+LUjtF8NF11XXup0UYunx3o7dV5ZUCtJLtBhf+uqld23WjeT9vo\n5WX9VrahboHtA4Y8XzvJ2qraIxv22B5EaAJI8l9VdZfu8zfSNsZ5WXd8ZlXdtc/6RpK8jQXaHg5w\nwOtfaO0tP1Q9htOh3Da7qq4PM7G15TeBU4AHj25fJBlKO6SJuq4ex9KmpfwbbZ7pi6rqU70WtrHR\n7ZWLu/m8PwaGthDrvbSuI6O5z4+lXRz92bxf0YMuAD+q7zoW8HzalInXTHisWB/sepPkdVV1SObp\nsT6k1fRJtupaHd6E9nvz7rHHbjLQ+bvzGVLXh6cA7+o6FOwDnFhV/9pzTeskGfV4P5+2xuXjbLi7\n2CB24uz8Oq238plJXk2bOjmk7d83T7JFd3Fxfzac0jWYXFVVT1rK85I8saa8icY8nkF7v78iyW/p\nabBzMP+AV9GsDHM/nBY4PpfW0/S9DOuNfJKnVNXrk/w5cFPahP130OZODckx3Sr6v6ctfrgB8NKF\nv2TqbllVrxg7/oe0NniDkORvq+rVWb8d5wZqyttwzqeqDupuS7+kqr7Ydz3zGE0/mIUe6+8GHkyb\n+zypldPg5u8uoPe/BXPuZLweeDPwReDz4yvsB2A0kPS97uNa3QcM4Ps4x+NpQfi5tPmx2zGcxdXQ\ntlf+fJKf0bomnAKQ5LYMqyvFUh1MayXaq6EMds7qVIoLgXmvbgd25Tuad3gAbUrFvrQexscPcBSW\nsakJrwdOrqrjh3QLa5Z0i7C+QrulCm0O5z2r6gXzf9X0JHlIVX0082zHOZARhHWG/nOYtmnCcUOb\nUzpJWnuH7YbUieDqGMJUim5qwnyqqnq/ozEuyYFdV58Fz/VhaN0xFtItELwlbTvtX3XndgFuMKCL\noSUZ0ntrN+C1Mxsuqv/CVGuY0WD8I9pCoYmjrzW2Fe/QdP/oBwKPrKr7913PXN08uW2BnYC70FYD\nn1xVd++1sM7Y7cCJhnBRNDb3PbRpP6OG75sDlw9lDjysC3P/NJSwvpChzD9bSJJTgX2r6vd917KY\nJGdX1Z1o3z7CAAAYiklEQVT6ruOaSHLaENYVdHc0Dqyq9/Vdy2ImXUwM4QJjbh1JPlhVQxolXrUG\n9O//NNro9a1pbeX2BP5z2heXszqV4kdVdXjfRVwdXWukY7qPIXoqbWXt+dV2bropY9vvDmAV+CBu\ntSxkKLeDlqLa9qv37buOJRrNP7syyW8Y2GLbzvnAF9P6q4/v2NX7BdsEZyS5R63fansw5ltoOTIa\nkRtCKIZ1LQ7/hraGYJCS7AfsD2yb5Mixh7aidS4YgvHBrlma0jPrhjLF82Baq8bTqmqftE1+jph2\nEbMajIfyj7jqVNUfgTPGjn8OjHdReAc9bvww5LsBIzPYReHMLsh9gA3D3KBa+czIBcf/dB+bMfyL\nuHsBj03yXdq/++hCYwj9bCcttBwZxILLCT6d5AW0cDz+ezSUxYw/pPXQfygb9ta+jDaPdwhqns+1\nsoayduO3VfXbJCS5dvd3dNfFv2x5zepUillbOb1qDGUuUga8nW2SY7oFY+NzD8c30BjUH/XMSO9q\ngCQPBe7XHZ5cVR/rs575JLleVf267zoWko038gGgqr477VpWgyTfmXC6qmpQI59da8sAu3SnetlE\nYZIkV7L+Iu26wOh3aIh3h2ZGkpvTRl5vVVX7dW0a711Vb+m5tA0kOZ52h/oQ2sXvL4Etq2r/qdYx\ni8FY/RnQXKSNAvqAQvs9ge9V1Y+74yfSVlRfALzMi7qrJ8mraLfZ3tWdejSwtgbS9B8gyb2Bt9AW\n4Gyf5C7AM6rq2T2XNlGSvYCdq+rYJGtodU8KeL3p2jHegQ0X4xzXX0WzLW2ntuNo70ehdXx44rQX\nOGl6kpxIa8P6d1V1l7SdTb825DUG3c/pjYBPTHvNhsFYV8mAgvF/AXvXhtvZfn4Iv+iZkV2RRpLc\nmrb96miu8Sm00fjBbG0KrWMKcNduus9o4eDXBnLrH4AkX6Z1H/nI6CItyder6o79VraxJIfRdhXb\ntap2SXIr4ANVNZg5512Ne9OC8QnAfsCpQ/sdGpmFEJ/kdOAxVfWt7ngX4D1DWWCt5Zfkq1V1j/HB\nowxrI5LrAM8EbgucDbyletyAZlbnGKs/Q1lt/xrgtCSjVmgHAv/YYz3jNh8bFX4kcExVfRD4YJIz\ne6xrPsfSetse2B0/rjv3gN4qmt/WwOh7e6M+C5lPVX2/dUNb58r5ntuzvwDuRremoKp+mGRo86If\nQeuO87WqenJ3S/idPdc00XwhnjY6OyRbjkIxQFX9dze9QqvXr7qF9AXrWs0Nqd/y22mbdp1C+725\nA20hXi8MxtpAkg/RbgWfOBqZGzegVeDHJVnL+kU4D6/hbGc7E7sijVlTVePzjN+W5JDeqpnfK4Gv\ndXO3Q5tr/KJ+S9rI95PcB6gubBwMnNtzTfP5fVVVktEfy+v3XdAEv+k6PlyRZCvgp7Rb/0M0KyF+\nbZJ/Z31tj6UtytPq9XzaRli3SfJFYA3t53Uo7jC625vkLbT+/70Z4h9p9ev/0Sa/H5nkA8Cx46ML\nfZtwy+XoPm+5zGPWdkX6eZLH0eqGNnf35ws8vxdV9Z4kJ9PmGQO8cDSPe0CeSdv9bFvgB7QdI5/T\na0Xze3+SNwNbJ3k6bUvjf+u5prnWJtmaVtfpwOW0XtZDNCsh/lm0n8nRzpan0N73tUpV1RndnN1d\naYMKg1lw2VlXS1VdMeeO29Q5x1gTJbkRLSD9HfB92h+md/b9y5TkfWx4y+WCqhrc6GZmaFekrjvB\nUcC9abfavgT8VQ1sB6p52t9dAnx3gBdHMyHJA4D/Tftj+cmqOqnnkuaVZEdgq6o6q+dSJkry/4AX\nA48C/poW4s+sqicv+IXSCkvy8AmnLwHOrqqfTrueuca6kcCGHUl66UZiMNZGurlIj6PtV/9DWheA\nvYA7VdXePZa2wW5d3crarwxhMaBWXpLTaD20z6K9Yd4ROIc21/hZNYAt1pPsRFtouSNjd+Sq6qF9\n1bSYbnRzvNbBdE1J8hfAZ6vqku54a9qi2w/3W9nChhzikzwYeAWwA+3f3VZoq1ySj9MGPkYtRPem\n3YHZCTi8qt7RU2mD5FQKbaDrI7grbSOPh1TVj7qH3tfN6e3boG65rAZzdsEauYTWCu0/pl3PAn4I\nPLW6nRe7XpyHA38LfIg2baFvH6bN0f8osNEc/SFJ8gzg5cBvabWGdsdgSH13D6uq40cHVXVxt8ht\nkMG4G5nbi/Z9PJV2ETc0rwMeThstdGRs07AFrSPST2BdX+PjaJv8fIH2914dg7HmOrKqPjfpgara\nY9rFTHCXJJd2nwe4bnfsqMfVdx3gdrSd76D1XP4O7Xu9z4CmquxSY9uRV9U30nYZPH9AF0i/rapJ\nFxpD9ALgjlX1s74LWcBmE84N8u9WN5Xitqyfq/+MJH9WVUObY/594OuG4k3KdqNQ3Plpd+4XSYY0\n13gQBvkGo+kbn4M0aT5SDWR74KravO8aVqE7A/etqisBkryJNod7L9oCx6E4p6vtvd3xI4FvJLk2\nY3cSevb6bkTzU8DvRieHNq+88z+s31lsqNYmeS3wxu74OWy4nfGQ7EsblRt1+Xg7barP0PwtcEKS\nz7Phz+hr+ytJK+zkJB9jw8GPk7tONBf3V9YwGYw18pAFHivarWqtTjcGbsD6jhnXB25SVVcm+d38\nXzZ1TwKeTdsuFOCLtFHPPwD79FTTXHeizc3fl/VTKYr1bQWH5FDgS92mJOMB6a/m/5Kpex7w98D7\nuuOTGG6Xj/OA7YHRltrbdeeG5h9pCwOvA1yr51o0Hc+hTZ/ZqzteC9y8Wxg+lPfOwTAYCwBXTm/S\nXg2c2bVCG/UHPqIbTfh0n4WNq6rfdLerPzahheDlfdQ0wYHAn0x7C9Or6c3AZ2l3BQY5H7r7wz20\nXtUbSPJR2sXPDYFzk3ylO74XPfdjncethrgTo1ZO16/8fGBP2nvUd4AP9lvVcNmVQgAkeVxVvTPJ\n8yc97m221S3JLYF7dodfraof9lnPJEkeCvwzcK2q2inJXWkrqgfT8SHJh4GDhtACaTHj28MOTZLX\nVdUhY6FzAwP7N//ThR6vqs9Pq5alSPJq4NND6OKildW1CH109/Ez2p2XF1TVDr0WNnCOGGtktOvV\n0LaE1QpLW7l2f9pI5+FJtk9yz6oa2mjXYbTwfjJAVZ3ZtUcbkq2Bbyb5KhtOTxhMkBtzYpKDaB00\nxmsdQru20Sr5f+m1iiWYG3zntr8boGcBL+imSf0BFy6vZt+krRd5cFWdB5Dk//Zb0vA5Yixt4roF\nbX8E9q2q2ye5MW1jknss8qVTleS0qtpzfKQzyVlVdee+axuZb/RwaKOGAEm+M+F0VdUg2rUl2Rw4\nrqoe23ctS9FdZBzOnPZ3Q/l+LlWS3ca7v2h2JXkYbcOZ+wKfoC1c/veqGtqAwqAM+apWPZjFDQp0\njd2rqnZP8jWAqvplkiEuyjknyWOAzZPsTNvS9ks917SBIQbg+Qz9j2O3+HOHJNeakTnbf8Pw298t\nxTtoG+loxnUb4Xy4Wy9yAG3h8s26wZDjnU4zmcFYc83MBgVaNn/oRudGbabWMMx/++fRtij/HfBu\n4JO0HbwGI8llrJ8Tey1gS+BXQ7xNneQJk85X1XHTrmUB5wNfTPIR1m8ZO9Q1D7PQ/m4pBtMUXMuj\nW8T6buDd3R3BA4EXMoxNkQbHYKy5ZmmDAi2PI4HjaSMJ/wg8gtYia2geVFV/RwvHACQ5kPW9OXtX\nVevm6Hdztw+grQQfovGpMtehzTM/g7Yj1lD8T/exGcNf/zAL7e+WwvmVq1hV/RI4pvvQBM4x1ga6\nW9U7MxsbFGiZJLkdLRgF+ExVndtzSRtJckZV7b7YuaEZcveHcUm2Bt5bVQ/su5a5klyvqgY9Gtu1\naTuVOe3vqurtvRV1NczC75S0khwx1lyztEGBlkGSd1TV42krmOee612S/YD9gW2TjN/N2Aq4op+q\nJpuza+RmwB60xViz4FfAoOYdJ7k3bWrXDYDtk9wFeEZVPbvfyibasqomtrucMbMwn1taMQZjzTVL\nGxRoeew2ftDNN757T7VM8kPaTk0PZcPtgC8DhtZ6aHwHySuAC2jTKQZnTo/gzYA7AO/vr6KJXgf8\nOfARgKr6ryT367ekeQ25/d06Se4LnFlVv0ryONpCu9dX1XcBqmqoU3+kqXAqhTYwSxsU6JpJcijw\nYuC6tEVDo0U3vweOqapD+6ptkiRbVtUf+q5jtZjTWu4K4LtVdWFf9UyS5MtVda85Lfr+q6ru0ndt\ncw29/d1IkrOAuwB3Bt4G/Dvwl1W14EYl0qbCEWPNNUsbFOgaqKpXAq9M8sqhheB57JjklbSRzeuM\nTg4peHQdPZ7Oxu0On9JXTfOZkdZy309yH6CSbAkcDAxu/jsMv/3dmCu6LYIPAN5QVW9J8tS+i5KG\nwmCsuQ7ruwBNV1Ud2rXw2ZkNA+cX+qtqomNpP5//CuwDPJk2BWBI/oO209SngSt7rmVB3XzofwJu\nRrtbMMQd0J4JvB7YFvgBbVHwc3qtaI4kf1tVr+4+P7CqPjD22BFV9eL+qpvosu5u0eOA+yXZjNZW\nUBJOpZA2eUmeRhuJuzVwJq292H9W1aAWXCY5varunuTsqrrT+Lm+axtJcmZV3bXvOpYiyXnAQ4bY\ngWSWjHdxmNvRYYgdHpLcAngM8NWqOiXJ9sDeA+tfLfXGEWNtIMmewFHA7WkbFGzOQDco0LI5mNbT\n9rSq2qdr3XZEzzVN8rtudOvbSZ5LG0G8Qc81zfWxJPtX1Ql9F7IEPxl6KJ6RnTgzz+eTjntXVT8G\nXjt2/D2G1bta6pXBWHO9gba3+gdoraaeAOzSa0Vaab+tqt8mIcm1q+qbSXbtu6gJDgauR9sK+hW0\n6RRP7LWijR0MvDjJ74A/MMzpCSNrk7yPttvl+HqCD/VX0kZmYSfOmufzSce9SXJqVe01Z3dGGPbP\nqDR1TqXQBpKsrao9kpxVVXfuzs3EBgW6epIcT5uvewitX/UvaT1Z9++1sFUoyW5VdU7fdQAkOXbC\n6RrSQsFRV4q+61hIkitpPaDD+g4vdMfXqSrn70ozxGCsDST5AvBntBY+PwZ+BDxpiO2RtPy6Fl43\nAj4xtF7WSU4CDqyqi7vjG9N2avvzfitbuiHOOZ1PkkO7ziV91uBOnJKmyqkUmuvxtJX+z6VtnrAd\n8H96rUgrqptXfk5VXVZVn0+yFXA34Ms9lzbXNqNQDFBVv0xysz4LuhoGN+d0AQcCvQZj3IlT0pQZ\njAVAku2r6nuj3Y9o29i+vM+aNDVvou1+NXL5hHND8MfRzylAkh0Y0BzOJZqleocQ4t2JU9JUDa0H\nqPrz4dEnST7YZyGautTYnKqq+iPDvGj+O+DUJO9I8k7gC8AsbEwyq4YQ4r9O23RIkqZiiH/81I/x\n0aHB7CSmqTg/yV/RRokBng2c32M9E1XVJ5LsTuuzDHBIVf1s9PiQFrYtYJZGPocwYuxOnJKmymCs\nkYVaDml1eyZwJPAS2r/9Z4CDeq1oHl0Q/tg8D7+Dnqd/JPlMVd1/vnNVtefkrxykDyz+lBXnTpyS\npsquFAIWbTlkj8tN2BC6EyxFn20Fk1yH1mP5c8DerB9t3YrW4eN2fdS1kCRrgKez8eYZg2nXJknT\n5oixAKiqzfuuQYM1hO4ES9HnVf4zaH2gbwWMtxK7lLZpzhD9B3AK8Gngyp5rmWjOZhTXArbEnTgl\nrSCDsaTFDGGu6aBV1euB1yd5XlUd1Xc9S3S9qnph30UspKpuOPo8SYADWD/HXJKWnV0pJC1mVuZb\nDWFh21uTvCTJMQBJdk7y4L6LmsfHkszM7obVfBiYmQ1dJM0e5xhLWtBQtgRfbGHbECR5H3A68ISq\numOS6wFfqqq79lzaRrppCtendXv4AwNcT5Dk4WOHmwF7AH9aVffuqSRJq5xTKSQtptfuBGML27bp\ntoEeX9i2bW+FTXabqnpkkkcDVNWvuykAgzM+TWHAHjL2+RXABbTpFJK0IgzG0iZuse4EVXVEP5Wt\nM0sL236f5Lp000+S3Iax/rtDkOR2VfXNrif0RqrqjEnn+1BVT+67BkmbFqdSSJu4JF+idSc4nbHu\nBFU1qB0QZ2FhW5IH0PpB3wH4FHBf4ElVdXKfdY1LckxVHZTkcxMerqrad+pFzcOWcpKmzWAsbeKS\nnDnEObBzJbk+8H+B7btgtzOwa1XNt+FHL5LclNY5IcBp47vz6aqZlYs2SauHwVjaxCX5B9oCsRP6\nrmUhs7CwLcnhVfXSsePNgHdU1WN7LGuibu72s4G9aFM/TgGOrqrf9lrYmFm5aJO0etiuTdLBtNZd\nv0lyaZLLklzad1ET3KaqXk3roEBV/Zrh9VjeLsmhAEmuDRwPfLvfkuZ1HLAbcBRtrvZutG21h2Sm\nWspJmn2OGEuaCd1t9fsDX6yq3buFbe+pqnv2XNo6XQeKdwFnA/sAJ1bVv/Zb1WRJvlFVd1jsXJ9m\noaWcpNXFrhTSJmqWuhN0DgM+QRuVfRfdwrZeK+rM+R6+Hngz8EXg80l2H+D3EuCMJHtW1WkASe4F\nrO25pg0s1lIuyW5Vdc606pG0+jliLG2iZqk7wchQF7bN8z0cGdT3MsnZtDnFWwK7At/rjncAvjmk\nEePFJDmjqiZe2EnS1WEwljQTZmlh25Al2WGhx6vqu9Oq5Zoayq6MklYPF99Jm7gk10ny/CQfSvLB\nJId0HQuGZvAL25IckWTrseMbd10/BqOqvjv+AfyGNmI8+pgls1avpIFzxFjaxCV5P3AZ8M7u1GOA\nravqwP6q2tgsLGybNII51Nv9SR4KvIa2o+BPaVMpzq2q3Xot7CoY6vdW0uxy8Z2kO86ZV/q5JN/o\nrZo5Zmxh2+ZJrl1VvwPotoe+ds81zecVtPnan66quyXZB3hczzVdVb/vuwBJq4vBWNLQuxO8Zs7x\nL2lbLr+Gdit9MAvbaCPan0lybHf8ZODtPdazkD9U1c+TbJZks6r6XJLX9V3UuCSfqar7z3euqvbs\npzJJq5XBWNpEzelO8KUkG3Qn6LO2cVW1T981LFVV/VOSs2j9lgFeUVWf7LOmBVyc5AbAF4B3Jfkp\n8KueawLW7cp3PWCbJDdm/UYuWwHb9laYpFXPOcbSJmrWuhMkOQJ4dVVd3B3fGPjrqnpJv5XNliS3\nBW4OnElbeLcZ8FjaBdHHq+r0HssDIMnBwCG0+c8/HHvoUuDfquoNvRQmadUzGEsCIMnNgHXdKKrq\nez2Ws5EhL2xLcmpV7dXt1Db+pjq4ndqSfAw4tKrOnnP+TsARVfWQfirbWJLnVdVRfdchadNhuzZp\nE5fkoUm+DXwH+DxwAXBir0VNtnnXpg0Y3MK260Pbqa2qthr7uOGQQnHn5nNDMUB3bsfpl7OgtyZ5\nSZJjAJLsnOTBfRclafUyGEsadSf476raiTY/9rR+S5potLDtqUmeCpzEcBa2zdKtt60XeOy6U6ti\nad5K6zxxn+74B8Cg+kJLWl1cfCdp8N0JYPAL226W5PnzPVhVr51mMYtYm+TpVfVv4yeTPA3ofX7x\nHLepqkcmeTRAVf2662ctSSvCYCxpsN0J5qqqExnoNA/gBqzvnjBkhwDHJ3ks64PwHsC1gL/orarJ\nft9NmSmAJLcBftdvSZJWMxffSZuoWehOALOxsG0oiwCvim5Djzt2h+dU1Wf7rGeSJA8AXkLrW/0p\n4L7Ak6rq5D7rkrR6GYylTdSsdCeY1I1iaGahxlmV5Ka0OfABTquqn/VckqRVzMV30qZrVroTzMLV\n+/0Xf4quqiSHV9XPq+rjVfUx4BdJ3tV3XZJWL+cYS5uuWelOMPiFbVX1i75rWKW2S3JoVb2ya9X3\nfuBrfRclafVyxFjadK1N8vS5JwfYnWC0sO2G83xo9XoKcKckhwIfBU6uqpf1W5Kk1cw5xtImKsnN\ngeNpfWI36k5QVT/uq7Zxs7iwTddMkvF/7y2BNwNfBN4CUFVn9FGXpNXPYCxt4obencCFbZueJJ9b\n4OGqqn2nVoykTYrBWNKgJbmJc3glSdPgHGNJg2Yo3nQlOSLJ1mPHN07iltCSVozBWJI0VPtV1cWj\ng6r6JbB/j/VIWuUMxpKkodq8a9MGQLc99LUXeL4kXSP2MZYkDdW7gM8kObY7fjLw9h7rkbTKufhO\nkjRYSfZj/c6CJ1XVJ/usR9LqZjCWJEmScCqFJGlgkpxaVXsluQwYH70JrY/xVj2VJmmVMxhLkobm\n+gBV5ZbfkqbKrhSSpKFxjp+kXjhiLEkampslef58D1bVa6dZjKRNh8FYkjQ0mwM3oM0plqSpsSuF\nJGlQkpxRVbv3XYekTY9zjCVJQ+NIsaReOGIsSRqUJDepql/0XYekTY/BWJIkScKpFJIkSRJgMJYk\nSZIAg7EkSZIEGIwlSZIkwGAsSZIkAfD/AyYw36NaL2MjAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x8b76f1c780>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "importance = pd.Series(np.abs(logit_model.coef_.ravel()))\n",
    "importance.index = training_vars\n",
    "importance.sort_values(inplace=True, ascending=False)\n",
    "importance.plot.bar(figsize=(12,6))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "toc_cell": false,
   "toc_position": {
    "height": "583px",
    "left": "0px",
    "right": "1324px",
    "top": "107px",
    "width": "326px"
   },
   "toc_section_display": "block",
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}