03.2_Outliers.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Outliers\n",
    "\n",
    "An outlier is a data point which is significantly different from the remaining data. “An outlier is an observation which deviates so much from the other observations as to arouse suspicions that it was generated by a different mechanism.” [D. Hawkins. Identification of Outliers, Chapman and Hall , 1980.]\n",
    "\n",
    "\n",
    "### Should outliers be removed?\n",
    "\n",
    "Depending on the context, outliers either deserve special attention or should be completely ignored. Take the example of revenue forecasting: if unusual spikes of revenue are observed, it's probably a good idea to pay extra attention to them and figure out what caused the spike. In the same way, an unusual transaction on a credit card is usually a sign of fraudulent activity, which is what the credit card issuer wants to prevent. So in instances like these, it is useful to look for and investigate further outlier values.\n",
    "\n",
    "If outliers are however, due to mechanical error, measurement error or anything else that can't be generalised, it’s a good idea to filter out these outliers before feeding the data to the modeling algorithm. \n",
    "\n",
    "\n",
    "### Which machine learning models are sensitive to outliers?\n",
    "\n",
    "Some machine learning models are more sensitive to outliers than others. For instance, AdaBoost may treat outliers as \"hard\" cases and put tremendous weights on outliers, therefore producing a model with bad generalisation.\n",
    "\n",
    "Decision trees tend to ignore the presence of outliers when creating the branches of their trees. Typically, trees make decisions by asking if variable x >= value t, and therefore the outlier will fall on each end of the branch, but it will be treated equally than the remaining values, regardless of its magnitude.\n",
    "\n",
    "Linear models, in particular Linear Regression, can be sensitive to outliers.\n",
    "\n",
    "A recent research article suggests that Neural Networks could also be sensitive to outliers, provided the number of outliers is high and the deviation is also high. I would argue that if the number of outliers is high (>15% as suggested in the article), then they are no longer outliers, and rather a fair representation of that variable. A link to this article can be found in the \"Additional reading resources\" lecture within this section of the course.\n",
    "\n",
    "**For an overview on whether different machine learning models are sensitive to outliers refer to the \"Bonus lecture: Machine Learning algorithms overview\".**\n",
    "\n",
    "\n",
    "### How can outliers be identified?\n",
    "\n",
    "Outlier analysis and anomaly detection are a huge field of research devoted to optimise methods and create new algorithms to reliably identify outliers. There are a huge number of ways optimised to detect outliers in different situations. These are mostly targeted to identify outliers when those are the observations that we indeed want to focus on, for example for fraudulent credit card activity.\n",
    "\n",
    "In this course, I would rather focus on identifying outliers introduced by mechanical error, so that we can process them before using them in machine learning algorithms.\n",
    "\n",
    "\n",
    "#### Extreme Value Analysis\n",
    "\n",
    "The most basic form of outlier detection is Extreme Value Analysis of 1-dimensional data. The key for this method is to determine the statistical tails of the underlying distribution of the variable, and then finding the values that sit at the very end of the tails.\n",
    "\n",
    "In the typical scenario, the distribution of the variable is Gaussian and thus outliers will lie outside the mean plus or minus 3 times the standard deviation of the variable.\n",
    "\n",
    "If the variable is not normally distributed, a general approach is to calculate the quantiles, and then the interquantile range (IQR), as follows:\n",
    "\n",
    "IQR = 75th quantile - 25th quantile\n",
    "\n",
    "An outlier will sit outside the following upper and lower boundaries:\n",
    "\n",
    "Upper boundary = 75th quantile + (IQR * 1.5)\n",
    "\n",
    "Lower boundary = 25th quantile - (IQR * 1.5)\n",
    "\n",
    "or for extreme cases:\n",
    "\n",
    "Upper boundary = 75th quantile + (IQR * 3)\n",
    "\n",
    "Lower boundary = 25th quantile - (IQR * 3)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "## Real Life example: \n",
    "\n",
    "### Predicting Survival on the Titanic: understanding society behaviour and beliefs\n",
    "\n",
    "Perhaps one of the most infamous shipwrecks in history, the Titanic sank after colliding with an iceberg, killing 1502 out of 2224 people on board. Interestingly, by analysing the probability of survival based on few attributes like gender, age, and social status, we can make very accurate predictions on which passengers would survive. Some groups of people were more likely to survive than others, such as women, children, and the upper-class. Therefore, we can learn about the society priorities and privileges at the time.\n",
    "\n",
    "====================================================================================================\n",
    "\n",
    "To download the Titanic data, go ahead to this [website](https://www.kaggle.com/c/titanic/data)\n",
    "\n",
    "Click on the link 'train.csv', and then click the 'download' blue button towards the right of the screen, to download the dataset. Save it in a folder of your choice.\n",
    "\n",
    "**Note that you need to be logged in to Kaggle in order to download the datasets**.\n",
    "\n",
    "If you save it in the same directory from which you are running this notebook, and you rename the file to 'titanic.csv' then you can load it the same way I will load it below.\n",
    "\n",
    "===================================================================================================="
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# to display the total number columns present in the dataset\n",
    "pd.set_option('display.max_columns', None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "3            4         1       1   \n",
       "4            5         0       3   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                           Allen, Mr. William Henry    male  35.0      0   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  \n",
       "0      0         A/5 21171   7.2500   NaN        S  \n",
       "1      0          PC 17599  71.2833   C85        C  \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      0            113803  53.1000  C123        S  \n",
       "4      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's load the titanic dataset\n",
    "\n",
    "data = pd.read_csv('titanic.csv')\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0xaeb8c66e10>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfIAAAFXCAYAAABZQMyNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl0VHWeP/z3rS1JLdkreyqEkBBCgCxsiiCCEXXUsTtq\nQId5umXmZ9v2Y3PatqE97TLiCP5mnp7T7aN2t/3Y0z+6bVFnxha1XdiMrJJAAglLIIQK2VPZqypJ\nbff5IySKklSWSm7dqvfrHM4h3NStz5dK6l33e7+LIIqiCCIiIpIlhdQFEBER0eQxyImIiGSMQU5E\nRCRjDHIiIiIZY5ATERHJGIOciIhIxlRSFzCW9vY+n5wnKkqLri67T84lB8HWXoBtDhZsc3AI9jYb\njYYJPTYorshVKqXUJcyoYGsvwDYHC7Y5OLDNExMUQU5ERBSoGOREREQyxiAnIiKSMQY5ERGRjDHI\niYiIZIxBTkREJGMMciIiIhljkBMREckYg5yIiEjGGOREREQyxiAnIiKSMQY5ERGRjPn17mdERDQz\nDlQ0+vR8q/OSfXo+Gh2vyImIiGSMQU5ERCRjDHIiIiIZY5ATERHJGIOciIhIxhjkREREMsYgJyIi\nkjEGORERkYwxyImIiGSMQU5ERCRjXpdo9Xg8eO6553D+/HloNBq88MILSEtLGzm+b98+vPLKK1Cp\nVCguLsYDDzwAt9uNX/ziF6irq4MgCPiXf/kXZGVlwWw2Y+vWrRAEAZmZmXj22WehUPCzBBER0WR5\nTdE9e/bA4XBg165deOKJJ7Bjx46RY06nE9u3b8cbb7yBnTt3YteuXbBYLNi/fz8A4K233sLmzZvx\nH//xHwCA7du3Y/PmzXjzzTchiiL27t07Tc0iIiIKDl6DvLy8HCtXrgQA5OXloaqqauRYbW0tTCYT\nIiIioNFoUFhYiOPHj+PWW2/Ftm3bAABNTU0IDw8HAFRXV2Pp0qUAgFWrVuHw4cM+bxAREVEw8dq1\nbrVaodfrR75WKpVwuVxQqVSwWq0wGAwjx3Q6HaxW69CJVSps2bIFn332GX79618DAERRhCAII9/b\n19c35nNHRWmhUikn3qrrMBoN3r8pgARbewG2OViwzdPDoA/16fmmWjNf5/HzGuR6vR42m23ka4/H\nA5VKdd1jNpvtmmB/6aWX8NOf/hQPPPAAPvzww2vuh9tstpEr9dF0ddnH35IxGI0GtLeP/aEhkARb\newG2OViwzdOnzzrg0/NNpeZgf50nGuheu9YLCgpQWloKAKioqEBWVtbIsYyMDJjNZnR3d8PhcKCs\nrAz5+fl477338Nvf/hYAEBYWBkEQoFAokJOTg2PHjgEASktLsXjx4gkVS0RERNfyekVeVFSEQ4cO\nYf369RBFES+++CJ2794Nu92OkpISbN26FZs2bYIoiiguLkZ8fDxuu+02/PznP8dDDz0El8uFp556\nCqGhodiyZQuefvpp/PKXv8Ts2bOxbt26mWgjERFRwBJEURSlLmI0vupaCbZummBrL8A2Bwu2efoc\nqGj06flW5yVP+rHB/jr7vGudiIiI/BeDnIiISMYY5ERERDLGICciIpIxBjkREZGMMciJiIhkjEFO\nREQkYwxyIiIiGWOQExERyRiDnIiISMYY5ERERDLGICciIpIxBjkREZGMMciJiIhkjEFOREQkYwxy\nIiIiGWOQExERyRiDnIiISMYY5ERERDLGICciIpIxBjkREZGMMciJiIhkjEFOREQkYwxyIiIiGWOQ\nExERyRiDnIiISMYY5ERERDLGICciIpIxBjkREZGMMciJiIhkjEFOREQkYwxyIiIiGWOQExERyRiD\nnIiISMYY5ERERDLGICciIpIxBjkREZGMqbx9g8fjwXPPPYfz589Do9HghRdeQFpa2sjxffv24ZVX\nXoFKpUJxcTEeeOABOJ1OPPXUU2hsbITD4cCjjz6KtWvX4syZM3jkkUcwa9YsAMCGDRtw5513Tlvj\niIiIAp3XIN+zZw8cDgd27dqFiooK7NixA6+99hoAwOl0Yvv27Xj33XcRFhaGDRs2YM2aNfj8888R\nGRmJf/u3f0N3dzfuvfderF27FtXV1fj+97+Phx9+eNobRkREFAy8Bnl5eTlWrlwJAMjLy0NVVdXI\nsdraWphMJkRERAAACgsLcfz4cdx+++1Yt24dAEAURSiVSgBAVVUV6urqsHfvXqSlpeGpp56CXq/3\neaOIiIiChdcgt1qt14StUqmEy+WCSqWC1WqFwWAYOabT6WC1WqHT6UYe+/jjj2Pz5s0AgIULF+L+\n++9Hbm4uXnvtNbzyyivYsmXLqM8dFaWFSqWcdOO+zmg0eP+mABJs7QXY5mDBNk8Pgz7Up+ebas18\nncfPa5Dr9XrYbLaRrz0eD1Qq1XWP2Wy2kWBvbm7GY489hgcffBB33303AKCoqAjh4eEjf9+2bduY\nz93VZZ9gc67PaDSgvb3PJ+eSg2BrL8A2Bwu2efr0WQd8er6p1Bzsr/NEA93rqPWCggKUlpYCACoq\nKpCVlTVyLCMjA2azGd3d3XA4HCgrK0N+fj4sFgsefvhhPPnkk7jvvvtGvn/Tpk04deoUAODIkSOY\nP3/+hIolIiKia3m9Ii8qKsKhQ4ewfv16iKKIF198Ebt374bdbkdJSQm2bt2KTZs2QRRFFBcXIz4+\nHi+88AJ6e3vx6quv4tVXXwUAvP7663juueewbds2qNVqxMbGer0iJyIiorEJoiiKUhcxGl91rQRb\nN02wtRdgm4MF2zx9DlQ0+vR8q/OSJ/3YYH+dJ9q17vWKnMifDb/5GPShU77HN5U3HiIiqXBlNyIi\nIhljkBMREckYg5yIiEjGGOREREQyxiAnIiKSMQY5ERGRjDHIiYiIZIxBTkREJGMMciIiIhljkBMR\nEckYg5yIiEjGGOREREQyxiAnIiKSMQY5ERGRjDHIiYiIZIz7kRNdNby3ua9wf3Mimgm8IiciIpIx\nBjnRVaIoos/uQLd1EKIoSl0OEdG4sGudgpooirjY2IMrrVZYegYw4HADAELUSiTEaJEcq8PspHAo\nFILElRIRXR+DnIJW/6ALh043o8liBwDoQlVISzBAqRDQ0mGHuaUP5pY+1Fzpxk0LExGu00hcMRHR\ntzHIKSg1tltx6HQLBhxuJMXqsHx+PPRh6pHjQ93sTlRetKCuuQ8fHL6MxdlxyEyJgCDw6pyI/AeD\nnIJOQ7sV+8oboRAELM42Yl5a1LfCWRAEhOs0WLkoCSlxvThW3Yqj1a3otTmwODtOosqJiL6NQU5B\npcfqwBeVzVAqBNy2NBXGyDCvj0lPDEdcVBj2HG/AmctdCNUokTs7ZgaqJSLyjqPWKWg4nG7sP9kI\np8uDG3ITxhXiw3ShaqxdnAJtqAonaiy40NAzjZUSEY0fg5yCgkcU8cWpZvTaHJifHoXZSeETPoc+\nTI1bF6dAo1bgaFULGtqs01ApEdHEMMgpKNTUd6Ox3YakWC3ys4yTPk+kPgRrC1OgUAg4eLoZ9gGX\nD6skIpo4BjkFvEGHGxUXLVCrFFixIBGKKY46N0aGoXCuEQ6nB4erWrh4DBFJikFOAa+y1gKH04OF\nGTEIC/HN+M65pkgkxWrRZLHh/JVun5yTiGgyGOQU0Lqtgzhf3w2DVo3stEifnVcQBNyYmwiNWoHy\nc+3osTp8dm4ioolgkFNAKz/XDlEECucaoVT49sddG6rCDfMT4PaIOHS6mV3sRCQJBjkFrMZ2Gxot\nNiREa5Eap5+W50hLMCAtwQBLzwAuNfVOy3MQEY2FQU4B6/SlDgDA4mzjtC6rOnS1L+BETTucLs+0\nPQ8R0fUwyCkgWbr70dbVj6RYHaLDQ6f1ufRhasxPj0b/oHvkwwMR0UxhkFNAqr7cBQDImRU1I8+X\nOzsa2lAVztR1oc/OgW9ENHMY5BRwrHYn6lv6EGUIQWKMdkaeU6VUoHCuER5RRNm59hl5TiIigEFO\nAeisuQsihq7GZ3LL0VkJBsRFheFKmxXtXf0z9rxEFNy8BrnH48EzzzyDkpISbNy4EWaz+Zrj+/bt\nQ3FxMUpKSvD2228DAJxOJ5588kk8+OCDuO+++7B3714AgNlsxoYNG/Dggw/i2WefhcfDgUHkWw6n\nGxcbehAWosSsxImvpz4VgiAgPzMWAFBx0TKjz01EwctrkO/ZswcOhwO7du3CE088gR07dowcczqd\n2L59O9544w3s3LkTu3btgsViwfvvv4/IyEi8+eab+P3vf49t27YBALZv347NmzfjzTffhCiKIwFP\n5CsXGnrgdHuQbYqCUjFzV+PD4qO1SIzRornDjhqu+EZEM8BrkJeXl2PlypUAgLy8PFRVVY0cq62t\nhclkQkREBDQaDQoLC3H8+HHcfvvt+PGPfwwAEEURSqUSAFBdXY2lS5cCAFatWoXDhw/7vEEUvERR\nRM2VbigVArJSfbeK20TlzRm6Kn/vi0uS1UBEwcPrwtNWqxV6/VeLaSiVSrhcLqhUKlitVhgMhpFj\nOp0OVqsVOp1u5LGPP/44Nm/eDGDojXb4nqVOp0NfX9+Yzx0VpYVKpZx4q67DaDR4/6YAEiztNei/\nmlpmG/Sgz+5ElikSsdE6SWsymbtwrr4bzT0DWDhn8ruteRMsr/PXsc3T4+u/S74w1Zr5Oo+f1yDX\n6/Ww2WwjX3s8HqhUquses9lsI8He3NyMxx57DA8++CDuvvtuAIDia0tk2mw2hIePfQ+zq8s+gaaM\nzmg0oL197A8NgSSY2ttnHQAw9CZ06uLQaHFTnH7k36WSmx6N+pY+/GF3NX7+UMG0DLoLptd5GNs8\nfXz9OzOVmoP9dZ5ooHvtWi8oKEBpaSkAoKKiAllZWSPHMjIyYDab0d3dDYfDgbKyMuTn58NiseDh\nhx/Gk08+ifvuu2/k+3NycnDs2DEAQGlpKRYvXjyhYolG43R5YG7pgzZUhYQZmnI2ltiIUORnxuJi\nQw/OmrukLoeIApjXIC8qKoJGo8H69euxfft2/PznP8fu3buxa9cuqNVqbN26FZs2bcL69etRXFyM\n+Ph4/OY3v0Fvby9effVVbNy4ERs3bsTAwAC2bNmCl19+GSUlJXA6nVi3bt1MtJGCQF1TD5wuDzKS\nwqe837iv3HXjLADA346ax/5GIqIpEEQ/3rLJV10rwdZNE0ztPVDRCADYf6IRV9qsuHdlOsJ1Gomr\nGrI6Lxn/9peTOGvuwrPfW4K0BN/e8wum13kY2zx9hn+XfGV1XvKkHxvsr/NEu9a93iMn8ne2fieu\ntFlhjAz1mxAHht4Yk406nDV34Y8fn8OqvKRJn2sqb4pEFNi4shvJ3vD2oRnJERJX8m2JMVpEh4fA\n3NLHNdiJaFowyEnWRFFEbVMvlAoBs3zcde0LgiBgfno0RADVdZ1Sl0NEAYhBTrLWbXWg1+bArMRw\naNS+WXPA19LiDdCHqXGxsRf9gy6pyyGiAMMgJ1kztwwNDslI8b9u9WEKhYD56VHweEScr+eyrUTk\nWwxykjVzax+UCgFpCTO7QcpEzU6KgEatQM2Vbri5WRAR+RCDnGSr0WJDj9WBpFid33arD1OrFMhM\nicCAw43LzcE1rYaIpheDnGSr/HwbAPh8fvZ0mZsaBQHAOXMX/Hj5BiKSGQY5yVbZuXYoBAEpcdJt\nkDIReq0aqfF6dPQOor1b2rXgiShwMMhJllo67WhotyIpVguNj3bImwnZaVEAhq7KiYh8gUFOsiS3\nbvVh8VFhiDKEwNzaB9uAU+pyiCgAMMhJlsrOtUOpEJASp5e6lAkRBAHZaZEQRaCGU9GIyAcY5CQ7\nlu5+mFv7MG9WFEL8fLT69aQnhl+ditYDt5tT0YhoahjkJDsVFy0AgIJMo8SVTI5KqUBWSiQGnW7U\ncSoaEU0Rg5xkp/JqkC/MiJG4ksmba4qEIABnORWNiKaIQU6y0j/owrn6bpji9YgOD5W6nEnThalh\nijegq28QbV39UpdDRDLGICdZqa7rhNsjIm9OrNSlTFl2WiQATkUjoqlhkJOsDHerLwqAII+LDEN0\neAjq26yw9nMqGhFNDoOcZMPjEVFZ24EInUZ288evRxAEZJuiIIrgrmhENGkMcpKNS829sPY7sWhO\nDBSCIHU5PpGeaECIWokLDd1wcSoaEU0Cg5xkI5C61YcplUO7ojmcnpG91YmIJoJBTrJRcdECtUqB\nnFnRUpfiU1mmyKFd0di9TkSTwCAnWbB096Ox3YZ5afJczW0s+jA1kuP06OgZgKWHU9GIaGIY5CQL\npy51AAAWyXgRmLFkm4amop0386qciCaGQU6yUHWpEwCQOzswgzwxRguDVo26lj4MOFxSl0NEMsIg\nJ7/ndHlw1tyFhGgtjJFhUpczLQRBwFxTJDweERcbeqQuh4hkRCV1ARR8DlQ0Tuj7mztsGHS6EWUI\nmfBj5SQjOQInayyoudKDnPTogJliR0TTi1fk5PeaLDYAQFKsTuJKpleIWon0pHBY+51oardJXQ4R\nyQSDnPxeY7sNSoWA+OjA7Fb/upFBb5yKRkTjxCAnv2YfcKLb6kB8tBYqZeD/uEaHh8IYGYpGiw19\ndofU5RCRDAT+OyPJWqPFDgBIDvBu9a+ba4oCwKtyIhofBjn5tWC5P/51aQl6hGqUuNjYw/XXicgr\nBjn5LY9HRLPFBn2YGuE6tdTlzBil4qv11+uauf46EY2NQU5+y9IzAIfLg6RYLYQgm4qVmTq0/vr5\n+i6Ioih1OUTkxxjk5LeCsVt9mD5MjZQ4PTp7B2HpGZC6HCLyYwxy8luNFhsEAUiMCb4gB4C5nIpG\nROPAICe/NOBwoaNnAHFRYVCrgvPHNDFGi3CdBpeb+9DLqWhENAqv75AejwfPPPMMSkpKsHHjRpjN\n5muO79u3D8XFxSgpKcHbb799zbHKykps3Lhx5OszZ85g5cqV2LhxIzZu3IiPPvrIR82gQNMUhNPO\nvkkQBMxNjYRHFPFFZZPU5RCRn/K61vqePXvgcDiwa9cuVFRUYMeOHXjttdcAAE6nE9u3b8e7776L\nsLAwbNiwAWvWrEFsbCxef/11vP/++wgL+2o1rurqanz/+9/Hww8/PH0tooAQzPfHvy4jORwnL7Tj\nwMkm3LEsDQpFcA36IyLvvF6Rl5eXY+XKlQCAvLw8VFVVjRyrra2FyWRCREQENBoNCgsLcfz4cQCA\nyWTCyy+/fM25qqqqcODAATz00EN46qmnYLVafdkWChCiKKLJYkNYiBJRhhCpy5GURq3E7KRwdPQO\noLLWInU5ROSHvF6RW61W6PX6ka+VSiVcLhdUKhWsVisMBsPIMZ1ONxLO69atQ0NDwzXnWrhwIe6/\n/37k5ubitddewyuvvIItW7aM+txRUVqoVMoJN+p6jEaD928KIP7cXoM+dMzj7V12DDjcyE6LQrhh\n/OurezuvXOXPjUfNlR4cPN2C226cfc0xf36dpwvbPD18/fsz1Zr5Oo+f1yDX6/Ww2b7aicnj8UCl\nUl33mM1muybYv6moqAjh4eEjf9+2bduYz93VZfdW3rgYjQa0twfPwhr+3t4+69jTqWrquwAAxshQ\nr987zKAf//fKTYhKQGZKBE7WtOP0+VYkRGsB+P/rPB3Y5unj69+fqdQc7K/zRAPda9d6QUEBSktL\nAQAVFRXIysoaOZaRkQGz2Yzu7m44HA6UlZUhPz9/1HNt2rQJp06dAgAcOXIE8+fPn1CxFByaLDYI\nCN5pZ9eztjAFALC3vMHLdxJRsPF6RV5UVIRDhw5h/fr1EEURL774Inbv3g273Y6SkhJs3boVmzZt\ngiiKKC4uRnx8/Kjneu6557Bt2zao1WrExsZ6vSKn4ONwutHe3Y+YiFCEanxzWyUQFGQZEWUIwcFT\nzfjOynRoQ4NnyVoiGpvXIFcoFHj++eev+beMjIyRv69ZswZr1qy57mNTUlKumZI2f/58vPXWW5Ot\nlYJAc4cdoggkG3k1/nUqpQJrC1Pw7oFalFY24/ZlJqlLIiI/EZwrbZDfauS0s1HdnJcEjVqBveVX\n4PZwVzQiGsIgJ78xPO1Mo1YgJiIwR6BPhS5UjRULEtHRO4gTNZyKRkRDGOTkN3qsDtgHXEiK0UER\nZLudjVfR4lQAwKfH6yWuhIj8BYOc/MZwtzrvj48uIVqLhRkxqG3sxXlzp9TlEJEfYJCT3xhelpXT\nzsZ225Khq/K/ll6SuBIi8gcMcvILTpcHrZ39iDKEQBvqdTJFUJuXFoUUox6HTjWhg3uVEwU9Bjn5\nhdZOOzyiGNS7nY2XIAgoWpICj0fEvhNcIIYo2DHIyS+MTDvj/fFxWZ4Tj0h9CD6vaMKAwyV1OUQk\nIQY5+YUmiw1qpQJxkePfJCWYqVVK3HHjLNgHXTh0ukXqcohIQgxyklyvzYE+uxOJsVrutz0Bd9w4\nCyqlgD1lV+ARRanLISKJMMhJck1czW1SogyhWJ6TgNaufpyq7ZC6HCKSCIOcJMcgn7xbFw/tivbp\nl1wghihYMchJUm63By2ddkToNNCHcUeviTLFGzA/PRrn6rtR29QjdTlEJAEGOUmqpbMfLrfI1dym\n4M7laQCAvx3lVTlRMGKQk6Qa2q0AgBSjXuJK5CvbFIn0xHCcqGkfuU1BRMGDQU6SEUURje02qFUK\nxEVx2tlkCYIwclX+8TFelRMFGwY5SabX5oC134mkGE47m6r8rFgkRGtxpLoFnb1ctpUomDDISTIN\n7cO7nbFbfaoUgoA7lpvg9oj49PgVqcshohnEICfJNLZz21JfumF+AqIMIThQ0Yheu0PqcohohjDI\nSRIOpxutXXbEhIciLIS7nfmCSqnA7ctMcDg9+IxX5URBg++gJInmDjtEkVfjvnbzoiR8eMSMPeUN\nWLfUxLn5NGPcHhGnazvQbR2E2y3iyzOtMMUbUHzzbKhVSqnLC2i8IidJjEw7i2OQ+5JGrcQdy0wY\ndLh5VU4zxunyYP+JBpyq7UB9qxWNFhvO1Xfj0+NX8L//chK9Nt7qmU4Mcppxw9POQjVKxISHSl1O\nwFmdlwyDVo095VdgH3BKXQ4FuIGrHxqbLHYkG3W4b3UGNtyaideeuBnLc+JR29iLbX8sG/nwTr7H\nIKcZZ+kZwIDDjWSjDoLAaWe+FqJR4valJvQPurGnrEHqciiADTrc+ORYPSw9A5idFI5b8pOhDVVB\nrVIgRK3EP9+dg3tXpqOjdwDb/1SOjh5OjZwODHKacVdahz6Zp8Zx2tl0uaUgGfowNT49fgX2AZfU\n5VCAqrhoQY/NgWxTJFYsSPjWehCCIOCeFen4h9uy0D/oxlt7L0hUaWBjkNOMu9JmhVIhcLezaRSq\nUWHd0lTYB1349DhXeyPf6+obRE19N8J1GhRmx43Zu3ZLfjIyUyJQXtOO05e45a6vMchpRjV32NBj\ncyApVgeVkj9+02ltYQrCtUNX5dZ+3isn3xFFEV+ebYUIYEl2HJReVmYUBAH/cNtcKAQBf/6sBk6X\ne2YKDRJ8J6UZdfKCBQBgime3+nQL1ahw5w2zMOBw46OjZqnLoQBS32pFa2c/Uoy6cU8hTY3TY21h\nCtq6+vE37gngUwxymlEna9ohCFyWdabckp+EKEMI9pU3oNs6KHU5FABcbg/KzrVBIQCLs+Mm9Ni/\nvykdEToNPjxihqWnf5oqDD4Mcpox3dZB1Db1Ij5Ki1ANF4iYCWqVEnevmAWHy4MPD/OqnKaupr4b\ntgEX5s2KRrhOM6HHakNVKL45A06XhzMqfIhBTjOm4mq3Okerz6ybFiQiLjIMByoaeRVEUyKKIs5f\n6YZSISA3PXpS51g+Px7hOg0OnmrGoIP3yn2BS7TSjDlxoR0AkMr74xN2oKLxW/9m0Ieizzq+eblZ\npggcPNWP3/y1Gr/4x8W+Lo+CRJPFhj67E3OSIxAyyV41lVKB1XlJeP/QZRypbsHq/GQfVxl8eEVO\nM6J/0IWzl7tgitdz/W8JpCeGI8oQgktNvahv7ZO6HJKpc/XdAIC5psgpnefmvGQoFQL2ljdAFEVf\nlBbUGOQ0IyovWuD2iCjINEpdSlASBAGFc4f+7985UCtxNSRHfXYHGtttiI0IRUzE1JZWjjKEoHCu\ncWRNdpoaBjnNiC/PtgEAlsyb2ChX8p2kWB0SY7SorutEVR0X5aCJOX81cLPTpnY1PuzWwlQAwN5y\nDnqbKgY5TTvbgBOnL3UgNU6PxBiu5ialwrlGCADe2V8LD7s0aZxcbg8uNvYgVKNEWoLBJ+fMSA5H\nWrwBJy+0cxDmFDHIadqdqGmH2yNiKa/GJRcdHorl8xNwpc2KI1UtUpdDMnG5uQ8OpwdzUiKgVPgm\nNgRBwNrCFIgiUFrZ7JNzBiuvr4jH48EzzzyDkpISbNy4EWbztXNR9+3bh+LiYpSUlODtt9++5lhl\nZSU2btw48rXZbMaGDRvw4IMP4tlnn4XH4/FRM8ifDXerL50XL3ElBADfXTUbapUC//V5LQYc3FCF\nvKtt7AEAZKX6plt92JJ5cQhRK3HsTAsHvU2B1yDfs2cPHA4Hdu3ahSeeeAI7duwYOeZ0OrF9+3a8\n8cYb2LlzJ3bt2gWLZWiu8Ouvv45f/OIXGBz8ajWp7du3Y/PmzXjzzTchiiL27t07DU0if9Jrc+Ds\n5S7MTgqHMTJM6nIIQExEKNYtNaHb6sBHR7lUJo3NNuBEa1c/4qPCfD7jJEStRH5mLNq7B1DXzNkU\nk+U1yMvLy7Fy5UoAQF5eHqqqqkaO1dbWwmQyISIiAhqNBoWFhTh+/DgAwGQy4eWXX77mXNXV1Vi6\ndCkAYNWqVTh8+LDPGkL+qfx8GzyiyKtxP3PnchMi9Rp88mU970/SmMxXA3ZWYvi0nH9pztB7w7Ez\nrdNy/mDgdUEYq9UKvf6rBTyUSiVcLhdUKhWsVisMhq8GPuh0OlitQ3tNr1u3Dg0N145GFEVxZKs7\nnU6Hvr6xP4FFRWmhUvlmKU+j0TcDNOTCX9p74mIHBAG4fUU6YiKGrsgN+qlNXRnNdJ3Xn02mzcM/\nGw/fk4tfvnkC7x82Y8s/LvF1adPGX362Z9JMtHm0n6X6NisEAZifEYuwkPGvITbemldH6fDGh2dR\nXtOGx0qRUIvTAAAgAElEQVTyR3ZS4+s8fl5fFb1eD5vNNvK1x+OBSqW67jGbzXZNsH+T4muDJGw2\nG8LDx/6E19Vl91beuBiNBrS3B0+3jb+0t7N3AGcudSAzNRIeh2ukpvGuRjYRE1nlLFBMts3Dr0NO\nagRmJ4XjYGUTVpTXY64pytcl+py//GzPpJlq8/V+lnptDrR19SMpVguX04U+5/jHVEyk5oIsI0or\nm3DoxBXMS4sK+td5ooHutWu9oKAApaWlAICKigpkZWWNHMvIyIDZbEZ3dzccDgfKysqQn58/6rly\ncnJw7NgxAEBpaSkWL+ZSkYHsy7NtEAEs42h1v6QQBGy4NRMA8OaeC3Bz8Cl9w+WWoWBJn6Zu9WHD\n7xHsXp8cr0FeVFQEjUaD9evXY/v27fj5z3+O3bt3Y9euXVCr1di6dSs2bdqE9evXo7i4GPHxo98L\n3bJlC15++WWUlJTA6XRi3bp1Pm0M+Q9RFHHwdDNUSmHCWx3SzMlIisCK3KHpaPtPfHs9dwpul5t7\noVAI077R0VxTFCL0GpSfb4PLzQ+UE+W1a12hUOD555+/5t8yMjJG/r5mzRqsWbPmuo9NSUm5Zkpa\neno6/vSnP022VpKRS829aLLYsCQ7DgbtxLY6pJl1/y1zcPKCBf/zxSUsmRePiAluTUmBqatvEN1W\nB0zxemjU07vtsEIhYEl2HPaUNaCqrhOJCRHT+nyBhgvC0LT44uoCDysXJkpcCXkTrtPguzfPRv+g\nG2/vuyh1OeQnLjf3AgBm+WglN2+WXR29/uVZdq9PFIOcfG7Q4caXZ1sRHR6CnFmT27OYZtbqvGSk\nxRtwpLoF5+u7pC6HJCaKIi639EGlFJAyzd3qw2YnhiMmPASVFzvgdLF7fSIY5ORzx8+1YcDhxk0L\nEqG4OpWE/JtCIeAf1mVBAPCnT2t4nzLI9dgc6LM7kRSrg0o5MzEhCALyM43oH3ShqtYyI88ZKBjk\n5HNfnGqCAOCmBexWl5OMpAjcnJeERosNHx01e38ABayGtqH1QKZ7kNs35WfGAgCOVXMfgIkY/+x+\nonFo6bTjQkMPcmZFIZZLsvqlAxWjj05PiNEiLESF9w9ehkcUEakPGfNcq/OSfV0e+YErbVYIAJKN\nMxvkmamR0IWqcLSqGd+9adbIAmI0Nl6Rk099UdkEALiJg9xkSaNWYllOHDyiiCNV3MgiGPUPutDe\nPYC4qDCEaqZ3tPo3qZQKLMyIRUfPwMgcdvKOQU4+M+h0o7SyCfowNQqzjFKXQ5NkijcgLcGA9u4B\nnK/vlrocmmEN7dJ0qw8ryBrqXj95gffJx4tBTj5zuKoFtgEXVucnQ+2jNfJJGkvnxUGjVuBETTus\n/U6py6EZdKVtaNntmRqt/k256THQqBQ4eaFdkueXIwY5+YRHFPHZ8StQKgSsKeB9U7kLC1Fh8dw4\nuNwijlW3sos9SLjcHjRbbIjQaxAu0cJAIRolFmUZ0dhuQ5uP9tsIdAxy8omqSx1o6bRjWU681wFS\nJA8ZyeFIjNGi0WLjXtFBoslig9sjInWGB7l90/LcoTE27F4fH45aD2BjjU6eKG+jkz89fgUAcNuS\nVJ89J0lLEAQsnx+P3Ycu4/jZNiTFahGq4VtGIGu42q2eGi9tkC/NSYAA4ERNO9YtNUlaixzwipym\nrKHNijOXu5BtioQpPvj2EA5kBq0GeZmxGHS6cfxsm9Tl0DTyiCIa2q0I1SgRGzHxfe59KdIQgoyU\nCFxs7OEYjXFgkNOUfVo2dDVexKvxgJSdFoXYiFDUNffhytWFQijwWLoHMOBwIyVO7xfzt/PmxEIU\ngVNc5c0rBjlNiaW7H0eqWhAfFYZFc2KlLoemgUIQcGNuAhQKAUeqWtA/6JK6JJoGwx/STBKNVv+m\n4feTiosdElfi/xjkNCXvH74Mt0fEPTelQ+EHn+JpekQaQlCQFYsBh5sLxQSohjYrVEoBCTFaqUsB\nACTFaBEbEYqqSx1c+98LBjlNWmunHYdPtyApVodl8+KlLoem2by0KCTEaNHQbsOFhh6pyyEfaum0\no8fmQGLMzG2S4o0gCMibM/Th8fwVLkw0Fv94xUiW/nqoDh5RxN/flM5dzoKAIAhYsSABGpUCZefa\n0GtzSF0S+UjF1WleUq3mNppFVzdRqeQ0tDExyGlSGi02HKtuRWqcHoVzuRxrsNCFqrFsfjxcbhEH\nTzXD7WGXZyAYXkUtJU4ncSXXmpsaibAQJSouWng7ZwwMcpqUvx6sgwjg3pW8Nx5s0hPDkZ5ogKVn\nAB8c5nanctdrd+BiYw+MkWF+t06ASqnA/PQYWHoG0NTBVd5GwyCnCbvY2IOyc21ITzQgjyPVg9Ky\nnHhoQ1XYfegyapt4v1zOTl3sgChKvwjMaPLmxAAAKi+ye300DHKaEI9HxJ8/rQEAlKzJ9Iv5pjTz\nNGolVixIgEcU8fvdZzDocEtdEk3ScLe61MuyjmbB7BgIAlDBIB8Vg5wm5PPKJphb+3DD/ARkpUZK\nXQ5JKDFGh9uWpKK1qx9/2XtB6nJoEhxON6ovdyIhWosIvTSbpHhj0GqQkRyB2sYe9Nk5wPJ6GOQ0\nbtZ+J/7781qEapR44JYMqcshP1B882ykxulRWtmEo9UtUpdDE3TG3AWH04O8TP++RfbVKm9cHOZ6\nGOQ0bv/1eS1sAy7ce1M6IrjDGQFQq5R49N5chGiU+OPH59HcYZO6JJqA4Wln+X4e5MOrvPE++fUx\nyGlcLjb2oLSiCcmxOqwpTJG6HPIjCdFafP+ObAw63Xj1vSoMOnm/XA48oojKixYYtGpkJEVIXc6Y\nkmK0MEaGoqquk6u8XQeDnLxyujz4/QdnAAAb1831m5WfyH8snRePWwqS0dhuGxkMSf6trrkXPTYH\nFmXE+v2CToIgYNHwKm/1XOXtm/iOTF6dqGlHW1c/1i01cYAbjWr9mkykJRhw8HQzDp1ulroc8kIu\n3erD8ti9PioGOY2pyWLD+fpuJMfq8J1V6VKXQ35MrVLg0XtzERaiws5PzqOxnVue+rOKCxaoVQrk\nzIqWupRxyeIqb6NikNOoBp1uHD7dAkEA/umuHKhVSqlLIj8XFxmGh++cB4fLg1ffq8KAg1ue+qO2\nLjsaLTbMnxWNEI08fq9VSgVyh1d5s3BQ5dcxyOm6RFHE4dMtsA+6sGhOLNISDFKXRDJRONeIosWp\naO6wY+cn53n15IeGu9X9fdrZNy26usobF4e5FoOcrqv6cheutFmREK1Fbro8ut7If9x/SwZmJ4Xj\nSHUr9pQ1SF0OfcPJCxYI+Gpal1wszIiFIACVFzmf/OsY5PQtLR12nDzfjrAQFVYuSvT7Ea3kf1RK\nBX54by4idBq8te8Cqur4xusvrP1OXGjoweykcETo/HM1t9How9SYc3WVt16u8jaCQU7XsA+4UFrZ\nBAjAzXmJCAvxr92QSD6iw0Pxo+8ugFIh4DfvVaOlk7tX+YPKixZ4RFF23erD8jJjIYKj17+OQU4j\n3G4PPq9oxIDDjcK5RsRFaaUuiWQuIzkC/9ft2bAPuvDrd0/BPuCUuqSgV35+aJOUgiyjxJVMTkHm\nUN0naxjkwxjkBODq4LaqFrR3DyA90YB5aVFSl0QBYsWCRNy+1ISWTjt+8341PB4OfpNK/6ALVXWd\nSDbqkBijk7qcSYmP1iLFqENVXSf6BzkrAmCQ01WnL3WirrkPsRGhuDE3gduTkk/dtzoDC2bHoOpS\nJ945cFHqcoJWZa0FLrcHi+fGSV3KlBRkGeFye3D6EsdeAABvgBLMLX2ouGCBNlSFWwqSoeQSrDRO\nByoax/2989OjUN/ah0++vAJrvxMZydeu7706L9nX5dE3lJ8b6lZfPFee3erDCufG4f1Dl3Giph1L\n58VLXY7kvL5jezwePPPMMygpKcHGjRthNpuvOb5v3z4UFxejpKQEb7/99piPOXPmDFauXImNGzdi\n48aN+Oijj6ahSTQR7d39OHiqGSqlgLWFyRzcRtNGo1biloJkaFQKHKlqRVtXv9QlBZUBhwunLnUg\nMUaLpFh5dqsPSzHqEBcZhsraDjhd3KTHa5Dv2bMHDocDu3btwhNPPIEdO3aMHHM6ndi+fTveeOMN\n7Ny5E7t27YLFYhn1MdXV1fj+97+PnTt3YufOnbjzzjunr2XkVa/NgX3ljfB4RKxalIQoQ6jUJVGA\nC9dpsCovCSJE7D/RiF4bpxDNlNOXOuF0DXWry/3WmSAIKJhrxKDDjerLXVKXIzmvQV5eXo6VK1cC\nAPLy8lBVVTVyrLa2FiaTCREREdBoNCgsLMTx48dHfUxVVRUOHDiAhx56CE899RSsVq7FLJUBhwt7\nyxsw6HRj2fx4pMTppS6JgkRSrA7L58dj0OnGnrIGDliaIWXn2gAAi7PlfX98WOHVUfcnro7CD2Ze\ng9xqtUKv/+pNXqlUwuVyjRwzGL5aulOn08FqtY76mIULF+JnP/sZ/vznPyM1NRWvvPKKL9tC4+Ry\ne7D/RCP67E7kzo7mjmY04zJTIrEwIwbWfif2lTfC6eIe09Np0OnGqdoOxEeFIcUo7271YelJ4YjU\na3DyQjvcnuD++fF6Q1Sv18Nm+2qBeo/HA5VKdd1jNpsNBoNh1McUFRUhPDwcAFBUVIRt27aN+dxR\nUVqofLRRh9EYXGuFG40GGPTf7ir3iCI+OWpGe/cAMlMjsSo/ZVzdbL78/7teXf58Xn8m5zbflJcM\nh9uDc5e7cOh0C+4vyoZa5X2gZbD9LgNTb/OR000YdLqxqiAFcXHh1/0eX/8sTbXm8Tx+xaJkfHio\nDq29DizKlPcAPmDy/2deg7ygoAD79+/HnXfeiYqKCmRlZY0cy8jIgNlsRnd3N7RaLcrKyrBp0yYI\ngnDdx2zatAlPP/00Fi5ciCNHjmD+/PljPndXl29WgjIaDWhv7/PJueRguL191oFvHTt+tg2XGnsQ\nHx2GpfOMsNoGx3VOX/7/Xa+uqTLoQ6flvP4sENq8OMuIPqsD9a192PGfx/C/7p4/5pLAwfa7DPim\nzZ8dHRpwPC81YtRz+fpnaSo1j7fNOaZIfHgI+OzoZSRFyvdDLXBtmyca6F6DvKioCIcOHcL69esh\niiJefPFF7N69G3a7HSUlJdi6dSs2bdoEURRRXFyM+Pj46z4GAJ577jls27YNarUasbGxXq/IybfO\nXO7EWXMXIvQarM5PhlLBaWYkLYVCwM35SfjseAO+PNsGbagaG2/Lkv1gLH9iH3Di5AULEmO0SIsP\nrN6MuamRiNBrUHauDQ8VZUEVpFNnvQa5QqHA888/f82/ZWRkjPx9zZo1WLNmjdfHAMD8+fPx1ltv\nTbZWmoL61j6UnWtHWIgSawtTEKKWxx7EFPhUSgXWFibj0OkWHDjZCF2oCsU3Z3h/II1L2fl2uNye\ngFzoSaEQsGxePD49fgWnL3UgPwC61ycjOD++BBlLzwC+qByaK76mIAX6MLXUJRFdQ6NW4icleYiL\nCsOHR8x4/1Cd1CUFjMOnmwEAy3MSJK5keiyfP7QgzNHqVokrkQ6DPMBZ+53Yf6IBbo+IlYuSEBMh\n7/tIFLgidBo8uT4fsRGheO+LOnx45LLUJclee3c/ahp6kG2KDNjf/bR4AxKitai4aAnaqYxcxiuA\nOVxu7CtvQP+gG0uy45A6hbniE1mKk2iyYiJC8bMN+djx5gn81+eXoFQocPsyk9RlydaR6hYAwI25\niRJXMn0EQcDynHi8d7AOJ2rasWJB4LZ1NLwiD1BujwelFU3otjow1xSJebO4mxnJQ2xkGH62IR+R\neg3e3n8Rfztm9v4g+pbhHQ01KgUKZb62ujfLhrvXzwRn9zqDPACJoog/f1qDJosdyUYdlgTISk4U\nPOKitNjyYAGiDCF4Z38t75lPQm1TL9q6+lEw1xjweyjER2kxOykcZy53oicIl/1lkAeg9z6vxYGK\nJkQZQrBqUdKY83KJ/FV8tBZbHyoYuWf+36W1EEXuZT5eh6uudqvPD8xBbt+0LCceogh8eTb4rsoZ\n5AHmRE07/vBBNSL1GqwpTB7XSllE/soYGYYtDxYgLioMHxw243fvnYaHYe5V/6ALR6tbEKnXBM1t\ntaXz4qEQBBw81Rx0H/j4Lh9AzC19+N3uaoSolfjxfYugC+U0M5K/mIhQbHmwAMlGHT44WIff7z4D\nlzu419b25nBVCwYcbtwSRAs/Reg0yM+MxZU2Ky4190pdzowKjlc4CHT1DeJX71bC6fTgpw8VIi0h\nsFZwouAWZQjBlgcLkJ0WhaNnWvHyf53GoIP7UF+PRxSxt7wBKqWAm/OSpS5nRq3OH2rvgRPBNcsm\nsEdABIlBhxu/fvcUuq0OPHDLHCzLTQy69ahJ/sYzxXFVfjJ6bQ6cvtSBX/z+GG4pSIY29PpvY6uD\nLMSGnbnciZZOO26Yn4BwnUbqcmbUvFlRiIsKw5fn2lCyNjNoFr/iFbnMeUQRv//gDMytfVi5MBHr\nlqZKXRLRtFGrlLilIBkZyeHo6B3AR0fN6OyV98Yxvra3rAEAcOviFIkrmXkKQcDqvGQ4XZ6RwX7B\ngEEuc/9TegnlNe3INkVi47q5AbeWMtE3KRUCbsxNQH5WLOwDLnx8rB4NbVapy/ILbd39OFXbgYyk\ncKQnXn+70kC3YkECVEoBB042Bs2gNwa5jB063YwPj5gRHxWGH35nQdDu/EPBRxAELJgdg5vzkiCK\nwP4TjTh7uSto3rhHs6+8ASKAtYXBdzU+zKDVYHF2HFo67ThX3y11OTOC7/wyVXOlG//5t3PQharw\n4/sXBc29IKKvS0swYN3SVIRolDh+rg1fnm2DxxOcYW4fcOKLU82I0A0FWTC75eqgt/0ng2PQG4Nc\nhtq67Ph///s0AOCH9+YiIVorcUVE0omNDMOdN6QhUq/B+fpu7C1vwEAQjmj/+Msr6B904bYlqUHf\nOzcnOQIpRj1OnG9HW5dd6nKmXXC/2jJkH3DiV++egrXfiY3r5mLerGipSyKSnD5MjTuWpyHFqENz\nhx0fHr6Myy3BM5e41+7AZ2VXEK7TYE0Qd6sPEwQBf3dDGjyiiI+OBv5a/QxyGXG5PXjtvSo0d9ix\nbmkqVi1KkrokIr+hVilwS0EyFs2JgW3AhRd3nkBpZVNQ3Df/+Gg9Bh1u3HVDGkLUSqnL8QtLsuMQ\nH63FodMt6OgJ7JkNDHKZEEURf9lzAdWXu5A3Jxb3r54jdUlEfkcQBCyaE4u1hckIUSvwn387h9c/\nOBPQ+1R3Wwex90QDosNDgm4BmLEoFALuuiENbo+Ij4/VS13OtGKQy8THX9Zj/8lGpMbp8b/uyeFG\nKERjSDbq8ez3lmB2UjiOVrfiX/7zOMwtgblI0geHL8Pp8uCeFencW+EbluXEIzYiFJ9XNqHHOih1\nOdOGr7oMHKlqwTv7axFlCMGP71uIUA0X5CPyJjYyDFsfKsAdy01o6+rHC/+nDB8dNQfUqPbWTjs+\nr2hCXGQYbswNjl3OJkKlVODO5WlwuT345MsrUpczbRjkfq6qrgNvfHQW2hAVfvLAIkSHh0pdEpFs\nqJQK3L96Dn7ywNAUzXcP1OKlN0+grbtf6tKmzCOK+MPfzsHtEXHf6oygH6k+mhULEhFlCMG+kw3o\n6gvMq3K+8n6srrkXr/xPFQRBwOP3LUSyUS91SUSylDs7Bs9vWorFc4240NCDZ9/4EvtPNMh6S9TP\nTzai5ko38jNjUTjXKHU5fkutUuCeFbPgcHrwzv6LUpczLRjkfqq+tQ+/3FUBh9ONR+7JQVZqpNQl\nEcmaQavBo/fm4p/vzoFSELDz0xr87z+fQHOHTerSJqyjZwBvH6iFNkTFpZnHYeXCJMxKMODomVac\nr++SuhyfY5D7oUaLDf/PrgrYB1x4+M55KJwb3Ks0EfmKIAi4YX4CXvjnZSjMMqKmoQfPvnEc7x+q\ng9Mlj0VkRFHEHz8+h0GHG+vXZiJSHyJ1SX5PoRCGPvAA+NNnNQG3nz2D3M+0dtrx7385iT67Extv\nn4sVCxKlLoko4ETqQ/DYdxfgse/kQhemwntf1OEXvz+GiosWqUvzak95A6rqOpGbHo0VCzjAbbzS\nE8OxclESGttt2Bdg+5UzyP3IlTYrtv/5BHpsDmy4NTNo91MmmimFc+Pw4j8vx21LUtHRM4hfv3sK\n//F2JRra/XM3tZMX2vHW3gsI12nwvTuy2aU+QcU3z4YuVIX3vrgUUNvfMsj9RG1jD1768wn02hx4\n8NZMFC3mvuJEMyEsRIX1azPxLw8vwby0KJy+1IFn3/gSb3x01q9GOV9u6cVv36+GWqnAj+9byBks\nk2DQanD/LXMw4HDjN+9XB0wXO4PcD1TXdeLf36rAgMONTX83D7cyxIlmXLJRj5+uz8Pm+xciKUaH\ng6easfW3R/DmZzWSB7qlux+/eucUnE4PHrlnftDuNe4LKxcmYkl2HC429OC/P78kdTk+wZVFJLbv\nRAPe/OwCFArgh9/JRUEWp5EQSUUQBCzMiEVuegwOnW7G+4cuY095Aw5UNGHVokTctiQVcVEzu9vg\nOXMn/nVn+dAtt7WZyOd7xJQIgoDv3ZGN+jYrPv6yHpkpEbL/P2WQS8Tl9uDNPRdw4GQjDFo1HvvO\nAk4xI/ITCoWAlYuScENuAg5XteCDw5ex70Qj9p9oRF5mLG5bkoqs1Mhpv0d9uKoZf/z4PFxuD9av\nzUTREvbW+UJYiAqP3ZuLF/5PGX7/4Vk8Y9QhfoY/oPkSg1wCnb0D+N3uM6i50o0Uox6P37cAsRFh\nUpdFRN+gUiqwalESbsxNQNn5Nnx2/ApOXrDg5AULEmO0uGlBIm7ITfD5FLBeuwPvfVGHAycboQtV\n4f/+7gLkzo7x6XMEu5Q4PTaum4v/78Oz+Le/nMTPNuTPeG+LrzDIZ9jRMy3Y+UkN+gddKJxrxKa/\nm8e104mmwYEK308xumlhIuaaonDO3IX6NiveOVCLdz+vRXy0FqY4PVLj9dCFqsc8x1izUQYdbnx6\nvB5/O1aPAYcbiTFaPPNPyxHCwenTYsWCRPTaHHjnQC1eevMktjwozzBngsyQHusg3tp3EcfOtCJE\nrcT37sjGyoWJnD5CJCOCICAuKgxxUWEYdLpR19yLS429aOmwo6XDji/PtiFSr0F8tBZxkWGIjQyF\nLlQ95m6F/YMunL7UgYoLFlTWdqB/0AWDVo3imzNwc14SEuMMaG8PzJ3b/MEdy9MAYCTMf7YhH/HR\n8gpzBvk0c7k92FPWgPcP1WHA4cbspHD88905sr4fQ0RAiFqJbFMUsk1RsPU7caXNiittVrR19aPb\n2o3z9d0AAEEA9GFq6MLUUCkEVF/qBASgu28Qlp4B9NgcI+eMCQ9B0eIUrFtqQlgI355nyh3L0wAB\neGd/LZ7/43F87455WJItnxU1+ZMyTVxuD46fbcP7h+rQ2tUPXagK/3BbFm7OS4JSwVl/RIFEF6ZG\ndloUstOi4PaI6OwdQFtXPzp7B9Bnd179YwcANLQPre2uVAiICQ/FvLQoZKZEoCDLiNQ4PXvpJHLH\nsjSEazXY+el5vPZeFc7kJWH92kyEqJVSl+YVg9zHBhwuHDrdgo+P1aOjdwAKQcDaghT8/cp06MPG\nvndGRPKnVAgwRobBGHntAFa32wO3KOKGnAR4RMAQNnaXO828FQsSMTspHL/5azU+r2jCmcuduPem\n2ViWE+/XrxWD3Ac8HhFnzJ04XNWCEzXtcDg9UKsUWFuQgnVLUxEbyRHpRMFOqVRAiaHVxch/Jcbo\n8It/LMR/l17CnrIGvP7BGXx01Iy7V8xCfqYRapX/9agyyCdBFEW09wzgnLkLVXWdOHu5E7YBFwAg\nLjIMN+YmYHV+MsJ1/IUlIpIbtUqJkjWZWFuQgvcPXcahqmb85q/V0IaosDg7Dsty4jEnOcJvQt1r\nkHs8Hjz33HM4f/48NBoNXnjhBaSlpY0c37dvH1555RWoVCoUFxfjgQceGPUxZrMZW7duhSAIyMzM\nxLPPPgvFDN4v9ogiBGDc96AGHW702B2wdPejrbsfbV39qG/tg7mlbyS4gaEBKkvnxeOG3ARkJIXz\nHhcRUQCIjQzDw383D3csN6G0sgnHzrSitLIJpZVNUCkVmJ1oQJYpCncsk3Zwotdn3rNnDxwOB3bt\n2oWKigrs2LEDr732GgDA6XRi+/btePfddxEWFoYNGzZgzZo1OHHixHUfs337dmzevBnLli3DM888\ng71796KoqGjaGwkAg043fv7bI+izO6ELVUEXpkaIWgmlQoBCIUAE4HR64HR7MOhwoc/uhMN1/QX1\n46LCMD89GhnJEchNj0ZCtJbhTUTjMtH57QZ9KPqs19+pizskzozEGB1K1mTi/tVzcK6+CydrLLjQ\n0I0LjT2oaehBWrwehXOlG+XuNcjLy8uxcuVKAEBeXh6qqqpGjtXW1sJkMiEiIgIAUFhYiOPHj6Oi\nouK6j6mursbSpUsBAKtWrcKhQ4dmLMjVKgUWzYlFQ7sVtv6hoO50DcLt8cDtEQEAGpUSapUCIWoF\nEmN1MGjVMIRpYIwMHRm8kmLUQetlwQciIgo8CoWAnFnRyJkVDWBoDYC2rn6kxuklrctrkFutVuj1\nXxWpVCrhcrmgUqlgtVphMBhGjul0Olit1lEfI4riyJWrTqdDX9/YixwYjYYxj09EfFw4frpxic/O\n5++MRgPuL8qWugwikgl/e7/w5fv/dDKlRPnsXJNts9cb1Hq9HjabbeRrj8cDlUp13WM2mw0Gg2HU\nx3z9frjNZkN4OLfiIyIimgqvQV5QUIDS0lIAQEVFBbKyskaOZWRkwGw2o7u7Gw6HA2VlZcjPzx/1\nMTk5OTh27BgAoLS0FIsXL/Z5g4iIiIKJIIqiONY3DI9Ar6mpgSiKePHFF3HmzBnY7XaUlJSMjFoX\nRRHFxcV46KGHrvuYjIwM1NXV4emnn4bT6cTs2bPxwgsvQKn0/1VziIiI/JXXICciIiL/5R+z2YmI\niMQqassAAAeOSURBVGhSGOREREQyFrBLtPb19eHJJ5+E1WqF0+nE1q1bkZ+fj4qKCvzrv/4rlEol\nbrrpJvzoRz+SulSf8bYKX6BwOp146qmn0NjYCIfDgUcffRRz5syRdNXAmdLR0YHvfve7eOONN6BS\nqQK+zb/97W+xb98+OJ1ObNiwAUuXLg3oNg+/VzU2NkKhUGDbtm0B+zpXVlbi3//937Fz585RV/18\n++238dZbb0GlUuHRRx/FLbfcInXZU/L1Np89exbbtm2DUqmERqPBSy+9hNjY2Mm1WQxQv/rVr8Q/\n/OEPoiiKYm1trXjvvfeKoiiK99xzj2g2m0WPxyP+0z/9k1hdXS1hlb71ySefiFu2bBFFURRPnjwp\n/uAHP5C4ounx7rvvii+88IIoiqLY1dUl3nzzzeIjjzwiHj16VBRFUXz66afFTz/9VMoSp4XD4RB/\n+MMfirfddpt48eLFgG/z0aNHxUceeUR0u92i1WoVf/3rXwd8mz/77DPx8ccfF0VRFA8ePCj+6Ec/\nCsg2/+53vxPvuusu8f777xdFUbxuG9va2sS77rpLHBwcFHt7e0f+LlffbPNDDz0knjlzRhRFUfzL\nX/4ivvjii5Nus/w/1o3ie9/7HtavXw8AcLvdCAkJgdVqhcPhgMlkgiAIuOmmm3D48GGJK/WdsVbh\nCyS33347fvzjHwMY2sBGqVR+a9XAQHpdh7300ktYv3494uKGloIM9DYfPHgQWVlZeOyxx/CDH/wA\nq1evDvg2p6enw+12w+PxwGq1QqVSBWSbTSYTXn755ZGvr9fGU6dOIT8/HxqNBgaDASaTCefOnZOq\n5Cn7Zpt/+ctfYt68eQC+yqjJtjkggvydd97BXXfddc2fy5cvIzQ0FO3t7XjyySfxk5/85Fsrzo1n\ndbn/v717CYlyj8M4/m1gLMxssFVZkbPpQkggRoIUSDAUShsFKaxoU1Fo47ULwkSTUNailZBEUYoX\naEyDok3FZISLpMWATDQbSVTKjEmLGJv3LKL3XNIOx3M4w/uf57N831n8H+byey/D8zrJQo16plm+\nfDlZWVnMzMxQXV3NqVOn/nFroNOEQiFycnLsAzXA+MzT09NEIhGuXbvG+fPnqa+vNz5zZmYmY2Nj\n7Nmzh+bmZqqqqozM7PP57GIxmP+zvFBzqFP9NfOPA/Lh4WE6Ojo4fPjwojMbcY+8oqKCioqKn7ZH\no1Fqa2tpbGxk+/btzMzM/NREZ1K73K9a+EwzPj7OiRMn2L9/P2VlZbS2ttr7THtfAe7evcuSJUt4\n8eIFIyMjNDU18eHDB3u/iZk9Hg9er5eMjAy8Xi9Lly5lYmLC3m9i5lu3blFcXExdXR3j4+McOnSI\nRCJh7zcxMzBv6+dCzaEmefDgAW1tbVy/fp2cnJxFZzbijHw+b968oaamhqtXr7Jr1y7g+6Bzu92M\njo5iWRaDg4NGtcv9qoXPJO/fv+fIkSM0NDRQXl4OmN8a2NnZSUdHB3fu3GHz5s1cunSJnTt3Gp25\noKCAZ8+eYVkWk5OTfPnyhaKiIqMzZ2dn2z/cK1euZG5uzvjPNsz//c3Pz+fly5d8/fqVT58+EYvF\njPpN6+/vt7/T69atA1h0ZmMLYY4fP040GiU39/tj/rKysmhra+PVq1e0tLTw7ds3iouL8fv9KV7p\nf2ehRj3TBINBHj58iNfrtbedO3eOYDCYFq2BVVVVBAIBXC6X8U2Jly9fZmhoCMuy8Pv9rF271ujM\ns7OznD17lnfv3pFIJDh48CBbt241MvPbt2+pra2lt7d3wdbP3t5eenp6sCyLo0eP4vP5Ur3sf+VH\n5q6uLoqKili9erV9haWwsJDq6upFZTZ2kIuIiKQDYy+ti4iIpAMNchEREQfTIBcREXEwDXIREREH\n0yAXERFxMA1ykTT2+vVrNm7cyKNHj1K9FBFZJA1ykTQWCoXw+Xx0d3eneikiskhm9neKyN+am5tj\nYGCAzs5OKisrGR0dZf369QwNDdmFHNu2bSMWi9mPmgwEAnz8+JFly5bR3NzMli1bUh1DJO3pjFwk\nTT19+pQ1a9aQl5fH7t276e7uJpFI0NjYSGtrK/fu3ftTV39TUxMNDQ309fVx4cIFo1oRRZxMg1wk\nTYVCIUpLSwHYu3cvfX19jIyMsGrVKjZt2gRgd9nPzs4SiUQ4c+YM+/bto66ujs+fPzM9PZ2y9YvI\nd7q0LpKGpqamCIfDRCIRbt++jWVZxONxwuEwyWTyp9cnk0kyMjLo7++3t01MTODxeP7PZYvIPHRG\nLpKGBgYG2LFjB+FwmMePH/PkyROOHTvG4OAg8XicaDQKwP379wFYsWIFGzZssAf58+fPOXDgQMrW\nLyK/00NTRNJQWVkZfr+fkpISe9vU1BQlJSXcuHGDYDCIy+UiLy+PeDxOe3s7sVjM/rOb2+0mEAiQ\nn5+fwhQiAhrkIvIHyWSSK1eucPLkSTIzM7l58yaTk5OcPn061UsTkQXoHrmI2FwuFx6Ph/Lyctxu\nN7m5uVy8eDHVyxKRX9AZuYiIiIPpz24iIiIOpkEuIiLiYBrkIiIiDqZBLiIi4mAa5CIiIg6mQS4i\nIuJgvwHxVdTl5FyvnAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xaeb8d2cb38>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot(data.Age.fillna(95))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are 2 numerical variables in this dataset, Fare and Age. So let's go ahead and find out whether they present values that we could consider outliers.\n",
    "\n",
    "### Fare"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x26ede2be10>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHvNJREFUeJzt3XuUHWWZ7/HvjxCujUQE2pAEEzSo3IykD+MIerpBhBEh\niMpEkcMoGl0LFZXjGPCCLszIqGEcQeYQDcecSbDNBDAZEBViWnQNGAgEQgIMAYIQQyL3NJdIwnP+\nqLdh06nsru707tq99++z1l5d9dbteba4n9RbVW8pIjAzM+tth7IDMDOz+uQCYWZmuVwgzMwslwuE\nmZnlcoEwM7NcLhBmZpbLBcKsHyQdIKl7EPf3E0nnpen3SFoziPtul7RysPZnzccFwuqOpDWSnpfU\nXfHZbwiO+0lJWyqO+aCkyyVN7FknIh6IiJaC++rqa72I+GRE/NN2ho6kHSWFpPEV++6KiIO3d9/W\nvFwgrF6dGBEtFZ8/93cHkkYM4Li/TwVgT+A9wIvArZLeOoB9VTXA+MyGjAuEDRuSdpC0QNKjkp6S\n1FX5wy1prqQfSfqVpGeBd0naRdJFkh6WtF7SpZJ26etYEbElIu6PiE8DNwHnp2O8SdLLww9IOjOd\n8WyU9ICkqZIOBS5Jx++W9FiV+OZK+mavPL8h6fF0BjO1ov0Pkv6hYr7yLOXG9HdlOuYHe3dZSTpY\n0u/Sd7dC0gm9vrsfSrou5XKTpAl9fU/W2FwgbLi5BpgIvB64C/j3Xss/CnwL2IPsh/17wATgsLTd\neOCr/TzmVcC7ejdKeg1wEXBsROwBHAncGRErgM+SzkYiYu8q8fU2Ni3bDzgTuFzSmwrE+O709+B0\nzCt7xboT2Xd3LbAP8EXg5732/VHg68BewJ+ACwoc1xqYC4TVq1+kf+k+JekXABHxUkT8NCI2RsQL\nwDeByZJ2r9ju6oi4KSJeIuse+hTwhYh4MiKeAb4DTKV//kz2o5kngEMk7RIR6yJiVR/7ejm+iNiU\ns/wl4PyI2BQRvwV+BXy4n/HmORLYCfheRLwYETcA1/Hq72JBRNwaES8C84BJg3BcG8ZcIKxenRwR\no9LnZMj67CV9N3XlPAOsTutW/gv94Yrp1wM7A3f0FBuyf0Xv289YxgBP9G5MBecjwFnAo5KukXRg\nH/t6uI/lj0fEcxXzD5GdTWyv/YA/xatH53yILLcej1ZMPwf0eTHeGpsLhA0n/wt4H3A02UXknu4R\nVaxT+QO4Hvgr8OaKYrNnROzZz+OeDPw+b0FEXBcR7wFGkxWsy3LieNUmfRzrdZJ2rZjfn+wMBuBZ\nYLeKZa/vx37/DIyTVPld7Q+s7WM7a2IuEDac7AFsAh4n+6GcUW3liNgC/AT4gaR9lBkr6b19HSid\nrRwg6VLgKHL64yWNlnSipN3ICtGzZF1EkBWnsZJG9iM/yP4/+U1JO0lqB/4OWJCWLQc+KGnXdKby\niV65Pg4csI39/hewGThH0khJR5MV25/3Mz5rIi4QNpz8X7J/Cf8ZWEn2o9eXc8i6UpYCTwO/IbtY\nvS3vSg/CPQP8lqwQtUVE3gNnI4AvA+vIfpzfSdbdBHA9cB+wXtKjOdtuyyNkhWYdMAf4ZETcl5Z9\nn+xMYQNwOTC317bnA1ek7rRTKhek6x0nAlOAx4AfAh+t2LfZVuQXBpmZWR6fQZiZWS4XCDMzy+UC\nYWZmuVwgzMws14612nEa7+ZGsgeVdiR7SvP8NO7Mp4C/pFXPi4hfpm3OJRteYAvw+Yj4dbVj7L33\n3jF+/PgBx/jss8+y++67971iA3CujauZ8m2mXKF2+S5btuyxiNinzxUjoiYfsoeXWtL0SOCPwDvI\nhkf43znrHwTcQVZQJgD3AyOqHWPy5MmxPZYsWbJd2w8nzrVxNVO+zZRrRO3yBW6NAr/jNetiSnH0\nvFhlZPpUu6d2CtAZ2Rg0D5I9lXpEreIzM7PqavocRBrvfhnZkAg/ioivpC6mj5M9tHQrcE5EPCnp\nEuDmiJibtp0NXBcRC3rtcxowDaC1tXVyZ2fngOPr7u6mpaU5hptxro2rmfJtplyhdvl2dHQsi4i2\nPlcscpqxvR9gFLAEOARoJXsCdQeyoRIuT+tcAnysYpvZwIeq7dddTMU518bVTPk2U64RDdzF1KsI\nPZUKxPERsT6yl7G8BPyYV7qR1gLjKjYbiwcSMzMrTc0KRBocbVSa3hU4FrhH0uiK1T5A9tIXgEXA\nVEk7pzdZTSQbP8fMzEpQs9tcyYY/npOuQ+wAzI+IayT9u6RJZBes1wCfBoiIlZLmA6vIRp08K7IR\nKs3MrAQ1KxARcSfw9pz206tsM4M+hnA2M7Oh4SepzcwslwuEmZnlquU1iLq3Yu3T/MP0a7dqX3Ph\nCSVEY2ZWX3wGYWZmuVwgzMwslwuEmZnlcoEwM7NcLhBmZpbLBcLMzHK5QJiZWS4XCDMzy+UCYWZm\nuVwgzMwslwuEmZnlcoEwM7NcLhBmZpbLBcLMzHK5QJiZWS4XCDMzy+UCYWZmuVwgzMwsV80KhKRd\nJC2VdIeklZK+ldr3knS9pPvS39dWbHOupNWS7pV0XK1iMzOzvtXyDGITcHREvA2YBBwv6R3AdGBx\nREwEFqd5JB0ETAUOBo4HLpU0oobxmZlZFTUrEJHpTrMj0yeAKcCc1D4HODlNTwE6I2JTRDwIrAaO\nqFV8ZmZWnSKidjvPzgCWAW8CfhQRX5H0VESMSssFPBkRoyRdAtwcEXPTstnAdRGxoNc+pwHTAFpb\nWyd3dnYOOL4NTzzN+ue3bj90zJ4D3me96u7upqWlpewwhkQz5QrNlW8z5Qq1y7ejo2NZRLT1td6O\ng37kChGxBZgkaRRwtaRDei0PSf2qUBExC5gF0NbWFu3t7QOO7+J5C5m5YuuvYM1pA99nverq6mJ7\nvqvhpJlyhebKt5lyhfLzHZK7mCLiKWAJ2bWF9ZJGA6S/G9Jqa4FxFZuNTW1mZlaCWt7FtE86c0DS\nrsCxwD3AIuCMtNoZwMI0vQiYKmlnSROAicDSWsVnZmbV1bKLaTQwJ12H2AGYHxHXSLoJmC/pTOAh\n4FSAiFgpaT6wCtgMnJW6qMzMrAQ1KxARcSfw9pz2x4FjtrHNDGBGrWIyM7Pi/CS1mZnlcoEwM7Nc\nLhBmZpbLBcLMzHK5QJiZWS4XCDMzy+UCYWZmuVwgzMwslwuEmZnlcoEwM7NcLhBmZpbLBcLMzHK5\nQJiZWS4XCDMzy+UCYWZmufosEJJ2l7RDmj5Q0kmSRtY+NDMzK1ORM4gbgV0kjQF+A5wO/LSWQZmZ\nWfmKFAhFxHPAKcClEfFh4ODahmVmZmUrVCAk/S1wGnBtahtRu5DMzKweFCkQZwPnAldHxEpJBwBL\nahuWmZmVbcdqCyWNAE6KiJN62iLiAeDztQ7MzMzKVfUMIiK2AEcNUSxmZlZHinQx3S5pkaTTJZ3S\n8+lrI0njJC2RtErSSklnp/ZvSloraXn6vK9im3MlrZZ0r6TjtiMvMzPbTlW7mJJdgMeBoyvaAriq\nj+02A+dExG2S9gCWSbo+LfuXiPh+5cqSDgKmkt0htR9wg6QD01mMmZkNsT4LRER8fCA7joh1wLo0\nvVHS3cCYKptMATojYhPwoKTVwBHATQM5vpmZbR9FRPUVpAOBfwNaI+IQSYeRXbj+duGDSOPJHrg7\nBPgS8HHgaeBWsrOMJyVdAtwcEXPTNrOB6yJiQa99TQOmAbS2tk7u7OwsGsZWNjzxNOuf37r90DF7\nDnif9aq7u5uWlpaywxgSzZQrNFe+zZQr1C7fjo6OZRHR1td6RbqYfgx8GbgMICLulHQFUKhASGoB\nrgS+EBHPSPo34AKybqoLgJnAJ4rsKx1/FjALoK2tLdrb24tuupWL5y1k5oqtv4I1pw18n/Wqq6uL\n7fmuhpNmyhWaK99myhXKz7fIRerdImJpr7bNRXaexmy6EpgXEVcBRMT6iNgSES+RFZ8j0uprgXEV\nm49NbWZmVoIiBeIxSW8k+xc/kj5EurZQjSQBs4G7I+KiivbRFat9ALgrTS8CpkraWdIEYCLQuzCZ\nmdkQKdLFdBZZl85bJK0FHgQ+VmC7I8kG9lshaXlqOw/4iKRJZAVnDfBpgPSU9nxgFdkZylm+g8nM\nrDxF7mJ6AHiPpN2BHSJiY5EdR8QfAOUs+mWVbWYAM4rs38zMaqvPAiHpS73mIbsDaVlELM/dyMzM\nhr0i1yDagM+QPcMwhqxL6Hjgx5L+sYaxmZlZiYpcgxgLHB4R3QCSzicb9vvdwDLgu7ULz8zMylLk\nDGJfYFPF/ItkD80936vdzMwaSJEziHnAHyUtTPMnAleki9arahaZmZmVqshdTBdI+hXwztT0mYi4\nNU2fVrPIzMysVEXOIABuI3uqeUcASftHxJ9qFpWZmZWuyG2unwPOB9YDW8iebQjgsNqGZmZmZSpy\nBnE28OaIeLzWwZiZWf0ochfTw2QPxpmZWRMpcgbxANAl6VoqbmutHIDPzMwaT5EC8af02Sl9zMys\nCRS5zfVbAJJ2i4jnah+SmZnVgz6vQUj6W0mrgHvS/NskXVrzyMzMrFRFLlL/ADgOeBwgIu4gG4fJ\nzMwaWJECQUQ83KvJL/IxM2twRS5SPyzpnUCkd0yfDdxd27DMzKxsRc4gPkP22tExZMNtTErzZmbW\nwIrcxfQYHpTPzKzpFLmL6buSXiNppKTFkv4i6WNDEZyZmZWnSBfTeyPiGeD9wBrgTcCXaxmUmZmV\nr0iB6OmGOgH4j4jwuExmZk2gSIG4RtI9wGRgsaR9gBf62kjSOElLJK2StFLS2al9L0nXS7ov/X1t\nxTbnSlot6V5Jxw00KTMz2359FoiImE72Nrm2iHgReBaYUmDfm4FzIuIg4B3AWZIOAqYDiyNiIrA4\nzZOWTQUOBo4HLpU0ov8pmZnZYChykfrDwIsRsUXS14C5wH59bRcR6yLitjS9kezZiTFkxWVOWm0O\ncHKangJ0RsSmiHgQWA0c0c98zMxskCgiqq8g3RkRh0k6Cvg28D3gGxHxN4UPIo0HbgQOAf4UEaNS\nu4AnI2KUpEuAmyNiblo2G7guIhb02tc0YBpAa2vr5M7OzqJhbGXDE0+z/vmt2w8ds+eA91mvuru7\naWlpKTuMIdFMuUJz5dtMuULt8u3o6FgWEW19rVfkSeqeYTVOAGZFxLWSvl00EEktwJXAFyLimawm\nZCIiJFWvUL1ExCxgFkBbW1u0t7f3Z/NXuXjeQmau2PorWHPawPdZr7q6utie72o4aaZcobnybaZc\nofx8i1ykXivpMuDvgV9K2rngdqShOa4E5kXEVal5vaTRafloYEPPcYBxFZuPTW1mZlaCIj/0pwK/\nBo6LiKeAvSjwHETqPpoN3N3r7XOLgDPS9BnAwor2qZJ2ljQBmAgsLZSFmZkNuiJDbTwHXCVpX0n7\np+Z7Cuz7SOB0YIWk5antPOBCYL6kM4GHyAoQEbFS0nxgFdkdUGdFhEeNNTMrSZ8FQtJJwEyyO5c2\nAPuTFYiDq20XEX8AtI3Fx2xjmxnAjL5iMjOz2ivSxXQB2XMM/x0RE4D3ADfXNCozMytdkQLxYkQ8\nDuwgaYeIWAL0eXuUmZkNb0Vuc30q3ap6IzBP0gayp6nNzKyBFTmDmAI8B3wR+BVwP3BiLYMyM7Py\nVT2DkHQy2fDeKyLi17wyRIaZmTW4bRYISZeS3an0X8AFko6IiAuGLLISjZ9+bW77mgtPGOJIzMzK\nU+0M4t3A29IgfbsBvye7o8nMzJpAtWsQf+15UC09LLetZxrMzKwBVTuDeIukO9O0gDemeZGNs3dY\nzaMzM7PSVCsQbx2yKMzMrO5ss0BExENDGYiZmdWXQsN2m5lZ83GBMDOzXNssEJIWp7//PHThmJlZ\nvah2kXq0pHcCJ0nqpNdtrhFxW00jMzOzUlUrEN8Avk726s+Lei0L4OhaBWVmZuWrdhfTAmCBpK83\nyxAbZmb2iiKvHL0gvVXu3ampKyKuqW1YZmZWtj7vYpL0HeBssndFrwLOlvRPtQ7MzMzKVeSFQScA\nkyLiJQBJc4DbgfNqGZiZmZWr6HMQoyqm96xFIGZmVl+KFIjvALdL+mk6e1gGzOhrI0mXS9og6a6K\ntm9KWitpefq8r2LZuZJWS7pX0nEDScbMzAZPkYvUP5PUBfyP1PSViHi0wL5/ClwC/L9e7f8SEd+v\nbJB0EDCV7AVF+wE3SDqwZ7hxMzMbekWuQRAR64BF/dlxRNwoaXzB1acAnRGxCXhQ0mrgCOCm/hzT\nzMwGTxljMX1O0p2pC+q1qW0M8HDFOo+kNjMzK4kionY7z84gromIQ9J8K/AY2ZPYFwCjI+ITki4B\nbo6IuWm92cB16WG93vucBkwDaG1tndzZ2Tng+DY88TTrny++/qFjhu/1+e7ublpaWsoOY0g0U67Q\nXPk2U65Qu3w7OjqWRURbX+tV7WKSNAJYGRFvGYygImJ9xb5/DPQ8cLcWGFex6tjUlrePWcAsgLa2\ntmhvbx9wPBfPW8jMFYV62TIrns1tXnPhCQOOYah0dXWxPd/VcNJMuUJz5dtMuUL5+VbtYkoXie+V\ntP9gHEzS6IrZDwA9dzgtAqZK2lnSBGAisHQwjmlmZgNT5J/PrwVWSloKvPxP6Ig4qdpGkn4GtAN7\nS3oEOB9olzSJrItpDfDptK+VkuaTPam9GTjLdzCZmZWrSIH4+kB2HBEfyWmeXWX9GRR4vsLMzIZG\nkecgfifpDcDEiLhB0m7AiNqHZmZmZSoyWN+ngAXAZalpDPCLWgZlZmblK/IcxFnAkcAzABFxH7Bv\nLYMyM7PyFSkQmyLirz0zknYku8hsZmYNrEiB+J2k84BdJR0L/Afwn7UNy8zMylakQEwH/gKsILst\n9ZfA12oZlJmZla/IXUwvpWG+/0jWtXRv1HJ8DjMzqwt9FghJJwD/B7gfEDBB0qcj4rpaB2dmZuUp\n8qDcTKAjIlYDSHojcC3gAmFm1sCKXIPY2FMckgeAjTWKx8zM6sQ2zyAknZImb5X0S2A+2TWIDwO3\nDEFsZmZWompdTCdWTK8H/mea/guwa80iMjOzurDNAhERHx/KQMzMrL4UuYtpAvA5YHzl+n0N921m\nZsNbkbuYfkE2TPd/Ai/VNhwzM6sXRQrECxHxw5pHYmZmdaVIgfhXSecDvwE29TRGxG01i8rMzEpX\npEAcCpwOHM0rXUyR5s3MrEEVKRAfBg6oHPLbzMwaX5Enqe8CRtU6EDMzqy9FziBGAfdIuoVXX4Pw\nba5mZg2sSIE4v+ZRmJlZ3SnyPojfDWTHki4H3g9siIhDUttewM/JHrpbA5waEU+mZecCZwJbgM9H\nxK8HclwzMxscfV6DkLRR0jPp84KkLZKeKbDvnwLH92qbDiyOiInA4jSPpIOAqcDBaZtLJY3oRx5m\nZjbI+iwQEbFHRLwmIl5DNkjfB4FLC2x3I/BEr+YpwJw0PQc4uaK9MyI2RcSDwGrgiGIpmJlZLWgg\nbw+VdHtEvL3AeuOBayq6mJ6KiFFpWsCTETFK0iXAzRExNy2bDVwXEQty9jkNmAbQ2to6ubOzs9/x\n99jwxNOsf37Am7/s0DF7bv9Oaqy7u5uWlpaywxgSzZQrNFe+zZQr1C7fjo6OZRHR1td6RQbrO6Vi\ndgegDXhhO2IDICJCUr+rU0TMAmYBtLW1RXt7+4BjuHjeQmauKHKdvro1pw08hqHS1dXF9nxXw0kz\n5QrNlW8z5Qrl51vk17HyvRCbyS4uTxng8dZLGh0R6ySNBjak9rXAuIr1xqY2MzMrSZG7mAbzvRCL\ngDOAC9PfhRXtV0i6CNgPmAgsHcTjmplZP1V75eg3qmwXEXFBtR1L+hnQDuwt6RGy5ykuBOZLOhN4\nCDg17WylpPnAKrKzlLMiYkt/EjEzs8FV7Qzi2Zy23cmeVXgdULVARMRHtrHomG2sPwOYUW2fZmY2\ndKq9cnRmz7SkPYCzgY8DncDMbW1nZmaNoeo1iPTk85eA08ieWzi858lnMzNrbNWuQXwPOIXsltJD\nI6J7yKIyM7PSVXuS+hyyO4q+Bvy5YriNjQWH2jAzs2Gs2jWIIu+KMDOzBuUiYGZmuVwgzMwslwuE\nmZnlcoEwM7NcLhBmZpbLBcLMzHK5QJiZWS4XCDMzy+UCYWZmuVwgzMwslwuEmZnlcoEwM7NcLhBm\nZpbLBcLMzHK5QJiZWS4XCDMzy+UCYWZmubb5RrlakrQG2AhsATZHRJukvYCfA+OBNcCpEfFkGfGZ\nmVm5ZxAdETEpItrS/HRgcURMBBaneTMzK0k9dTFNAeak6TnAySXGYmbW9BQRQ39Q6UHgabIupssi\nYpakpyJiVFou4Mme+V7bTgOmAbS2tk7u7OwccBwbnnia9c8PePM+HTpmz9rtvJ+6u7tpaWkpO4wh\n0Uy5QnPl20y5Qu3y7ejoWFbRe7NNpVyDAI6KiLWS9gWul3RP5cKICEm5lSsiZgGzANra2qK9vX3A\nQVw8byEzV9TuK1hzWnvN9t1fXV1dbM93NZw0U67QXPk2U65Qfr6ldDFFxNr0dwNwNXAEsF7SaID0\nd0MZsZmZWWbIC4Sk3SXt0TMNvBe4C1gEnJFWOwNYONSxmZnZK8roYmoFrs4uM7AjcEVE/ErSLcB8\nSWcCDwGnlhCbmZklQ14gIuIB4G057Y8Dxwx1PGZmlq+ebnM1M7M6UtZdTE1t/PRrc9vXXHjCEEdi\nZrZtPoMwM7NcLhBmZpbLBcLMzHK5QJiZWS4XCDMzy+W7mIYB3/VkZmXwGYSZmeVygTAzs1wuEGZm\nlssFwszMcrlAmJlZLhcIMzPL5dtc68i2bmft7/q+/dXMBoMLRA319wffzKyeuIvJzMxyuUCYmVku\nFwgzM8vlaxA2qHzh3KxxuEA0kYFcNN/WD7svwJs1PhcIsxrx2ZQNd3VXICQdD/wrMAL4SURcWHJI\nNoz190e62pmRf9it2dRVgZA0AvgRcCzwCHCLpEURsarcyMzMylHmmWhdFQjgCGB1RDwAIKkTmAK4\nQPRD3n9Q5xy6mfr7n7u8//gH8xqKr8dYo1JElB3DyyR9CDg+Ij6Z5k8H/iYiPluxzjRgWpp9M3Dv\ndhxyb+Cx7dh+OHGujauZ8m2mXKF2+b4hIvbpa6X6+ydlHyJiFjBrMPYl6daIaBuMfdU759q4minf\nZsoVys+33h6UWwuMq5gfm9rMzGyI1VuBuAWYKGmCpJ2AqcCikmMyM2tKddXFFBGbJX0W+DXZba6X\nR8TKGh5yULqqhgnn2riaKd9myhVKzreuLlKbmVn9qLcuJjMzqxMuEGZmlqspC4Sk4yXdK2m1pOll\nxzMYJF0uaYOkuyra9pJ0vaT70t/XViw7N+V/r6Tjyol6YCSNk7RE0ipJKyWdndobLl9Ju0haKumO\nlOu3UnvD5dpD0ghJt0u6Js03cq5rJK2QtFzSramtfvKNiKb6kF38vh84ANgJuAM4qOy4BiGvdwOH\nA3dVtH0XmJ6mpwP/nKYPSnnvDExI38eIsnPoR66jgcPT9B7Af6ecGi5fQEBLmh4J/BF4RyPmWpHz\nl4ArgGvSfCPnugbYu1db3eTbjGcQLw/nERF/BXqG8xjWIuJG4IlezVOAOWl6DnByRXtnRGyKiAeB\n1WTfy7AQEesi4rY0vRG4GxhDA+Ybme40OzJ9ggbMFUDSWOAE4CcVzQ2ZaxV1k28zFogxwMMV84+k\ntkbUGhHr0vSjQGuabpjvQNJ44O1k/7JuyHxTl8tyYANwfUQ0bK7AD4B/BF6qaGvUXCEr9jdIWpaG\nEYI6yreunoOw2omIkNRQ9zRLagGuBL4QEc9IenlZI+UbEVuASZJGAVdLOqTX8obIVdL7gQ0RsUxS\ne946jZJrhaMiYq2kfYHrJd1TubDsfJvxDKKZhvNYL2k0QPq7IbUP++9A0kiy4jAvIq5KzQ2bL0BE\nPAUsAY6nMXM9EjhJ0hqyrt+jJc2lMXMFICLWpr8bgKvJuozqJt9mLBDNNJzHIuCMNH0GsLCifaqk\nnSVNACYCS0uIb0CUnSrMBu6OiIsqFjVcvpL2SWcOSNqV7F0p99CAuUbEuRExNiLGk/3/8rcR8TEa\nMFcASbtL2qNnGngvcBf1lG/ZV/HL+ADvI7vz5X7gq2XHM0g5/QxYB7xI1jd5JvA6YDFwH3ADsFfF\n+l9N+d8L/F3Z8fcz16PI+m7vBJanz/saMV/gMOD2lOtdwDdSe8Pl2ivvdl65i6khcyW7k/KO9FnZ\n81tUT/l6qA0zM8vVjF1MZmZWgAuEmZnlcoEwM7NcLhBmZpbLBcLMzHL5SWqzfpK0BVhR0XRyRKwp\nKRyzmvFtrmb9JKk7IloGsN2OEbG5FjGZ1YK7mMwGgaTxkn4v6bb0eWdqb0/ti4BVqe1j6R0PyyVd\nJmlEqcGbbYMLhFn/7Zp+3JdLujq1bQCOjYjDgb8Hflix/uHA2RFxoKS3puVHRsQkYAtw2lAGb1aU\nr0GY9d/z6ce90kjgEkk9P/oHVixbGtn4/QDHAJOBW9Los7vyymBsZnXFBcJscHwRWA+8jezM/IWK\nZc9WTAuYExHnDmFsZgPiLiazwbEnsC4iXgJOJ3u1bZ7FwIfS+P897x9+wxDFaNYvLhBmg+NS4AxJ\ndwBv4dVnDS+LiFXA14DfSLoTuJ7sHdtmdce3uZqZWS6fQZiZWS4XCDMzy+UCYWZmuVwgzMwslwuE\nmZnlcoEwM7NcLhBmZpbr/wMG4Fa7+ufJbAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x26eddd8f60>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# First let's plot a histogram to get an idea of the distribution\n",
    "\n",
    "fig = data.Fare.hist(bins=50)\n",
    "fig.set_title('Fare Distribution')\n",
    "fig.set_xlabel('Fare')\n",
    "fig.set_ylabel('Number of Passengers')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The distribution of Fare is skewed, so in principle, we shouldn't estimate outliers using the mean plus minus 3 standard deviations methods, which assumes a normal distribution of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x26ee3647b8>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFrxJREFUeJzt3X+QVed93/H3lx9eNGC7yMg7VMiBtig2Ckg4O4ojd9ql\nMkHTxKAZpTKeWINmVJiVgHGDPAHcH5JnSiTFsieJZESgTU3d1BKOirVyNBaIcNsmsiVBI0NAkYUi\nVFCQFIh+eKm7XeDbP/Yg3xUH2F3ds3eveL9mdu45zz3n7HeZy372ec45z4nMRJKkdxvT7AIkSaOT\nASFJKmVASJJKGRCSpFIGhCSplAEhSSplQEiSShkQkqRSBoQkqdS4ZhfwXkyZMiWnT5/e7DKkUseP\nH2fixInNLkM6w+7du49m5iXn266lA2L69Ons2rWr2WVIpWq1Gp2dnc0uQzpDRLw8mO0cYpIklTIg\nJEmlDAhJUikDQpJUyoCQJJUyIKQGW7BgAWPGjGHevHmMGTOGBQsWNLskaVgMCKmBFixYwLZt2+jq\n6uLRRx+lq6uLbdu2GRJqSS19H4Q02mzfvp1bb72V9evXU6vVWL9+PQAbNmxocmXS0NmDkBooM7nr\nrrsGtN1111347He1okoDIiIORsTeiHg2InYVbRdHxPaIeKF4nVy3/dqIOBARz0eEfXK1nIhg7dq1\nA9rWrl1LRDSpImn4RqIHMS8zr8rMjmJ9DbAjM2cCO4p1ImIWsBi4ArgOWB8RY0egPqlh5s+fzwMP\nPMBtt91GT08Pt912Gw888ADz589vdmnSkEWVXd+IOAh0ZObRurbngc7MPBIRU4FaZv58RKwFyMy7\niu0eB+7MzB+c7fgdHR3pXEwabRYsWMD27dvJTCKC+fPn8/jjjze7LOkdEbG77o/2s6q6B5HAExGx\nOyKWFW3tmXmkWH4VaC+WLwUO1e17uGiTWsrjjz/OqVOn2LlzJ6dOnTIc1LKqvorpH2fmKxHxUWB7\nRPxV/ZuZmRExpC5METTLANrb26nVag0rVmqknp4eP59qaZUGRGa+Ury+HhFbgauB1yJiat0Q0+vF\n5q8Al9XtPq1oe/cxNwIboX+IyemUNVo53bdaXWVDTBExMSI+eHoZ+BXgL4FuYEmx2RLgkWK5G1gc\nEW0RMQOYCTxdVX2SpHOrsgfRDmwtLu8bB/zXzPx+RDwDbImIW4CXgRsBMnNfRGwB9gMngOWZebLC\n+iRJ51BZQGTmXwNXlrQfA649yz7rgHVV1SRJGjzvpJYklTIgJEmlDAhJUikDQpJUyoCQJJUyICRJ\npQwISVIpA0KSVMqAkCSVMiAkSaUMCElSKQNCklTKgJAklTIgJEmlDAhJUikDQpJUyoCQJJUyICRJ\npQwISVIpA0KSVMqAkCSVMiAkSaUMCElSKQNCklTKgJAklTIgJEmlDAhJUikDQpJUyoCQJJUyICRJ\npSoPiIgYGxF/ERHfK9YvjojtEfFC8Tq5btu1EXEgIp6PiAVV1yZJOruR6EF8EXiubn0NsCMzZwI7\ninUiYhawGLgCuA5YHxFjR6A+SVKJSgMiIqYBvwr8h7rmRcDmYnkzcH1d+4OZ2ZuZLwEHgKurrE+S\ndHZV9yB+F/gt4FRdW3tmHimWXwXai+VLgUN12x0u2iRJTTCuqgNHxK8Br2fm7ojoLNsmMzMicojH\nXQYsA2hvb6dWq73XUqVK9PT0+PlUS6ssIIBPAwsj4p8DE4APRcR/AV6LiKmZeSQipgKvF9u/AlxW\nt/+0om2AzNwIbATo6OjIzs7OCn8EafhqtRp+PtXKKhtiysy1mTktM6fTf/L5TzPzC0A3sKTYbAnw\nSLHcDSyOiLaImAHMBJ6uqj5J0rlV2YM4m7uBLRFxC/AycCNAZu6LiC3AfuAEsDwzTzahPkkSIxQQ\nmVkDasXyMeDas2y3Dlg3EjVJks7NO6klSaUMCElSKQNCklTKgJAklTIgJEmlDAhJUikDQpJUyoCQ\nJJUyICRJpQwISVIpA0KSVMqAkCSVMiAkSaUMCElSKQNCklTKgJAklTIgJEmlDAhJUikDQpJUyoCQ\nJJUyICRJpQwISVIpA0KSVMqAkCSVMiAkSaUMCElSKQNCklTKgJAklTIgJEmlDAhJUqnKAiIiJkTE\n0xHxo4jYFxFfKdovjojtEfFC8Tq5bp+1EXEgIp6PiAVV1SZJOr8qexC9wD/LzCuBq4DrIuJTwBpg\nR2bOBHYU60TELGAxcAVwHbA+IsZWWJ8k6RwqC4js11Osji++ElgEbC7aNwPXF8uLgAczszczXwIO\nAFdXVZ8k6dzGVXnwogewG/hHwDcy86mIaM/MI8UmrwLtxfKlwA/rdj9ctL37mMuAZQDt7e3UarWK\nqpfem56eHj+fammVBkRmngSuioi/B2yNiF941/sZETnEY24ENgJ0dHRkZ2dno8qVGqpWq+HnU61s\nRK5iysw3gZ30n1t4LSKmAhSvrxebvQJcVrfbtKJNktQEVV7FdEnRcyAiLgLmA38FdANLis2WAI8U\ny93A4ohoi4gZwEzg6arqkySdW5VDTFOBzcV5iDHAlsz8XkT8ANgSEbcALwM3AmTmvojYAuwHTgDL\niyEqSVITVBYQmbkHmFvSfgy49iz7rAPWVVWTJGnwvJNaklTKgJAklRpUQES/L0TEvyvWPxYR3sQm\nSe9jg+1BrAd+Gfh8sf4T4BuVVCRJGhUGe5L6lzLzkxHxFwCZ+UZEfKDCuiRJTTbYHkRfcblqQv89\nDsCpyqqSWticOXOICObNm0dEMGfOnGaXJA3LYAPi94GtwEcjYh3wZ8BvV1aV1KLmzJnD3r17GTOm\n/7/WmDFj2Lt3ryGhljSoIabM/KOI2E3//QsBXJ+Zz1VamdSC9u7dS0Tw1a9+lVmzZrF//36+9KUv\nsXfv3maXJg3ZeQOiGFral5kfp3+qDEnncOedd7Jq1SpqtRqrVq2ip6eHO+64o9llSUN23iGmYrqL\n5yPiYyNQj9TynnrqqXOuS61isOcgJgP7ImJHRHSf/qqyMKkVRQSPPfYYixYt4s0332TRokU89thj\nRESzS5OGbLCXuf7bSquQ3ieWL1/O/fffT3d3N93d3QPapVYz2JPU/73qQqT3i4hg7NixnDhxgnHj\nxnHypJMSqzUNdqqNT0XEMxHRExH/LyJORsTbVRcntZpNmzZx77330tfXx86dO+nr6+Pee+9l06ZN\nzS5NGrLBnoO4n/5pNl4ALgL+JU61IZ2ht7eXrq6uAW1dXV309vY2qSJp+AY9m2tmHgDGZubJzPxP\n9D8+VFKdtrY2NmzYMKBtw4YNtLW1NakiafgGe5L6/xRzLz0bEb8DHMGpwqUzLF26lNWrVwMwa9Ys\nvv71r7N69eozehVSKxhsQNxEfyCsAH4TuAy4oaqipFZ13333AfDlL3+Z3t5e2tra6OrqeqddaiWR\nmWd/M+Jjmfm/R7CeIeno6Mhdu3Y1uwypVK1Wo7Ozs9llSGeIiN2Z2XG+7c43TPTdugM+/J6rki4A\nzuaq94vzBUT97Z//oMpCpPeD07O5Lly4kK1bt7Jw4UJnc1XLOl9A5FmWJZXYu3cvc+fO5cUXX+SG\nG27gxRdfZO7cuc7mqpZ0vpPUVxY3xAVwUd3NcQFkZn6o0uqkFnTs2DG++c1vcvLkScaOHcvNN9/c\n7JKkYTlnDyIzx2bmhzLzg5k5rlg+vW44SCWmTZvGvHnzGDduHPPmzWPatGnNLkkaFu9lkBrsySef\nHDCb65NPPtnskqRhGex9EJIG4YorruDHP/7xgNlcx48fz+WXX97kyqShswchNdDbb79NX18f11xz\nDd/5zne45ppr6Ovr4+23ndtSrceAkBro0KFDzJ07l7feeovPfe5zvPXWW8ydO5dDhw41uzRpyBxi\nkhps27ZtTJky5Z07qY8ePcoll1zS7LKkIausBxERl0XEzojYHxH7IuKLRfvFEbE9Il4oXifX7bM2\nIg5ExPMRsaCq2qQq3XLLLedcl1pFlUNMJ4DbM3MW8ClgeUTMAtYAOzJzJrCjWKd4bzFwBf1Tia+P\niLEV1ic13OzZs+nu7h5wFVN3dzezZ89udmnSkFU2xJSZR+ifFpzM/ElEPAdcCiwCOovNNgM1YHXR\n/mBm9gIvRcQB4GrgB1XVKDXanj17mDNnzoCrmGbPns2ePXuaXJk0dCNykjoipgNzgaeA9iI8AF4F\n2ovlS4H6M3mHizappezZs4fMZOfOnWSm4aCWVflJ6oiYBDwM/KvMfDviZ/P/ZWZGxJDmeIqIZcAy\ngPb2dmq1WgOrlRqnp6fHz6daWqUBERHj6Q+HP8rM/1Y0vxYRUzPzSERMBV4v2l+h/0FEp00r2gbI\nzI3ARuh/HoTz7Wu08nkQanVVXsUUwH8EnsvMr9e91Q0sKZaXAI/UtS+OiLaImAHMBJ6uqj5J0rlV\n2YP4NP2PKt0bEc8WbV8G7ga2RMQtwMvAjQCZuS8itgD76b8CanlmnqywPknSOVR5FdOfMfCBQ/Wu\nPcs+64B1VdUkSRo8p9qQJJUyICRJpQwISVIpA0KSVMqAkCSVMiCkBlu5ciUTJkxg3rx5TJgwgZUr\nVza7JGlYfB6E1EArV65kw4YN3HPPPcyaNYv9+/ezevVqAO67774mVycNjT0IqYE2bdrEPffcw6pV\nq5gwYQKrVq3innvuYdOmTc0uTRoyA0JqoN7eXrq6uga0dXV10dvb26SKpOEzIKQGamtrY8OGDQPa\nNmzYQFtbW5MqkobPcxBSAy1dupTbb7+d22+/fUD7ihUrmlSRNHyROaTHMYwqHR0duWvXrmaXIb1j\n0qRJHD9+/Iz2iRMn0tPT04SKpDNFxO7M7DjfdvYgpAY6fvw4kyZNoq+vj97eXtra2hg/frzhoJZk\nQEgN1tPTw9e+9rV3LnN993CT1Co8SS012OTJkwdc5jp58uRmlyQNiwEhNdgbb7zBjBkzOHz4MDNm\nzOCNN95odknSsDjEJDVQW1sbp06d4uDBg9x0000AjB8/njFj/FtMrcdPrdRAS5cupa+vb0BbX18f\nS5cubVJF0vB5mavUQF7mqlYw2Mtc7UFIDXT8+HGmT59OZrJz504yk+nTp5eGhjTaGRBSgz3xxBPn\nXJdahQEhNdhnPvOZc65LrcKAkBpo4sSJHDx4cMBlrgcPHmTixInNLk0aMk9SSw02ZswY6v9fRQSn\nTp1qYkXSQJ6klppgzpw5ZCYLFy5k69atLFy4kMxkzpw5zS5NGjJ7EFIDRQRTpkzh2LFjZCYRwUc+\n8hGOHj1KK/9f0/uLs7lKTXL06NF3ljNzwLrUShxikioQEQNepVZkQEgVOD2c5LCSWpkBIUkqVVlA\nRMQfRsTrEfGXdW0XR8T2iHiheJ1c997aiDgQEc9HxIKq6pJGwq233sqjjz7Krbfe2uxSpGGr7Cqm\niPgnQA/wnzPzF4q23wH+LjPvjog1wOTMXB0Rs4BvA1cDfx94Arg8M0+e63t4FZNGm3Odc3C4SaNF\n0++DyMz/Afzdu5oXAZuL5c3A9XXtD2Zmb2a+BBygPyykljJ27NghtUuj2Uhf5tqemUeK5VeB9mL5\nUuCHddsdLtrOEBHLgGUA7e3t1Gq1aiqVhuGzn/0s3/3ud0vb/ayq1TTtPojMzIgYcp87MzcCG6F/\niKmzs7PRpUnD9vDDD5e2T5s2DT+rajUjfRXTaxExFaB4fb1ofwW4rG67aUWb1FLuv/9+4Mz7IE63\nS61kpAOiG1hSLC8BHqlrXxwRbRExA5gJPD3CtUkN430Qej+obIgpIr4NdAJTIuIwcAdwN7AlIm4B\nXgZuBMjMfRGxBdgPnACWn+8KJklStSoLiMz8/FneuvYs268D1lVVjyRpaLyTWpJUyoCQJJUyICRJ\npQwISVIpA0KSVMqAkCSVMiAkSaUMCKkC48ePH/AqtaKmTdYntZKhPlu6r69vwOtgj+HUHBpN7EFI\ng5CZg/pasWJF6f4rVqwY1P7SaGIPQmqg++67D4BNmzbR29tLW1sbS5cufaddaiWVPXJ0JPjIUY1m\n09f8CQfv/tVmlyGdoemPHJUktTYDQpJUyoCQJJUyICRJpQwISVIpA0KSVMqAkCSVMiAkSaUMCElS\nKafa0AXpyq9s462f9p1/w/do+po/qfT4H75oPD+641cq/R66cBkQuiC99dO+yqfBqNVqdHZ2Vvo9\nqg4gXdgcYpIklTIgJEmlDAhJUikDQpJUypPUuiB98BNrmL15TfXfaHO1h//gJwB85oSqYUDogvST\n5+72KibpPEbdEFNEXBcRz0fEgYgYgT/xJEllRlUPIiLGAt8A5gOHgWciojsz9ze3Mr0fjchf39+v\n/kY5qSqjKiCAq4EDmfnXABHxILAIMCDUUCPxrGifSa1WN9qGmC4FDtWtHy7aJEkjbLT1IM4rIpYB\nywDa29up1WrNLUgXhHnz5g1rv7hnaNvv3LlzWN9HqsJoC4hXgMvq1qcVbe/IzI3ARoCOjo6s+ioR\nCSAzh7zPSFzFJFVptA0xPQPMjIgZEfEBYDHQ3eSaJOmCNKp6EJl5IiJWAI8DY4E/zMx9TS5Lki5I\noyogADLzMeCxZtchSRe60TbEJEkaJQwISVIpA0KSVMqAkCSVMiAkSaViODcAjRYR8bfAy82uQzqL\nKcDRZhchlfi5zLzkfBu1dEBIo1lE7MrMjmbXIQ2XQ0ySpFIGhCSplAEhVWdjswuQ3gvPQUiSStmD\nkCSVGnWT9UmjXUScBPbWNV2fmQebVI5UGYeYpCGKiJ7MnDSM/cZl5okqapKq4BCT1AARMT0i/mdE\n/K/i65qivbNo7wb2F21fiIinI+LZiPiDiBjb1OKlszAgpKG7qPjl/mxEbC3aXgfmZ+Yngc8Bv1+3\n/SeBL2bm5RHxieL9T2fmVcBJ4DdGsnhpsDwHIQ3dT4tf7vXGA/dHxOlf+pfXvfd0Zr5ULF8L/CLw\nTEQAXER/uEijjgEhNcZvAq8BV9LfM/+/de8dr1sOYHNmrh3B2qRhcYhJaowPA0cy8xRwE/3PVC+z\nA/j1iPgoQERcHBE/N0I1SkNiQEiNsR5YEhE/Aj7OwF7DOzJzP/BvgG0RsQfYDkwdsSqlIfAyV0lS\nKXsQkqRSBoQkqZQBIUkqZUBIkkoZEJKkUgaEBETEv46IfRGxp5hC45cacMyFEbGmQfX1NOI40lB4\nmasueBHxy8DXgc7M7I2IKcAHMvNvBrHviMzQOtwZZKX3wh6E1H+j2tHM7AXIzKOZ+TcRcbAICyKi\nIyJqxfKdEfGtiPhz4FsR8cOIuOL0wSKiVmx/c0TcHxEfjoiXI2JM8f7EiDgUEeMj4h9GxPcjYncx\n6+vHi21mRMQPImJvRPz7Ef73kAADQgLYBlwWET+OiPUR8U8Hsc8s4DOZ+XngIeBGgIiYCkzNzF2n\nN8zMt4BngdPH/TXg8czso/+51Ssz8xeBL9F/RzbA7wEPZOZs4Mh7/gmlYTAgdMHLzB76Z1hdBvwt\n8FBE3Hye3boz86fF8hbg14vlG4E/Ltn+Ifqn+QZYXHyPScA1wHci4lngD/jZtBufBr5dLH9rSD+Q\n1CDO5ioBmXkSqAG1iNgLLAFO8LM/oia8a5fjdfu+EhHHImIO/SHQVfItuoHfjoiL6Q+jPwUmAm+W\nTB3+zqGH+eNIDWEPQhe8iPj5iJhZ13QV8DJwkP5f5gA3nOcwDwG/BXw4M/e8+82il/IM/UNH38vM\nk5n5NvBSRPyLoo6IiCuLXf6c/p4G+EAhNYkBIcEkYHNE7C9mWJ0F3Al8Bfi9iNhF/0OAzuWP6f+F\nvuUc2zwEfKF4Pe03gFuKWWD3AYuK9i8Cy4vezKVD+3GkxvAyV0lSKXsQkqRSBoQkqZQBIUkqZUBI\nkkoZEJKkUgaEJKmUASFJKmVASJJK/X9fYd4XUz5NgQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x26edfe0668>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# another way of visualising outliers is using boxplots and whiskers,\n",
    "# which provides the quantiles (box) and inter-quantile range (whiskers),\n",
    "# with the outliers sitting outside the error bars (whiskers).\n",
    "\n",
    "# All the dots in the plot below are outliers according to the quantiles + 1.5 IQR rule\n",
    "\n",
    "fig = data.boxplot(column='Fare')\n",
    "fig.set_title('')\n",
    "fig.set_xlabel('Survived')\n",
    "fig.set_ylabel('Fare')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    891.000000\n",
       "mean      32.204208\n",
       "std       49.693429\n",
       "min        0.000000\n",
       "25%        7.910400\n",
       "50%       14.454200\n",
       "75%       31.000000\n",
       "max      512.329200\n",
       "Name: Fare, dtype: float64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's look at the values of the quantiles so we can\n",
    "# calculate the upper and lower boundaries for the outliers\n",
    "\n",
    "# 25%, 50% and 75% in the output below indicate the\n",
    "# 25th quantile, median and 75th quantile respectively\n",
    "\n",
    "data.Fare.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(65.6344, -26.724, 23.0896)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's calculate the upper and lower boundaries\n",
    "# to identify outliers according\n",
    "# to interquantile proximity rule\n",
    "\n",
    "IQR = data.Fare.quantile(0.75) - data.Fare.quantile(0.25)\n",
    "\n",
    "Lower_fence = data.Fare.quantile(0.25) - (IQR * 1.5)\n",
    "Upper_fence = data.Fare.quantile(0.75) + (IQR * 1.5)\n",
    "\n",
    "Upper_fence, Lower_fence, IQR"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(100.2688, -61.358399999999996, 23.0896)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# And if we are looking at really extreme values\n",
    "# using the interquantile proximity rule\n",
    "\n",
    "IQR = data.Fare.quantile(0.75) - data.Fare.quantile(0.25)\n",
    "\n",
    "Lower_fence = data.Fare.quantile(0.25) - (IQR * 3)\n",
    "Upper_fence = data.Fare.quantile(0.75) + (IQR * 3)\n",
    "\n",
    "Upper_fence, Lower_fence, IQR"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The upper boundary for extreme outliers is a cost of 100 dollars for the Fare. The lower boundary is meaningless because there can't be a negative price for Fare."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total passengers: 891\n",
      "passengers that paid more than 100: 116\n",
      "passengers that paid more than 200: 53\n"
     ]
    }
   ],
   "source": [
    "# lets look at the actual number of passengers on the upper Fare ranges\n",
    "\n",
    "print('total passengers: {}'.format(data.shape[0]))\n",
    "\n",
    "print('passengers that paid more than 65: {}'.format(\n",
    "    data[data.Fare > 65].shape[0]))\n",
    "\n",
    "print('passengers that paid more than 100: {}'.format(\n",
    "    data[data.Fare > 100].shape[0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total passengers: 1.0\n",
      "passengers that paid more than 100: 0.13019079685746351\n",
      "passengers that paid more than 200: 0.05948372615039282\n"
     ]
    }
   ],
   "source": [
    "# and percentages of passengers\n",
    "total_passengers = np.float(data.shape[0])\n",
    "\n",
    "print('total passengers: {}'.format(data.shape[0] / total_passengers))\n",
    "\n",
    "print('passengers that paid more than 65: {}'.format(\n",
    "    data[data.Fare > 65].shape[0] / total_passengers))\n",
    "\n",
    "print('passengers that paid more than 100: {}'.format(\n",
    "    data[data.Fare > 100].shape[0] / total_passengers))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When using the 3  times interquantile range itnerval to find outliers, we find that 6% of the passengers have paid extremely high fares.\n",
    "We can go ahead and investigate the nature of this outliers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Ticket\n",
       "113503      1\n",
       "113760      4\n",
       "113781      4\n",
       "16966       2\n",
       "17421       4\n",
       "19950       4\n",
       "24160       3\n",
       "35273       3\n",
       "36928       2\n",
       "PC 17483    1\n",
       "PC 17558    2\n",
       "PC 17569    2\n",
       "PC 17582    3\n",
       "PC 17608    2\n",
       "PC 17611    2\n",
       "PC 17755    3\n",
       "PC 17757    4\n",
       "PC 17758    2\n",
       "PC 17760    3\n",
       "PC 17761    2\n",
       "Name: Fare, dtype: int64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's create a separate dataframe for high fare payers\n",
    "high_fare_df = data[data.Fare>100]\n",
    "\n",
    "# ticket: it indicates the people that bought their fares together\n",
    "high_fare_df.groupby('Ticket')['Fare'].count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A group of people who bought their tickets together, say they were a family, would have the same ticket number. And the fare attached to them is no longer the individual Fare, rather the group Fare. This is why, we see this unusually high values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ticket</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>113503</th>\n",
       "      <td>1</td>\n",
       "      <td>211.5000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>113760</th>\n",
       "      <td>4</td>\n",
       "      <td>120.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>113781</th>\n",
       "      <td>4</td>\n",
       "      <td>151.5500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16966</th>\n",
       "      <td>2</td>\n",
       "      <td>134.5000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17421</th>\n",
       "      <td>4</td>\n",
       "      <td>110.8833</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19950</th>\n",
       "      <td>4</td>\n",
       "      <td>263.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24160</th>\n",
       "      <td>3</td>\n",
       "      <td>211.3375</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35273</th>\n",
       "      <td>3</td>\n",
       "      <td>113.2750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36928</th>\n",
       "      <td>2</td>\n",
       "      <td>164.8667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PC 17483</th>\n",
       "      <td>1</td>\n",
       "      <td>221.7792</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          Ticket      Fare\n",
       "Ticket                    \n",
       "113503         1  211.5000\n",
       "113760         4  120.0000\n",
       "113781         4  151.5500\n",
       "16966          2  134.5000\n",
       "17421          4  110.8833\n",
       "19950          4  263.0000\n",
       "24160          3  211.3375\n",
       "35273          3  113.2750\n",
       "36928          2  164.8667\n",
       "PC 17483       1  221.7792"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "multiple_tickets = pd.concat(\n",
    "    [\n",
    "        high_fare_df.groupby('Ticket')['Fare'].count(),\n",
    "        high_fare_df.groupby('Ticket')['Fare'].mean()\n",
    "    ],\n",
    "    axis=1)\n",
    "\n",
    "multiple_tickets.columns = ['Ticket', 'Fare']\n",
    "multiple_tickets.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Therefore, the fare should be divided by the number of tickets bought together to find out the individual price. So we see how finding out and investigating the presence of outliers, can lead us to new insight about the dataset at hand. \n",
    "\n",
    "Go ahead and divide the Fare by the number of tickets bought together, and then repeat the finding outliers exercise on this newly created variable. Do you know how to do this in python?\n",
    "\n",
    "If not, don't worry, I will show you how to calculate individual ticket price in the final lecture of this course in the section \"Putting it all together\".\n",
    "\n",
    "For now, let's just go ahead and visualise a group of people that were seemingly travelling together and therefore bought the tickets together:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>258</th>\n",
       "      <td>259</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Ward, Miss. Anna</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17755</td>\n",
       "      <td>512.3292</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>679</th>\n",
       "      <td>680</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cardeza, Mr. Thomas Drake Martinez</td>\n",
       "      <td>male</td>\n",
       "      <td>36.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>PC 17755</td>\n",
       "      <td>512.3292</td>\n",
       "      <td>B51 B53 B55</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>737</th>\n",
       "      <td>738</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Lesurer, Mr. Gustave J</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17755</td>\n",
       "      <td>512.3292</td>\n",
       "      <td>B101</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Survived  Pclass                                Name  \\\n",
       "258          259         1       1                    Ward, Miss. Anna   \n",
       "679          680         1       1  Cardeza, Mr. Thomas Drake Martinez   \n",
       "737          738         1       1              Lesurer, Mr. Gustave J   \n",
       "\n",
       "        Sex   Age  SibSp  Parch    Ticket      Fare        Cabin Embarked  \n",
       "258  female  35.0      0      0  PC 17755  512.3292          NaN        C  \n",
       "679    male  36.0      0      1  PC 17755  512.3292  B51 B53 B55        C  \n",
       "737    male  35.0      0      0  PC 17755  512.3292         B101        C  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's have a look at the most extreme outliers\n",
    "data[data.Fare>300]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These three people have the same ticket number, indicating that they were travelling together. The Fare price in this case, 512 is the price of 3 tickets, and not one. This is why, it is unusually high."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Age"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x26ee407c88>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHTNJREFUeJzt3X+UHGWd7/H3hwACGSQEcAwBCSqCaADNqKCszoA/oiBw\nXOTABQ1e1sg96rK7ubsbVGQ9rALei1f3Kuuy/iBHhCwiCgYXjDGD695dIRE04ZcoBDBCAkhCJiCS\n8L1/1DOkd5zprp7pqupMfV7nzJmu6vrx6Z6e+U49T9VTigjMzKy+dqg6gJmZVcuFwMys5lwIzMxq\nzoXAzKzmXAjMzGrOhcDMrOZcCMxGkPQxSV/p4PaGJL00Pb5c0t93cNtflnRep7Zn9eRCYF1D0qCk\nJyS9oOB9/F7SJklPSlopaWHjPiPiMxHxZzm31XK5iOiJiPs6kP1MST8Zse2zI+KCiW7b6s2FwLqC\npFnAnwABnFDw7j4SEbsDM4AFwKnA9yWpkzuRtGMnt2dWFBcC6xbvB/4TuByY1/iEpL0kfS/9B3+r\npL9v/M9Y0iGSlkr6naR7JJ2SZ4cRsTkiBskKz1HAcWl7fyfpivR4F0lXSHpc0oa0/15JnyYrXF9M\nTT9fTMuHpA9Luhe4t2Heyxt2vXfKu0nSzZIOSMvNSss+X0CGjzokvRL4MnBU2t+G9Px/aWqS9EFJ\nv0rvxfWS9m14LiSdLene9Fq+1OniZ9snFwLrFu8Hvpm+3iGpt+G5LwGbgReTFYnnC4WkqcBS4Erg\nRWT/3V8q6dC8O46IB4EVZH/YR5oH7AHsD+wFnA08HREfB/6N7OiiJyI+0rDOScAbgLEynA5cAOwN\n3J5ec6uMd6V9/0fa37SRy0g6BrgQOIXsaOcBYPGIxY4HXgcclpZ7R6t92+TnQmCVk3Q0cABwdUSs\nBH4N/Lf03BTgT4HzI+KpiLgTWNSw+vHAmoj4ekRsiYjbgG8D720zxm+B6aPMf5asALw8IrZGxMqI\neLLFti6MiN9FxNNjPH9DRPw4Ip4BPk72X/7+beYdzenA1yLiZ2nb56Ztz2pY5qKI2JCK33LgiA7s\n17ZzLgTWDeYBP4iIx9L0lWz7r38fYEfgoYblGx8fALwhNXVsSE0mp5MdPbRjJvC7UeZ/A7gJWCzp\nt5I+K2mnFtt6KO/zETGU9rvv2Ivnti/ZUUDjth8ne23DHml4/BTQ04H92nbOnVlWKUm7kjVRTJE0\n/EfqBcA0SYcDq4EtwH7AL9Pzjf89PwTcHBFvm0CG/YE5wMUjn4uIZ4FPAZ9K/1l/H7gH+CpZx/Zo\nWg3p+3x+ST1kRyK/BX6fZu8GDB91NBa0Vtv9LVlhHN72VLKjmbUt1rOa8xGBVe0kYCtZe/oR6euV\nZO3v74+IrcC1wN9J2k3SIWT9CcOWAK+Q9D5JO6Wv16XO1abS9t4CXAfcQvZHfuQyA5JmpyaqJ8ma\nip5LT68DXjqO1/wuSUdL2pmsr+A/I+KhiHiU7I/2GZKmSPrvwMsa1lsH7JfWG81VwAckHZFOh/0M\n8NOIWDOOjFYjLgRWtXnA1yPiwYh4ZPgL+CJwejqD5iNkHbaPkDXVXAU8AxARm4C3k3US/zYtczHZ\nUcVYvihpE9kf1s+T9SnMjYjnRln2xcA1ZEXgLuDmlAHgC8DJ6dqHf2jjNV8JnE/WJDQHOKPhuQ8C\nf03WpPMq4P81PPcj4A7gEUmPMUJE/BA4L72eh8mKyKlt5LKakm9MY9sbSRcDL46IeS0XNrOWfERg\nXS9dJ3CYMq8HzgK+U3Uus8nCncW2PdidrDloX7LmnEvI2vXNrAPcNGRmVnNuGjIzq7ntomlo7733\njlmzZrW93ubNm5k6dWrnA02Qc7WnW3NB92ZzrvZ0ay6YWLaVK1c+FhH7tFwwIrr+a86cOTEey5cv\nH9d6RXOu9nRrrojuzeZc7enWXBETywasiBx/Y900ZGZWcy4EZmY150JgZlZzLgRmZjXnQmBmVnMu\nBGZmNedCYGZWcy4EZmY150JgZlZz28UQE9YZsxbeMOr8NRcdV3ISM+smhRYCSWuATWS3ItwSEX2S\npgP/AswC1gCnRMQTReYwM7OxldE0NBARR0REX5peCCyLiIOAZWnazMwqUkUfwYnAovR4EdnNy83M\nrCKF3phG0v3ARrKmoX+KiMskbYiIael5AU8MT49Ydz4wH6C3t3fO4sWL297/0NAQPT09E3kJhagq\n16q1G0edP3vmHoDfr/Ho1mzO1Z5uzQUTyzYwMLCyoTVmTEV3Fh8dEWslvQhYKunuxicjIiSNWoki\n4jLgMoC+vr7o7+9ve+eDg4OMZ72iVZXrzLE6i0/vB/x+jUe3ZnOu9nRrLignW6FNQxGxNn1fT3az\n8dcD6yTNAEjf1xeZwczMmiusEEiaKmn34cfA24HVwPXAvLTYPHwTcjOzShXZNNQLfCfrBmBH4MqI\nuFHSrcDVks4CHgBOKTCDmZm1UFghiIj7gMNHmf84cGxR+zUzs/Z4iAkzs5pzITAzqzkXAjOzmnMh\nMDOrORcCM7OacyEwM6s534/AKuV7JJhVz0cEZmY150JgZlZzLgRmZjXnPgJrm9v1zSYXHxGYmdWc\nC4GZWc25EJiZ1ZwLgZlZzbkQmJnVnAuBmVnNuRCYmdWcC4GZWc25EJiZ1ZwLgZlZzbkQmJnVnAuB\nmVnNuRCYmdWcC4GZWc25EJiZ1ZwLgZlZzbkQmJnVnAuBmVnNuRCYmdWcC4GZWc0VXggkTZF0m6Ql\naXq6pKWS7k3f9yw6g5mZja2MI4JzgLsaphcCyyLiIGBZmjYzs4q0LASSpkraIT1+haQTJO2UZ+OS\n9gOOA77SMPtEYFF6vAg4qb3IZmbWSYqI5gtIK4E/AfYE/h24FfhDRJzecuPSNcCFwO7A/4yI4yVt\niIhp6XkBTwxPj1h3PjAfoLe3d87ixYvbemEAQ0ND9PT0tL1e0arKtWrtxqbP9+4K657eNj175h5t\nbWes5ceTqXFb3fpzhO7N5lzt6dZcMLFsAwMDKyOir9VyO+bYliLiKUlnAZdGxGcl3d5yJel4YH1E\nrJTUP9oyERGSRq1EEXEZcBlAX19f9PePuommBgcHGc96Rasq15kLb2j6/ILZW7hk1baPxJrT+9va\nzljLjydT47a69ecI3ZvNudrTrbmgnGy5CoGko4DTgbPSvCk51nsTcIKkdwG7AC+UdAWwTtKMiHhY\n0gxg/XiCm5lZZ+TpLD4HOBf4TkTcIemlwPJWK0XEuRGxX0TMAk4FfhQRZwDXA/PSYvOA68aV3MzM\nOqLpEYGkKcAJEXHC8LyIuA/48wns8yLg6tTU9ABwygS2ZWZmE9S0EETEVklHT3QnETEIDKbHjwPH\nTnSbZmbWGXn6CG6TdD3wLWDz8MyIuLawVGZmVpo8hWAX4HHgmIZ5AbgQmJlNAi0LQUR8oIwgZmZW\njTxXFr9C0jJJq9P0YZI+UXw0MzMrQ57TR/+Z7PTRZwEi4hdkp4OamdkkkKcQ7BYRt4yYt6WIMGZm\nVr48heAxSS8j6yBG0snAw4WmMjOz0uQ5a+jDZGP+HCJpLXA/cEahqczMrDR5zhq6D3irpKnADhGx\nqfhYZmZWlpaFQNJfjZgG2AisjIiWo5CamVl3y9NH0AecDcxMXx8C5gL/LOlvCsxmZmYlyNNHsB/w\n2ogYApB0PnAD8GZgJfDZ4uLZZDGrxb0QzKw6eY4IXgQ80zD9LNAbEU+PmG9mZtuhPEcE3wR+Kmn4\nvgHvBq5Mncd3FpbMzMxKkeesoQsk3Qi8Mc06OyJWpMct71tsZmbdLc8RAcDPgLXDy0t6SUQ8WFgq\nszGM1dew5qLjSk5iNnnkOX30o8D5wDpgKyCyq4wPKzaamZmVIc8RwTnAwenOYmZmNsnkOWvoIbIL\nyMzMbBLKc0RwHzAo6QYaTheNiM8VlsqsQ9ynYNZankLwYPraOX2Zmdkkkuf00U8BSNotIp4qPpKZ\nmZUpz60qj5J0J3B3mj5c0qWFJzMzs1Lk6Sz+PPAO4HGAiPg52ThDZmY2CeQpBETEQyNmbS0gi5mZ\nVSBPZ/FDkt4IhKSdyK4ruKvYWGZmVpY8RwRnk92ucibZMBNHpGkzM5sE8pw19BgeXM7MbNLKc9bQ\nZyW9UNJOkpZJelSSb15vZjZJ5GkaentEPAkcD6wBXg78dZGhzMysPHkKwXDz0XHAtyLC4w6ZmU0i\nec4aWiLpbuBp4H9I2gf4fbGxzMysLC2PCCJiIdndyfoi4llgM3Biq/Uk7SLpFkk/l3SHpOGhKqZL\nWirp3vR9z4m+CDMzG788ncXvBZ6NiK2SPgFcAeybY9vPAMdExOFkp5zOlXQksBBYFhEHAcvStJmZ\nVSRPH8F5EbFJ0tHAW4GvAv/YaqXIDKXJndJXkB1NLErzFwEntZ3azMw6RhHRfAHptoh4jaQLgVUR\nceXwvJYbl6YAK8nONPpSRPytpA0RMS09L+CJ4ekR684H5gP09vbOWbx4cdsvbmhoiJ6enrbXK1pV\nuVatbd7P37srrHt62/TsmXu0tZ2xls+z72bbany/2t33eLK2w5+x9jhX+yaSbWBgYGVE9LVaLk8h\nWEJ2RfHbgNeSdRrfkpp8cpE0DfgO8FHgJ41/+CU9ERFN+wn6+vpixYoVeXf3vMHBQfr7+9ter2hV\n5RrrJi3DFszewiWrtp0/MNbNW8Zzs5dW+262rcb3q919F31jGn/G2uNc7ZtINkm5CkGepqFTgJuA\nd0TEBmA6bV5HkNZbDswF1kmakULOANa3sy0zM+usPGcNPRUR1wIbJb2ErK3/7lbrSdonHQkgaVey\nI4q7geuBeWmxecB148xuZmYd0PI6AkknAJeQnSm0HngJ2R/0V7VYdQawKPUT7ABcHRFLJP0HcLWk\ns4AHyI44zErlexmbbZPngrILgCOBH6ZO4wGg5VhDEfEL4I86lCPiceDYdoOamVkx8vQRPJv+eO8g\naYeIWA607HwwM7PtQ54jgg2SeoAfA9+UtJ7s6mIzM5sE8hwRnAg8BfwlcCPwa+DdRYYyM7PyND0i\nkHQS2cVgqyLiJrZdEWxmZpPEmEcEki4lOwrYC7hA0nmlpTIzs9I0OyJ4M3B4GmxuN+DfyM4gMjOz\nSaRZIfhDRGyF7KKyNC6Q1Ui7w0IUte8Fs7dwZoVZYOz34vK5U0tOYtZ5zQrBIZJ+kR4LeFmaFtng\noocVns7MzArXrBC8srQUZmZWmTELQUQ8UGYQMzOrRp7rCMzMbBJzITAzq7lm1xEsS98vLi+OmZmV\nrVln8QxJbwROkLSY7Gyh50XEzwpNZmZmpWhWCD4JnAfsB3xuxHMBHFNUKDMzK0+zs4auAa6RdF5E\n+IpiM7NJquUw1BFxQbpL2ZvTrMGIWFJsLDMzK0vLs4YkXQicA9yZvs6R9Jmig5mZWTny3JjmOOCI\niHgOQNIi4DbgY0UG6xTfm7Y8VY5NZGbjl/c6gmkNj/coIoiZmVUjzxHBhcBtkpaTnUL6ZmBhoanM\nzKw0eTqLr5I0CLwuzfrbiHik0FRmZlaaPEcERMTDwPUFZzEzswp4rCEzs5pzITAzq7mmhUDSFEl3\nlxXGzMzK17SPIN24/h5JL4mIB8sKZfn4Ggkz64Q8ncV7AndIugXYPDwzIk4oLJWZmZUmTyE4r/AU\nZmZWmTzXEdws6QDgoIj4oaTdgCnFRzMzszLkGXTug8A1wD+lWTOB7xYZyszMypPn9NEPA28CngSI\niHuBF7VaSdL+kpZLulPSHZLOSfOnS1oq6d70fc+JvAAzM5uYPIXgmYj4w/CEpB3J7lDWyhZgQUQc\nChwJfFjSoWTjFC2LiIOAZXjcIjOzSuUpBDdL+hiwq6S3Ad8CvtdqpYh4ePi+xhGxCbiLrFnpRGBR\nWmwRcNJ4gpuZWWfkKQQLgUeBVcCHgO8Dn2hnJ5JmAa8Bfgr0prGLAB4BetvZlpmZdZYiWrfySNoZ\nOISsSeiexqaiHOv2ADcDn46IayVtiIhpDc8/ERF/1E8gaT4wH6C3t3fO4sWL8+7yeUNDQ9y/ceuo\nz82eWd1tFYaGhujp6Znwdlat3Tjq/LFe21jLD+vdFdY9PeFYHZcn13hf80S3c+AeUzrys+y0Tn3G\nOs252jeRbAMDAysjoq/Vci0LgaTjgC8Dvya7H8GBwIci4l9bblzaCVgC3BQRn0vz7gH6I+JhSTPI\n7oF8cLPt9PX1xYoVK1rt7o8MDg5y5o2bR32uyqtvBwcH6e/vn/B22r2yuNUdxBbM3sIlq3INSFuq\nPLnG+5onup3L507tyM+y0zr1Ges052rfRLJJylUI8jQNXQIMRER/RLwFGAD+T44AAr4K3DVcBJLr\ngXnp8TzguhwZzMysIHn+/dsUEb9qmL4P2JRjvTcB7wNWSbo9zfsYcBFwtaSzgAeAU9rIa2ZmHTZm\nIZD0nvRwhaTvA1eT9RG8F7i11YYj4idkTUmjObbNnGZmVpBmRwTvbni8DnhLevwosGthiczMrFRj\nFoKI+ECZQczMrBot+wgkHQh8FJjVuLyHoTYzmxzydBZ/l+zsn+8BzxUbx2xyaHbaarunLvsGRFa0\nPIXg9xHxD4UnMTOzSuQpBF+QdD7wA+CZ4ZnD4wiZmdn2LU8hmE12PcAxbGsaijRtZmbbuTyF4L3A\nS9sZX8isbO0OJdGNOvUaZi28gQWzt3DmiO25T8HGkmeIidXAtJZLmZnZdinPEcE04G5Jt/Jf+wh8\n+qiZ2SSQpxCcX3gKMzOrTMtCEBE3lxHEzMyqkefK4k1su0fxzsBOwOaIeGGRwczMrBx5jgh2H36c\n7jFwItnN6M3MbBLIc9bQ8yLzXeAdBeUxM7OS5Wkaek/D5A5AH/D7whKZmVmp8pw11Hhfgi3AGrLm\nITMzmwTy9BH4vgRmZpNYs1tVfrLJehERFxSQx8zMStbsiGDzKPOmAmcBewGTshB47Hczq5tmt6q8\nZPixpN2Bc4APAIuBS8Zaz8zMti9N+wgkTQf+CjgdWAS8NiKeKCOYmZmVo1kfwf8C3gNcBsyOiKHS\nUpmZWWmaHREsIBtt9BPAx7OLigEQWWexh5joUpNhbH4zK0+zPoK2rjo2M7Ptk//Ym5nVnAuBmVnN\nuRCYmdWcC4GZWc25EJiZ1ZwLgZlZzeUZhtqsNnwNhtVRYUcEkr4mab2k1Q3zpktaKune9H3PovZv\nZmb5FNk0dDkwd8S8hcCyiDgIWJamzcysQoUVgoj4MfC7EbNPJBu8jvT9pKL2b2Zm+Sgiitu4NAtY\nEhGvTtMbImJaeizgieHpUdadD8wH6O3tnbN48eK29z80NMT9G7eO+tzsmXuMOn/V2o1tLT8eQ0ND\n9PT0THg7Y2Udr95dYd3THd1kR3RrLoAD95gy6s+y2c+m3c/eeLYz2nvWyc/weHXqs99p3ZoLJpZt\nYGBgZUT0tVquskKQpp+IiJb9BH19fbFixYq29z84OMiZN452f52xbzRTxo1pBgcH6e/vn/B2Ot2x\nuWD2Fi5Z1X3nD3RrLoDL504d9WfZ7GfT7mdvPNsZ7T3rhpsrdeqz32ndmgsmlk1SrkJQ9umj6yTN\nAEjf15e8fzMzG6HsQnA9MC89ngdcV/L+zcxshMKOtyVdBfQDe0v6DXA+cBFwtaSzgAeAU4rav1m3\n6rZrFXyfbiusEETEaWM8dWxR+zQzs/Z5iAkzs5pzITAzq7nuPCdvkhir7fXyuVNLTmJFWbV2I2d2\nWZu/Wbt8RGBmVnMuBGZmNedCYGZWc+4j6ICizwvvtvPOrR7avb6gU8s3W8eK4SMCM7OacyEwM6s5\nFwIzs5qrbR9Ble3uY5177nZRa0e7n2H3NdlYfERgZlZzLgRmZjXnQmBmVnMuBGZmNedCYGZWcy4E\nZmY150JgZlZzLgRmZjVX2wvKzKwcjReyLZi9JdeNfNodwM4mxkcEZmY150JgZlZzLgRmZjXnPoKc\nyhiwy4OCmTVXdN9BXfsmfERgZlZzLgRmZjXnQmBmVnPuIzCztkyGvqyRryHv9Q2TlY8IzMxqzoXA\nzKzmXAjMzGrOfQRmtt2r6vz/Tu23Wb/L5XOntrWt8ajkiEDSXEn3SPqVpIVVZDAzs0zphUDSFOBL\nwDuBQ4HTJB1adg4zM8tUcUTweuBXEXFfRPwBWAycWEEOMzMDFBHl7lA6GZgbEX+Wpt8HvCEiPjJi\nufnA/DR5MHDPOHa3N/DYBOIWxbna0625oHuzOVd7ujUXTCzbARGxT6uFurazOCIuAy6byDYkrYiI\nvg5F6hjnak+35oLuzeZc7enWXFBOtiqahtYC+zdM75fmmZlZBaooBLcCB0k6UNLOwKnA9RXkMDMz\nKmgaiogtkj4C3ARMAb4WEXcUtLsJNS0VyLna0625oHuzOVd7ujUXlJCt9M5iMzPrLh5iwsys5lwI\nzMxqblIWgm4awkLS1yStl7S6Yd50SUsl3Zu+71lypv0lLZd0p6Q7JJ3TDblShl0k3SLp5ynbp7ol\nW8oxRdJtkpZ0Sy5JayStknS7pBVdlGuapGsk3S3pLklHdUmug9N7Nfz1pKS/6JJsf5k+96slXZV+\nHwrPNekKQRcOYXE5MHfEvIXAsog4CFiWpsu0BVgQEYcCRwIfTu9R1bkAngGOiYjDgSOAuZKO7JJs\nAOcAdzVMd0uugYg4ouF8827I9QXgxog4BDic7H2rPFdE3JPeqyOAOcBTwHeqziZpJvDnQF9EvJrs\nZJpTS8kVEZPqCzgKuKlh+lzg3IozzQJWN0zfA8xIj2cA91Sc7zrgbV2YazfgZ8AbuiEb2TUvy4Bj\ngCXd8rME1gB7j5hXaS5gD+B+0gkp3ZJrlJxvB/69G7IBM4GHgOlkZ3QuSfkKzzXpjgjY9mYO+02a\n1016I+Lh9PgRoLeqIJJmAa8BfkqX5ErNL7cD64GlEdEt2T4P/A3wXMO8bsgVwA8lrUxDs3RDrgOB\nR4Gvp6a0r0ia2gW5RjoVuCo9rjRbRKwF/jfwIPAwsDEiflBGrslYCLYrkZX5Ss7hldQDfBv4i4h4\nsltyRcTWyA7b9wNeL+nVVWeTdDywPiJWjrVMhe/Z0en9eidZM9+buyDXjsBrgX+MiNcAmxnRpFHl\nZwwgXdB6AvCtkc9V9Bnbk2wAzgOBfYGpks4oI9dkLATbwxAW6yTNAEjf15cdQNJOZEXgmxFxbbfk\nahQRG4DlZH0sVWd7E3CCpDVkI+YeI+mKLsg1/J8kEbGerK379V2Q6zfAb9LRHMA1ZIWh6lyN3gn8\nLCLWpemqs70VuD8iHo2IZ4FrgTeWkWsyFoLtYQiL64F56fE8sjb60kgS8FXgroj4XLfkStn2kTQt\nPd6VrO/i7qqzRcS5EbFfRMwi+0z9KCLOqDqXpKmSdh9+TNamvLrqXBHxCPCQpIPTrGOBO6vONcJp\nbGsWguqzPQgcKWm39Dt6LFkHe/G5quqkKbjT5V3AL4FfAx+vOMtVZO19z5L9l3QWsBdZp+O9wA+B\n6SVnOprs8PIXwO3p611V50rZDgNuS9lWA59M8yvP1pCxn22dxVX/LF8K/Dx93TH8ea86V8pwBLAi\n/Sy/C+zZDblStqnA48AeDfMqzwZ8iuwfn9XAN4AXlJHLQ0yYmdXcZGwaMjOzNrgQmJnVnAuBmVnN\nuRCYmdWcC4GZWc25EJi1IOkkSSHpkKqzmBXBhcCstdOAn6TvZpOOC4FZE2k8pqPJLgQ8Nc3bQdKl\naZz9pZK+L+nk9NwcSTenAeBuGh4awKybuRCYNXci2Zj6vwQelzQHeA/Z0OKHAu8jG/p8ePym/wuc\nHBFzgK8Bn64itFk7dqw6gFmXO43sBiuQDTZ3Gtnvzbci4jngEUnL0/MHA68GlmZDxTCFbHgRs67m\nQmA2BknTyW5CM1tSkP1hD7IRPkddBbgjIo4qKaJZR7hpyGxsJwPfiIgDImJWROxPdtet3wF/mvoK\neskGoYPsTlL7SHq+qUjSq6oIbtYOFwKzsZ3GH//3/23gxWQjyd4JXEF2O82NEfEHsuJxsaSfk43q\n+sby4pqNj0cfNRsHST0RMSRpL+AW4E2RjcFvtt1xH4HZ+CxJN9DZGbjARcC2Zz4iMDOrOfcRmJnV\nnAuBmVnNuRCYmdWcC4GZWc25EJiZ1dz/B+6PF17AuXb6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x26ee368b38>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# First let's plot the histogram to get an idea of the distribution\n",
    "\n",
    "fig = data.Age.hist(bins=50)\n",
    "fig.set_title('Age Distribution')\n",
    "fig.set_xlabel('Age')\n",
    "fig.set_ylabel('Number of Passengers')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Although the distribution of Age does not look strictly normal, we could assume normality and use the Gaussian approach to find outliers. See below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x26ee5ba588>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFQhJREFUeJzt3X+wX3V95/HnixAnlHSFFPxuVlnDtBRDcXGXOxWV2d4Q\nRanWMFYomdpJd7KTZmDvatmdmi5/WHc3Tjqzwyzjls7Gxm3Wtinx14JBQUjzbYvjWoOgglfFVVLF\nBFTEmoyj3OS9f9wTGiHJ/cE93++9Oc/HzJ3vOed7zve8L3O4r3w+53s+n1QVkqTuOm3YBUiShssg\nkKSOMwgkqeMMAknqOINAkjrOIJCkjjMIJKnjDAJJ6jiDQJI67vRhFzAd55xzTq1YsWLYZUjPcejQ\nIc4888xhlyEd1/333//dqjp3qv0WRBCsWLGCvXv3DrsM6Tn6/T6jo6PDLkM6riT7prOfXUOS1HEG\ngSR1nEEgSR1nEEhSxxkEktRxrQZBkt9N8nCSh5LsSLIkybIk9yR5pHk9u80apDbs2LGDiy++mNWr\nV3PxxRezY8eOYZckzVprXx9N8mLg3wMXVdWPkuwErgMuAnZX1ZYkm4BNwDvbqkOaazt27OCmm25i\n27ZtHD58mEWLFrF+/XoA1q5dO+TqpJlru2vodOCMJKcDPwN8G1gDbG/e3w5c3XIN0pzavHkz27Zt\nY9WqVZx++umsWrWKbdu2sXnz5mGXJs1Kay2CqnosyX8D/h74EfDJqvpkkl5V7W92OwD0jnd8kg3A\nBoBer0e/32+rVGlGxsfHOXz4MP1+n4MHD9Lv9zl8+DDj4+Nep1qQ2uwaOpvJf/2fDzwFfDDJ247d\np6oqSR3v+KraCmwFGBkZKZ/e1HyxcuVKFi1axOjo6DNPFu/Zs4eVK1f6lLEWpDa7hl4LfKOqvlNV\nTwMfAV4NPJ5kOUDz+kSLNUhz7qabbmL9+vXs2bOHiYkJ9uzZw/r167npppuGXZo0K22ONfT3wGVJ\nfobJrqHVwF7gELAO2NK83t5iDdKcO3pDeGxsjPHxcVauXMnmzZu9UawFK1XH7ZmZmw9P3g38BjAB\nPAD8W2ApsBP458A+4NqqevJknzMyMlIOOqf5yEHnNJ8lub+qRqbar9XRR6vqXcC7nrX5x0y2DiRJ\n84BPFktSxxkEktRxBoEkdZxBIEkdZxBIUscZBJLUcQaBNAsOQ61TSavPEUinIoeh1qnGFoE0Qw5D\nrVONQSDN0Pj4OJdffvlPbbv88ssZHx8fUkXS82MQSDO0cuVK7rvvvp/adt9997Fy5cohVSQ9PwaB\nNEMOQ61TjTeLpRlyGGqdalodhnquOAy15iuHodZ8Nt1hqO0akmZhbGyMJUuWsGrVKpYsWcLY2Niw\nS5JmrbUgSHJhkgeP+fmHJO9IsizJPUkeaV7PbqsGqQ1jY2PceuutnHXWWSThrLPO4tZbbzUMtGAN\npGsoySLgMeCVwA3Ak1W1Jckm4OyqeufJjrdrSPPJ4sWLWbRoEUeOHOHpp59m8eLFnHbaaRw+fJin\nn3562OVJz5hvXUOrgf9XVfuANcD2Zvt24OoB1SDNiYmJCSYmJtiyZQuf+MQn2LJlyzPbpIVoUEFw\nHXB0MJZeVe1vlg8AvQHVIM2Zq666ihtvvJElS5Zw4403ctVVVw27JGnWWv/6aJIXAG8Gfv/Z71VV\nJTlu31SSDcAGgF6vR7/fb7NMaUZ27drF9ddfzxVXXMH111/Prl27ALxOtSC1fo8gyRrghqq6sln/\nCjBaVfuTLAf6VXXhyT7DewSaT47eE6iqZ+4RJHnmnoE0X8ynewRr+cduIYA7gHXN8jrg9gHUIM2Z\njRs3MjExwbJlywBYtmwZExMTbNy4cciVSbPTatdQkjOB1wG/c8zmLcDOJOuBfcC1bdYgzbX3vve9\nALzvfe8D4KmnnuL6669/Zru00PhksfQ8+GSx5rP51DUkSZrHDAJJ6jiDQJoF5yzWqcRhqKUZcs5i\nnWpsEUgz5JzFOtXYIpBmaHx8nPe85z2sXr2aqiIJq1evds5iLVi2CKQZOuOMM7j33nvZuHEjH/vY\nx9i4cSP33nsvZ5xxxrBLk2bFIJBm6NChQyxdupRrrrmGJUuWcM0117B06VIOHTo07NKkWTEIpFm4\n+eabGRsb4/Wvfz1jY2PcfPPNwy5JmjWDQJqhJDzwwAM89NBD7N69m4ceeogHHniAJMMuTZoVh5iQ\njjGoP+YL4f87LXwOMSHNQlVN6+fKK698JjSScOWVV077WENA841BIM3C3XffzZEjR3jpO3dx5MgR\n7r777mGXJM2aQSBJHWcQSFLHGQSS1HGtBkGSs5J8KMmXk4wneVWSZUnuSfJI83p2mzVIkk6u7RbB\nLcBdVfUy4BJgHNgE7K6qC4DdzbokaUhaC4IkLwT+NbANoKp+UlVPAWuA7c1u24Gr26pBkjS1NlsE\n5wPfAf5XkgeS/EkzmX2vqvY3+xwAei3WIEmaQpvDUJ8O/CtgrKo+k+QWntUNVFWV5LhP1yTZAGwA\n6PV69Pv9FkuVZs9rUwtdm0HwLeBbVfWZZv1DTAbB40mWV9X+JMuBJ453cFVtBbbC5BATo6OjLZYq\nzdJdd+K1qYWuta6hqjoAfDPJhc2m1cCXgDuAdc22dcDtbdUgSZpa2zOUjQF/nuQFwNeBf8Nk+OxM\nsh7YB1zbcg2SpJNoNQiq6kHgeCPfrW7zvJKk6fPJYknqOINAkjrOIJCkjjMIJKnjDAJJ6jiDQJI6\nziCQpI4zCCSp4wwCSeo4g0CSOs4gkKSOMwgkqeMMAknqOINAkjrOIJCkjjMIJKnjWp2YJsmjwA+B\nw8BEVY0kWQbcBqwAHgWurarvt1mHJOnEBtEiWFVVr6iqozOVbQJ2V9UFwO5mXZI0JMPoGloDbG+W\ntwNXD6EGSVKj7SAo4N4k9yfZ0GzrVdX+ZvkA0Gu5BknSSbR6jwC4vKoeS/Ii4J4kXz72zaqqJHW8\nA5vg2ADQ6/Xo9/stlyrNjtemFrpWg6CqHmten0jyUeCXgceTLK+q/UmWA0+c4NitwFaAkZGRGh0d\nbbNUaXbuuhOvTS10rXUNJTkzyc8eXQauBB4C7gDWNbutA25vqwZJ0tTabBH0gI8mOXqev6iqu5J8\nFtiZZD2wD7i2xRokSVNoLQiq6uvAJcfZ/j1gdVvnlSTNjE8WS1LHGQSS1HEGgSR1nEEgSR1nEEhS\nxxkEktRxBoEkdZxBIEkdZxBIUscZBJLUcQaBJHWcQSBJHTdlECTpJdmW5BPN+kXNyKGSpFPAdFoE\nfwrcDfyzZv2rwDvaKkiSNFjTCYJzqmoncASgqiaAw61WJUkamOkEwaEkP8fkRPQkuQz4QatVSZIG\nZjoT09zI5PSSP5/kU8C5wFune4Iki4C9wGNV9aYky4DbgBXAo8C1VfX9GdYtSZojU7YIqupzwK8A\nrwZ+B/ilqvrCDM7xdmD8mPVNwO6qugDY3axLkoZkOt8aegvwZuBC4BeBX0uyOsmLpnHsS4A3An9y\nzOY1wPZmeTtw9UyLliTNnel0Da0HXgXsadZHgfuB85P856r6wEmO/e/A7wE/e8y2XlXtb5YPMDnJ\nvSRpSKYTBKcDK6vqcZh8rgD438Argb8BjhsESd4EPFFV9ycZPd4+VVVJ6gTHbwA2APR6Pfr9/jRK\nlQbPa1ML3XSC4LyjIdB4otn2ZJKnT3Lca4A3J/lVYAnwT5L8GfB4kuVVtT/J8ubznqOqtgJbAUZG\nRmp0dHQapUoDdtedeG1qoZvO10f7SXYlWZdkHXB7s+1M4KkTHVRVv19VL6mqFcB1wF9V1duY/AbS\numa3o58nSRqS6bQIbgDeAlzerO9lsp//ELBqFufcAuxshqnYB1w7i8+QJM2RKYOg6cf/OnAZcA3w\nDeDDMzlJVfWBfrP8PWD1TAuVJLXjhEGQ5BeBtc3Pd5l8CCxVNZtWgDRwl7z7k/zgRye7jTU3Vmy6\ns9XPf+EZi/n8u65s9RzqtpO1CL4M/C3wpqr6GkCS3x1IVdIc+MGPnubRLW9s9Rz9fr/1m8VtB410\nspvFbwH2A3uSvC/JaiCDKUuSNCgnDIKq+j9VdR3wMiYfJnsH8KIkf5zEdqoknSKmM9bQoar6i6r6\nNeAlwAPAO1uvTJI0EDOaqrKqvl9VW6vKb/1I0inCOYslqeMMAknqOINAkjrOIJCkjjMIJKnjDAJJ\n6jiDQJI6ziCQpI4zCCSp4wwCSeq41oIgyZIkf5fk80keTvLuZvuyJPckeaR5PbutGiRJU2uzRfBj\n4IqqugR4BfCGJJcBm4DdVXUBsLtZlyQNSWtBUJMONquLm58C1gDbm+3bgavbqkGSNLXpTF4/a0kW\nAfcDvwD8UVV9JkmvqvY3uxwAeic4dgOwAaDX69Hv99ssVaeotq+bgwcPDuTa9PpXm1oNgqo6DLwi\nyVnAR5Nc/Kz3K0md4NitwFaAkZGRans6QJ2C7rqz9WkkBzFV5SB+D3XbQL41VFVPMTnL2RuAx5Ms\nB2henxhEDZKk42vzW0PnNi0BkpwBvA74MnAHsK7ZbR1we1s1SJKm1mbX0HJge3Of4DRgZ1XtSvJp\nYGeS9cA+4NoWa5AkTSFVx+2in1dGRkZq7969wy5DC8zLt7982CXMmS+u++KwS9AClOT+qhqZar9W\nbxZLw/TD8S08uuWNrZ5jEDeLV2y6s9XPlxxiQpI6ziCQpI4zCCSp4wwCSeo4g0CSOs4gkKSOMwgk\nqeMMAknqOINAkjrOIJCkjjMIJKnjDAJJ6jiDQJI6ziCQpI4zCCSp49qcqvK8JHuSfCnJw0ne3mxf\nluSeJI80r2e3VYMkaWpttggmgP9QVRcBlwE3JLkI2ATsrqoLgN3NuiRpSFoLgqraX1Wfa5Z/CIwD\nLwbWANub3bYDV7dVgyRpagOZqjLJCuBfAp8BelW1v3nrANA7wTEbgA0AvV6Pfr/fep069Qxkmse7\n2j3HmYvx+lerWp+8PslS4K+BzVX1kSRPVdVZx7z//ao66X0CJ6/XfLVi052tz4sszdZ0J69v9VtD\nSRYDHwb+vKo+0mx+PMny5v3lwBNt1iBJOrk2vzUUYBswXlU3H/PWHcC6ZnkdcHtbNUiSptbmPYLX\nAL8FfDHJg822/wRsAXYmWQ/sA65tsQZJ0hRaC4Kqug/ICd5e3dZ5JUkz45PFktRxBoEkdZxBIEkd\nZxBIUscZBJLUcQaBJHWcQSBJHWcQSFLHGQSS1HEGgSR1nEEgSR1nEEhSxxkEktRxBoEkdZxBIEkd\n1+YMZe9P8kSSh47ZtizJPUkeaV5POlexJKl9bbYI/hR4w7O2bQJ2V9UFwO5mXZI0RK0FQVX9DfDk\nszavAbY3y9uBq9s6vyRpegZ9j6BXVfub5QNAb8DnlyQ9S5uT159UVVWSOtH7STYAGwB6vR79fn9Q\npUkz4rWphW7QQfB4kuVVtT/JcuCJE+1YVVuBrQAjIyM1Ojo6oBKlGbjrTrw2tdANumvoDmBds7wO\nuH3A55ckPUubXx/dAXwauDDJt5KsB7YAr0vyCPDaZl2SNEStdQ1V1doTvLW6rXNKkmbOJ4slqeMM\nAknqOINAkjrOIJCkjjMIJKnjDAJJ6jiDQJI6ziCQpI4zCCSp4wwCSeo4g0CSOs4gkKSOMwgkqeMM\nAknqOINAkjrOIJCkjhtKECR5Q5KvJPlakk3DqEGSNGngQZBkEfBHwFXARcDaJBcNug5J0qRhtAh+\nGfhaVX29qn4C/CWwZgh1SJJocc7ik3gx8M1j1r8FvHIIdUjPkWTmx/zhzM9TVTM/SGrJMIJgWpJs\nADYA9Ho9+v3+cAtSJ+zZs2dG+x88eJClS5fO+Dxez5pPhhEEjwHnHbP+kmbbT6mqrcBWgJGRkRod\nHR1IcdJM9Pt9vDa10A3jHsFngQuSnJ/kBcB1wB1DqEOSxBBaBFU1keTfAXcDi4D3V9XDg65DkjRp\nKPcIqurjwMeHcW5J0k/zyWJJ6jiDQJI6ziCQpI4zCCSp47IQnnBM8h1g37DrkI7jHOC7wy5COoGX\nVtW5U+20IIJAmq+S7K2qkWHXIT0fdg1JUscZBJLUcQaB9PxsHXYB0vPlPQJJ6jhbBJLUcQaBNIUk\nVyepJC8bdi1SGwwCaWprgfuaV+mUYxBIJ5FkKXA5sJ7JuTNIclqSW5N8Ock9ST6e5K3Ne5cm+esk\n9ye5O8nyIZYvTYtBIJ3cGuCuqvoq8L0klwJvAVYAFwG/BbwKIMli4L3AW6vqUuD9wOZhFC3NxLyd\ns1iaJ9YCtzTLf9msnw58sKqOAAeSHJ3o+ELgYuCeJDA58dL+wZYrzZxBIJ1AkmXAFcDLkxSTf9gL\n+OiJDgEerqpXDahEaU7YNSSd2FuBD1TVS6tqRVWdB3wDeBL49eZeQQ8Ybfb/CnBukme6ipL80jAK\nl2bCIJBObC3P/df/h4F/CnwL+BLwZ8DngB9U1U+YDI8/TPJ54EHg1YMrV5odnyyWZiHJ0qo6mOTn\ngL8DXlNVB4ZdlzQb3iOQZmdXkrOAFwD/xRDQQmaLQJI6znsEktRxBoEkdZxBIEkdZxCoU5LclOTh\nJF9I8mCSV87BZ745yaY5qu/gXHyONBPeLFZnNA963QyMVtWPk5wDvKCqvj2NY0+vqokB1Hiwqpa2\nfR7pWLYI1CXLge9W1Y8Bquq7VfXtJI82oUCSkST9ZvkPknwgyaeADyT5v8c+KZyk3+z/20n+R5IX\nJtmX5LTm/TOTfLN5wvjnk9zVjEr6t0fnNkhyfpJPJ/likv864P8eEmAQqFs+CZyX5KvNMNK/Mo1j\nLgJeW1VrgduAawGa4aWXV9XeoztW1Q+YfJr46Oe+Cbi7qp5mcm7jsWZU0v8I3Nrscwvwx1X1chyg\nTkNiEKgzquogcCmwAfgOcFuS357isDuq6kfN8k4mh5CAyUD40HH2vw34jWb5uuYcS5kcauKDSR4E\n/ieTrROA1wA7muUPzOgXkuaITxarU6rqMNAH+km+CKwDJvjHfxQtedYhh4459rEk30vyL5j8Y7/x\nOKe4A3hPM3LppcBfAWcCT1XVK05U1ix/HWlO2CJQZyS5MMkFx2x6BbAPeJTJP9oAvz7Fx9wG/B7w\nwqr6wrPfbFodn2Wyy2dXVR2uqn8AvpHkmqaOJLmkOeRTNDOfAb85899Kev4MAnXJUmB7ki8l+QKT\n/f9/ALwbuCXJXuDwFJ/xISb/cO88yT63AW9rXo/6TWB9Myrpw0zOfAbwduCGpnXy4pn9OtLc8Ouj\nktRxtggkqeMMAknqOINAkjrOIJCkjjMIJKnjDAJJ6jiDQJI6ziCQpI77/7J2TGhT+ImlAAAAAElF\nTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x26ee5634e0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# now let's plot the boxplots and whiskers to visualise\n",
    "# outliers\n",
    "\n",
    "# remember that the dots in the plot indicate outliers,\n",
    "# the box the interquantile range, and the whikers the \n",
    "# range IQR + or - 1.5 times the quantiles\n",
    "\n",
    "fig = data.boxplot(column='Age')\n",
    "fig.set_title('')\n",
    "fig.set_xlabel('Survived')\n",
    "fig.set_ylabel('Age')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    714.000000\n",
       "mean      29.699118\n",
       "std       14.526497\n",
       "min        0.420000\n",
       "25%       20.125000\n",
       "50%       28.000000\n",
       "75%       38.000000\n",
       "max       80.000000\n",
       "Name: Age, dtype: float64"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's visualise median and quantiles\n",
    "\n",
    "# 25%, 50% and 75% in the output below indicate the\n",
    "# 25th quantile, median and 75th quantile respectively\n",
    "\n",
    "data.Age.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(73.27860964406095, -13.88037434994331)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's calculate the boundaries outside which sit the outliers\n",
    "# assuming Age follows a Gaussian distribution\n",
    "\n",
    "Upper_boundary = data.Age.mean() + 3* data.Age.std()\n",
    "Lower_boundary = data.Age.mean() - 3* data.Age.std()\n",
    "\n",
    "Upper_boundary, Lower_boundary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The upper boundary for Age is 73-74 years. The lower boundary is meaningless as there can't be negative age. This value could be generated due to the lack of normality of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(64.8125, -6.6875, 17.875)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# now let's use the interquantile range to calculate the boundaries\n",
    "\n",
    "IQR = data.Age.quantile(0.75) - data.Age.quantile(0.25)\n",
    "\n",
    "Lower_fence = data.Age.quantile(0.25) - (IQR * 1.5)\n",
    "Upper_fence = data.Age.quantile(0.75) + (IQR * 1.5)\n",
    "\n",
    "Upper_fence, Lower_fence, IQR"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(91.625, -33.5, 17.875)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# and for extreme outliers\n",
    "\n",
    "IQR = data.Age.quantile(0.75) - data.Age.quantile(0.25)\n",
    "\n",
    "Lower_fence = data.Age.quantile(0.25) - (IQR * 3)\n",
    "Upper_fence = data.Age.quantile(0.75) + (IQR * 3)\n",
    "\n",
    "Upper_fence, Lower_fence, IQR"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The boundary using 1.5 times the interquantile range coincides roughly with the boundary determined using the Gaussian distribution (64 vs 71 years). The value using the 3 times the interquantile is a bit high according to normal human life expectancy, particularly in the days of the Titanic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "passengers older than 73 (Gaussian approach): 0.0028011204481792717\n",
      "passengers older than 65 (IQR): 0.011204481792717087\n",
      "passengers older than 91 (IQR, extreme): 0.0\n"
     ]
    }
   ],
   "source": [
    "# let's find out whether there are outliers according to the above boundaries:\n",
    "\n",
    "# let's remove first the passengers with missing data for Age\n",
    "data = data.dropna(subset=['Age'])\n",
    "\n",
    "total_passengers = np.float(data.shape[0])\n",
    "\n",
    "print('passengers older than 73 (Gaussian approach): {}'.format(\n",
    "    data[data.Age > 73].shape[0] / total_passengers))\n",
    "\n",
    "print('passengers older than 65 (IQR): {}'.format(\n",
    "    data[data.Age > 65].shape[0] / total_passengers))\n",
    "\n",
    "print('passengers older than 91 (IQR, extreme): {}'.format(\n",
    "    data[data.Age >= 91].shape[0] / total_passengers))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Roughly ~1-2 percent of the passengers were extremely old."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Wheadon, Mr. Edward H</td>\n",
       "      <td>male</td>\n",
       "      <td>66.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>C.A. 24579</td>\n",
       "      <td>10.5000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>97</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Goldschmidt, Mr. George B</td>\n",
       "      <td>male</td>\n",
       "      <td>71.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17754</td>\n",
       "      <td>34.6542</td>\n",
       "      <td>A5</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>116</th>\n",
       "      <td>117</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Connors, Mr. Patrick</td>\n",
       "      <td>male</td>\n",
       "      <td>70.5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>370369</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>493</th>\n",
       "      <td>494</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Artagaveytia, Mr. Ramon</td>\n",
       "      <td>male</td>\n",
       "      <td>71.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17609</td>\n",
       "      <td>49.5042</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>630</th>\n",
       "      <td>631</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Barkworth, Mr. Algernon Henry Wilson</td>\n",
       "      <td>male</td>\n",
       "      <td>80.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>27042</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>A23</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>672</th>\n",
       "      <td>673</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Mitchell, Mr. Henry Michael</td>\n",
       "      <td>male</td>\n",
       "      <td>70.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>C.A. 24580</td>\n",
       "      <td>10.5000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>745</th>\n",
       "      <td>746</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Crosby, Capt. Edward Gifford</td>\n",
       "      <td>male</td>\n",
       "      <td>70.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>WE/P 5735</td>\n",
       "      <td>71.0000</td>\n",
       "      <td>B22</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>851</th>\n",
       "      <td>852</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Svensson, Mr. Johan</td>\n",
       "      <td>male</td>\n",
       "      <td>74.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>347060</td>\n",
       "      <td>7.7750</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Survived  Pclass                                  Name  \\\n",
       "33            34         0       2                 Wheadon, Mr. Edward H   \n",
       "96            97         0       1             Goldschmidt, Mr. George B   \n",
       "116          117         0       3                  Connors, Mr. Patrick   \n",
       "493          494         0       1               Artagaveytia, Mr. Ramon   \n",
       "630          631         1       1  Barkworth, Mr. Algernon Henry Wilson   \n",
       "672          673         0       2           Mitchell, Mr. Henry Michael   \n",
       "745          746         0       1          Crosby, Capt. Edward Gifford   \n",
       "851          852         0       3                   Svensson, Mr. Johan   \n",
       "\n",
       "      Sex   Age  SibSp  Parch      Ticket     Fare Cabin Embarked  \n",
       "33   male  66.0      0      0  C.A. 24579  10.5000   NaN        S  \n",
       "96   male  71.0      0      0    PC 17754  34.6542    A5        C  \n",
       "116  male  70.5      0      0      370369   7.7500   NaN        Q  \n",
       "493  male  71.0      0      0    PC 17609  49.5042   NaN        C  \n",
       "630  male  80.0      0      0       27042  30.0000   A23        S  \n",
       "672  male  70.0      0      0  C.A. 24580  10.5000   NaN        S  \n",
       "745  male  70.0      1      1   WE/P 5735  71.0000   B22        S  \n",
       "851  male  74.0      0      0      347060   7.7750   NaN        S  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[data.Age>65]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the majority of the outliers did not survive.\n",
    "\n",
    "We have now identified a bunch of potential outliers. Let's see whether these affect the performance of the machine learning algorithms."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Measuring the effect of outliers on different machine learning algorithms"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "3            4         1       1   \n",
       "4            5         0       3   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                           Allen, Mr. William Henry    male  35.0      0   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  \n",
       "0      0         A/5 21171   7.2500   NaN        S  \n",
       "1      0          PC 17599  71.2833   C85        C  \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      0            113803  53.1000  C123        S  \n",
       "4      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's load the titanic dataset again\n",
    "\n",
    "data = pd.read_csv('titanic.csv')\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Age     0.198653\n",
       "Fare    0.000000\n",
       "dtype: float64"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's find out if the variables contain missing data\n",
    "\n",
    "data[['Age', 'Fare']].isnull().mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Age contains 20 % of missing data. For simplicity, I will fill the missing values with 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((623, 2), (268, 2))"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's separate into training and testing set\n",
    "\n",
    "# remember that to avoid overfitting and improve generalisation\n",
    "# machine learning models need to be built on a train set\n",
    "# and evaluated on a test set\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    data[['Age', 'Fare']].fillna(0),\n",
    "    data.Survived,\n",
    "    test_size=0.3,\n",
    "    random_state=0)\n",
    "\n",
    "X_train.shape, X_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Let's generate training and testing sets without outliers\n",
    "\n",
    "# For simplicity, I will replace outliers by the upper boundary values\n",
    "# this procedure is called capping or top-coding and I will cover it\n",
    "# extensively in the section of handling outliers later in the course\n",
    "\n",
    "# let's create a new dataset\n",
    "data_clean = data.copy()\n",
    "\n",
    "# replace outliers in Age\n",
    "# using the boundary from the Gaussian assumption method\n",
    "data_clean.loc[data_clean.Age >= 73, 'Age'] = 73\n",
    "\n",
    "# replace outliers in Fare\n",
    "# using the boundary of the interquantile range method\n",
    "data_clean.loc[data_clean.Fare > 100, 'Fare'] = 100\n",
    "\n",
    "# Let's divide into train and test sets\n",
    "X_train_clean, X_test_clean, y_train_clean, y_test_clean = train_test_split(\n",
    "    data_clean[['Age', 'Fare']].fillna(0),\n",
    "    data_clean.Survived,\n",
    "    test_size=0.3,\n",
    "    random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Outlier effect on Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LogReg Accuracy: 0.6940298507462687\n",
      "LogReg roc-auc: 0.7132142857142858\n"
     ]
    }
   ],
   "source": [
    "# model build on data with outliers\n",
    "\n",
    "# call model\n",
    "logit = LogisticRegression(random_state=44)\n",
    "\n",
    "# train model\n",
    "logit.fit(X_train, y_train)\n",
    "\n",
    "# make predicion on test set\n",
    "pred = logit.predict_proba(X_test)\n",
    "\n",
    "print('LogReg Accuracy: {}'.format(logit.score(X_test, y_test)))\n",
    "print('LogReg roc-auc: {}'.format(roc_auc_score(y_test, pred[:, 1])))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LogReg Accuracy: 0.7052238805970149\n",
      "LogReg roc-auc: 0.7133333333333333\n"
     ]
    }
   ],
   "source": [
    "# model build on data without outliers\n",
    "\n",
    "# call model\n",
    "logit = LogisticRegression(random_state=44)\n",
    "\n",
    "# train\n",
    "logit.fit(X_train_clean, y_train_clean)\n",
    "\n",
    "# make prediction on test set\n",
    "pred = logit.predict_proba(X_test_clean)\n",
    "\n",
    "print('LogReg Accuracy: {}'.format(logit.score(X_test_clean, y_test_clean)))\n",
    "print('LogReg roc-auc: {}'.format(roc_auc_score(y_test_clean, pred[:, 1])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Outliers did not seem to have a big impact in the performance of Logistic regression."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Outlier Effect on Adaboost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AdaBoost Accuracy: 0.7201492537313433\n",
      "AdaBoost roc-auc: 0.7464583333333332\n"
     ]
    }
   ],
   "source": [
    "# model built on data with outliers\n",
    "\n",
    "# call model\n",
    "ada = AdaBoostClassifier(n_estimators=200, random_state=44)\n",
    "\n",
    "# train model\n",
    "ada.fit(X_train, y_train)\n",
    "\n",
    "# make predictions\n",
    "pred = ada.predict_proba(X_test)\n",
    "\n",
    "print('AdaBoost Accuracy: {}'.format(ada.score(X_test, y_test)))\n",
    "print('AdaBoost roc-auc: {}'.format(roc_auc_score(y_test, pred[:, 1])))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AdaBoost Accuracy: 0.7089552238805971\n",
      "AdaBoost roc-auc: 0.7596130952380953\n"
     ]
    }
   ],
   "source": [
    "# model built on data without the outliers\n",
    "\n",
    "# call model\n",
    "ada = AdaBoostClassifier(n_estimators=200, random_state=44)\n",
    "\n",
    "# train model\n",
    "ada.fit(X_train_clean, y_train_clean)\n",
    "\n",
    "# make predictions\n",
    "pred = ada.predict_proba(X_test_clean)\n",
    "\n",
    "print('AdaBoost Accuracy: {}'.format(ada.score(X_test_clean, y_test_clean)))\n",
    "print('AdaBoost roc-auc: {}'.format(roc_auc_score(y_test_clean, pred[:,1])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On the other hand, we can see how removing outliers improves the performance of Adaboost: 0.759 vs 0.746 roc-auc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Outlier Effect on Random Forests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Random Forests Accuracy: 0.7089552238805971\n",
      "Random Forests roc-auc: 0.7324107142857142\n"
     ]
    }
   ],
   "source": [
    "# model built on data with outliers\n",
    "\n",
    "# call model\n",
    "rf = RandomForestClassifier(n_estimators=200, random_state=39)\n",
    "\n",
    "# train model\n",
    "rf.fit(X_train, y_train)\n",
    "\n",
    "# make predictions\n",
    "pred = rf.predict_proba(X_test)\n",
    "\n",
    "print('Random Forests Accuracy: {}'.format(rf.score(X_test, y_test)))\n",
    "print('Random Forests roc-auc: {}'.format(roc_auc_score(y_test, pred[:, 1])))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Random Forests Accuracy: 0.6977611940298507\n",
      "Random Forests roc-auc: 0.7285714285714286\n"
     ]
    }
   ],
   "source": [
    "# model built on data without outliers\n",
    "\n",
    "# call model\n",
    "rf = RandomForestClassifier(n_estimators=200, random_state=39)\n",
    "\n",
    "# train model\n",
    "rf.fit(X_train_clean, y_train_clean)\n",
    "\n",
    "# make predictions\n",
    "pred = rf.predict_proba(X_test_clean)\n",
    "\n",
    "print('Random Forests Accuracy: {}'.format(rf.score(X_test_clean, y_test_clean)))\n",
    "print('Random Forests roc-auc: {}'.format(roc_auc_score(y_test_clean, pred[:,1])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected, Random Forests do not benefit from removing outliers from the dataset.\n",
    "\n",
    "**Conclusion**:\n",
    "\n",
    "We can see that the presence of outliers affects the performance of AdaBoost, and when outliers are removed the roc-auc improved by 0.013. \n",
    "\n",
    "Logistic Regression's and Random Forests performances seemed unaffected by outliers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**That is all for this demonstration. I hope you enjoyed the notebook, and see you in the next one.**"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": "block",
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}