Tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## YOU CAN EXECUTE EACH CELL WITH SHIFT+ENTER"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Understand the Titanic Case\n",
    "\n",
    "The Titanic case is a famous Machine Learning problem.\n",
    "\n",
    "We have the passengers list, **and the goal is to predict who were likely to survive!**\n",
    "\n",
    "Yes, that is creepy ... but it works like a charm :)\n",
    "\n",
    "Learn it on [Kaggle](https://www.kaggle.com/c/titanic)!\n",
    "\n",
    "For this tutorial, you must fill the **TODO part** :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Theses imports are for charts (with matplotlib and seaborn)\n",
    "%matplotlib inline\n",
    "%pylab inline\n",
    "pylab.rcParams['figure.figsize'] = (16, 8)\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "sns.set_style('whitegrid')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 1: Load and analyze the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, load the dataset with [pandas](http://pandas.pydata.org) and analyze metadata"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import pandas library. By convention, pandas is renamed to 'pd'.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the dataset with [```pd.read_csv```](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html) method and affect it to ```df``` (df = DataFrame)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Show metadatas with the [```df.info()```](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.info.html) method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay. Let's understand what we have :\n",
    "\n",
    "```\n",
    "RangeIndex: 891 entries, 0 to 890\n",
    "```\n",
    "\n",
    "We have 891 rows in the dataset.\n",
    "\n",
    "Columns are:\n",
    "\n",
    "* ```id```: Unique identifier of the passenger\n",
    "* ```name```: Name of the passenger\n",
    "* ```gender```: Sex of the passenger (male OR female)\n",
    "* ```age```: Age of the passenger (could be used to determine if passenger is a child or an adult)\n",
    "* ```sibling_spouse```: Number of siblings OR spouses with the passenger (brother, sister, spouse, etc.)\n",
    "* ```parent_children```: Number of parent OR children with the passenger (father, mother, son, daugther, etc.)\n",
    "* ```fare```: Ticket price\n",
    "* ```passenger_class```: Class of the ticket. There was 3 classes on the Titanic: 1st to 3th (1st is the most rich).\n",
    "* ```survived```: Does the passenger survived to the shipwreck ? We want to **predict** this column.\n",
    "\n",
    "Remark: The ```age``` column has empty value (714/891). If we use this column, we have to fill empty value (otherwise algorithm will crash)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Show content with [```df.head()```](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.head.html) method (the first 5 lines):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 2: Create our first model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, it's time to create our first model. \n",
    "\n",
    "\n",
    "We will use a classic logistic regression ([```LogisticRegression```](http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression)) from [SciKit Learn](http://scikit-learn.org).\n",
    "\n",
    "But, we need to evaluate the model! So, we will use a cross validation function.\n",
    "\n",
    "This method will randomly split the dataset and check if the model can predict the real value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "import numpy as np\n",
    "\n",
    "def compute_score(model, X, y):\n",
    "    # Evaluate the model with random train/set, 5 times\n",
    "    scores = cross_val_score(model, X, y, cv=5, error_score='raise')\n",
    "    \n",
    "    # And return the average of the 5 scores\n",
    "    return np.mean(scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To test our models, we need to define the target.\n",
    "\n",
    "Affect the ```df.survived``` column to the ```y``` value:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For our first model, we will use ```fare```, ```sibling_spouse``` and ```parent_children```.\n",
    "\n",
    "Affect theses columns to the ```X``` dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, a ```X``` dataset can be trained with our ```y``` target.\n",
    "\n",
    "Instanciate a [```LogisticRegression```](http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression) classifier and create a score with the ```compute_score``` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This score is **good** !!!\n",
    "\n",
    "But we could improve it. Let's add some features to the dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 3: Add the ```passenger_class``` feature"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There was 3 passenger classes on the Titanic: 1st to 3th (1st is the most rich).\n",
    "\n",
    "**Can the ```passenger_class``` help to predict the ```survived``` ?**\n",
    "\n",
    "We will combine theses columns on a bar chart from [seaborn](https://stanford.edu/~mwaskom/software/seaborn).\n",
    "\n",
    "Use the [```sns.countplot()```](https://stanford.edu/~mwaskom/software/seaborn/generated/seaborn.countplot.html) method to show it (x is ```survived```, hue is ```passenger_class```):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's obvious! If you bought a 3th class ticket, you died.\n",
    "\n",
    "Let's add the ```passenger_class``` to our ```X``` dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And score with our [```LogisticRegression```](http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression) classifier:\n",
    "\n",
    "**WARNING**: Execute it but it cannot work!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It cannot work!\n",
    "\n",
    "We added a **Categorical column** to a numerical model (logistic regression).\n",
    "\n",
    "To convert categorical column to numerical column, we use the One Versus All trick.\n",
    "\n",
    "We have 3 columns, one by category (c1, c2 or c3). If the passenger is in a category, value is 1. Otherwise, value is 0.\n",
    "\n",
    "```\n",
    "cl1 = [1, 0, 0]\n",
    "cl2 = [0, 1, 0]\n",
    "cl3 = [0, 0, 1]\n",
    "```\n",
    "\n",
    "Fortunately, Pandas has a method [```pd.get_dummies```](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.get_dummies.html) to create column. \n",
    "\n",
    "The syntax is :\n",
    "\n",
    "```\n",
    "class_dummies = pd.get_dummies(df[columns], columns=columns)\n",
    "```\n",
    "\n",
    "(```columns``` is an array of names)\n",
    "\n",
    "Create all columns and affect them to the ```class_dummies``` value:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see the head of ```class_dummies```:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, use [```df.join```](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.join.html) method to merge previous columns with ```class_dummies``` and create the ```X``` dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And score with our [```LogisticRegression```](http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression) classifier:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We improved the score with **2 points** !!!\n",
    "\n",
    "Let's continue :)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 4: Add the ```gender``` feature"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On a boat, **Women and children first !**\n",
    "\n",
    "**Can the ```gender``` help to predict the ```survived``` ?**\n",
    "\n",
    "We will combine theses columns on a bar chart.\n",
    "\n",
    "Use the [```sns.countplot()```](https://stanford.edu/~mwaskom/software/seaborn/generated/seaborn.countplot.html) method to show it (x is ```survived```, hue is ```gender```):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's obvious ! Men **died** and women **lived**.\n",
    "\n",
    "Let's add the ```gender``` to our ```X``` dataset.\n",
    "\n",
    "Be carefull! ```gender``` is a categorical column, like ```passenger_class```.\n",
    "\n",
    "So, we need to use the [```pd.get_dummies```](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.get_dummies.html) method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, use [```df.join```](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.join.html) method to merge previous columns with ```class_dummies``` and create the ```X``` dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And score with our [```LogisticRegression```](http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression) classifier:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We improved the score with **10 points** !!! ```gender``` is a very good ```feature```.\n",
    "\n",
    "Let's continue to improve our score :)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 5: Create the is_child feature"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Can the ```age``` help to predict the ```survived``` ?**\n",
    "\n",
    "We will combine theses columns on a violin chart (because ```age``` is a discrete value).\n",
    "\n",
    "Use the [```sns.violinplot()```](https://stanford.edu/~mwaskom/software/seaborn/generated/seaborn.violinplot.html#seaborn.violinplot) method to show it (x is ```survived```, y is ```age```):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This time, it is not obvious.\n",
    "\n",
    "Be if we have a clother look at the ```survived``` violin, there is are lot of passenger under 8 years old who survived!\n",
    "\n",
    "So, we can create a new feature ```is_child``` to help our model.\n",
    "\n",
    "However, remember the data analysis ! The age column contains empty values. So we need to fill them. But, we need to carefully choose the value: it could downgrade our score. A good choice is to use the **median**.\n",
    "\n",
    "Combine [```df.fillna()```](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.fillna.html) method and [```df.median()```](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.median.html) method to fill empty value:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, all values are filled.\n",
    "\n",
    "Let's create a new column **is_child** if the age is stricly under 8 :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, create the ```X``` dataset with ```is_child```:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And score with our [```LogisticRegression```](http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression) classifier:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We improved the score with **1 point** !!!\n",
    "\n",
    "Let's change our algorithm :)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 6: Replace algorithm by a Random Forest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For previous steps, we used a logical regression. Let's try another algorithm, the Random Forest !\n",
    "\n",
    "Use the [```RandomForestClassifier```](http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html) with 100 estimators :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "# TODO\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We improved the score with **1 point** !!!\n",
    "\n",
    "You're a good datascientist :)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}