3GCIV-Machine-Learning-Tutorial-Pulsar Prediction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Build simple models to predict pulsar candidates\n",
    "\n",
    "In this notebook we will look at building machine learning models to predict Pulsar Candidate. The data comes from Rob Lyon at Manchester. This data is publically available. For more information check out https://figshare.com/articles/HTRU2/3080389/1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Lets start with the basic imports\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python2.7/dist-packages/sklearn/cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.\n",
      "  \"This module will be removed in 0.20.\", DeprecationWarning)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "# Some preprocessing utilities\n",
    "from sklearn.cross_validation import train_test_split # Data splitting\n",
    "from sklearn.utils import shuffle\n",
    "\n",
    "# The different classifiers\n",
    "from sklearn.tree import DecisionTreeClassifier # Decision Tree\n",
    "from sklearn.svm import SVC # Support Vector Machines\n",
    "from sklearn.neighbors import KNeighborsClassifier # Nearest Neighbor \n",
    "from sklearn.naive_bayes import GaussianNB # Bayesian Classifier\n",
    "\n",
    "# Model result function\n",
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Load dataset\n",
    "\n",
    "* Data is a csv file with each column as features and rows as samples of positive and negative candidates\n",
    "\n",
    "* Class label is the last column where \"1\" correspondes to true pulsar candidate and \"0\" a false candidate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset has 17898 rows and 9 columns including features and labels\n"
     ]
    }
   ],
   "source": [
    "data = np.loadtxt('Data/Pulsar/HTRU_2.csv',delimiter=',')\n",
    "\n",
    "# Show some information\n",
    "print 'Dataset has %d rows and %d columns including features and labels'%(data.shape[0],data.shape[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Get the features and labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Lets shuffle the rows of the data 10 times\n",
    "for i in range(10):\n",
    "    data = shuffle(data)\n",
    "\n",
    "# Now split the dataset into seperate variabels for features and labels\n",
    "features = data[:,0:-1] # Features are all columns till the second last one\n",
    "labels = data[:,-1] # Labels are usually the last column"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Split data to training and validation sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of training data points : 12528\n",
      "Number of testing data points : 5370\n"
     ]
    }
   ],
   "source": [
    "# Do a 70 - 30 split of the whole data for training and testing\n",
    "# The last argument specifies the fraction of samples for testing\n",
    "train_data,test_data,train_labels,test_labels = train_test_split(features,labels,test_size=.3)\n",
    "#Print some info\n",
    "print 'Number of training data points : %d'%(train_data.shape[0])\n",
    "print 'Number of testing data points : %d'%(test_data.shape[0])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Lets do the training on different algorithms\n",
    "\n",
    "We will be using the following algorithms\n",
    "\n",
    "* Decision Trees  [ https://en.wikipedia.org/wiki/Decision_tree_learning ]\n",
    "\n",
    "\n",
    "* Support Vector Machines (SVM)  [ https://en.wikipedia.org/wiki/Support_vector_machine ]\n",
    "\n",
    "\n",
    "* k-Nearest Neighbours (KNN) [ https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm ]\n",
    "\n",
    "\n",
    "* Naive Bayes Classifier [ https://en.wikipedia.org/wiki/Naive_Bayes_classifier ]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets start with default model parameters for each classifier.\n",
    "Check the link above each block for function definition\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Scikit Decision Tree\n",
    "\n",
    "http://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html#sklearn.tree.DecisionTreeClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=None,\n",
       "            max_features=None, max_leaf_nodes=None,\n",
       "            min_impurity_split=1e-07, min_samples_leaf=1,\n",
       "            min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
       "            presort=False, random_state=None, splitter='best')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dt = DecisionTreeClassifier() # Make the classifier object\n",
    "dt.fit(train_data,train_labels) # Train the model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Scikit SVM\n",
    "\n",
    "http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape=None, degree=3, gamma='auto', kernel='rbf',\n",
       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "  tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Support Vector Machine\n",
    "svm = SVC()\n",
    "svm.fit(train_data,train_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Scikit KNN\n",
    "\n",
    "http://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "           metric_params=None, n_jobs=1, n_neighbors=5, p=2,\n",
       "           weights='uniform')"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# K nearest neighbor\n",
    "knn = KNeighborsClassifier()\n",
    "knn.fit(train_data,train_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Scikit Naive Bayes\n",
    "\n",
    "http://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.GaussianNB.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GaussianNB(priors=None)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Naive Bayes\n",
    "nb = GaussianNB()\n",
    "nb.fit(train_data,train_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Fancy function to print results for model evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Pretty function to test  a model and print accuracy score\n",
    "def evaluate(model,modelname,test_data,test_labels):\n",
    "    predictions = model.predict(test_data) # Do the actual prediction\n",
    "    print '%s is %f%% accurate \\n'%(modelname,accuracy_score(test_labels,predictions)*100)\n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Making some stuff easy\n",
    "models =[dt,svm,knn,nb]\n",
    "model_names =['Decision Tree','Support Vector Machines','KNN','Naive Bayes']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Now Lets test each classifier and disply their accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "for i in range(0,4):\n",
    "    evaluate(models[i],model_names[i],test_data,test_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}