credit_risk_resampling.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Credit Risk Classification\n",
    "\n",
    "Credit risk poses a classification problem that’s inherently imbalanced. This is because healthy loans easily outnumber risky loans. In this Challenge, you’ll use various techniques to train and evaluate models with imbalanced classes. You’ll use a dataset of historical lending activity from a peer-to-peer lending services company to build a model that can identify the creditworthiness of borrowers.\n",
    "\n",
    "## Instructions:\n",
    "\n",
    "This challenge consists of the following subsections:\n",
    "\n",
    "* Split the Data into Training and Testing Sets\n",
    "\n",
    "* Create a Logistic Regression Model with the Original Data\n",
    "\n",
    "* Predict a Logistic Regression Model with Resampled Training Data \n",
    "\n",
    "### Split the Data into Training and Testing Sets\n",
    "\n",
    "Open the starter code notebook and then use it to complete the following steps.\n",
    "\n",
    "1. Read the `lending_data.csv` data from the `Resources` folder into a Pandas DataFrame.\n",
    "\n",
    "2. Create the labels set (`y`)  from the “loan_status” column, and then create the features (`X`) DataFrame from the remaining columns.\n",
    "\n",
    "    > **Note** A value of `0` in the “loan_status” column means that the loan is healthy. A value of `1` means that the loan has a high risk of defaulting.  \n",
    "\n",
    "3. Check the balance of the labels variable (`y`) by using the `value_counts` function.\n",
    "\n",
    "4. Split the data into training and testing datasets by using `train_test_split`.\n",
    "\n",
    "### Create a Logistic Regression Model with the Original Data\n",
    "\n",
    "Employ your knowledge of logistic regression to complete the following steps:\n",
    "\n",
    "1. Fit a logistic regression model by using the training data (`X_train` and `y_train`).\n",
    "\n",
    "2. Save the predictions on the testing data labels by using the testing feature data (`X_test`) and the fitted model.\n",
    "\n",
    "3. Evaluate the model’s performance by doing the following:\n",
    "\n",
    "    * Calculate the accuracy score of the model.\n",
    "\n",
    "    * Generate a confusion matrix.\n",
    "\n",
    "    * Print the classification report.\n",
    "\n",
    "4. Answer the following question: How well does the logistic regression model predict both the `0` (healthy loan) and `1` (high-risk loan) labels?\n",
    "\n",
    "### Predict a Logistic Regression Model with Resampled Training Data\n",
    "\n",
    "Did you notice the small number of high-risk loan labels? Perhaps, a model that uses resampled data will perform better. You’ll thus resample the training data and then reevaluate the model. Specifically, you’ll use `RandomOverSampler`.\n",
    "\n",
    "To do so, complete the following steps:\n",
    "\n",
    "1. Use the `RandomOverSampler` module from the imbalanced-learn library to resample the data. Be sure to confirm that the labels have an equal number of data points. \n",
    "\n",
    "2. Use the `LogisticRegression` classifier and the resampled data to fit the model and make predictions.\n",
    "\n",
    "3. Evaluate the model’s performance by doing the following:\n",
    "\n",
    "    * Calculate the accuracy score of the model.\n",
    "\n",
    "    * Generate a confusion matrix.\n",
    "\n",
    "    * Print the classification report.\n",
    "    \n",
    "4. Answer the following question: How well does the logistic regression model, fit with oversampled data, predict both the `0` (healthy loan) and `1` (high-risk loan) labels?\n",
    "\n",
    "### Write a Credit Risk Analysis Report\n",
    "\n",
    "For this section, you’ll write a brief report that includes a summary and an analysis of the performance of both machine learning models that you used in this challenge. You should write this report as the `README.md` file included in your GitHub repository.\n",
    "\n",
    "Structure your report by using the report template that `Starter_Code.zip` includes, and make sure that it contains the following:\n",
    "\n",
    "1. An overview of the analysis: Explain the purpose of this analysis.\n",
    "\n",
    "\n",
    "2. The results: Using bulleted lists, describe the balanced accuracy scores and the precision and recall scores of both machine learning models.\n",
    "\n",
    "3. A summary: Summarize the results from the machine learning models. Compare the two versions of the dataset predictions. Include your recommendation for the model to use, if any, on the original vs. the resampled data. If you don’t recommend either model, justify your reasoning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the modules\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from pathlib import Path\n",
    "from sklearn.metrics import balanced_accuracy_score\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from imblearn.metrics import classification_report_imbalanced\n",
    "from sklearn.metrics import classification_report\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split the Data into Training and Testing Sets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: Read the `lending_data.csv` data from the `Resources` folder into a Pandas DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>loan_size</th>\n",
       "      <th>interest_rate</th>\n",
       "      <th>borrower_income</th>\n",
       "      <th>debt_to_income</th>\n",
       "      <th>num_of_accounts</th>\n",
       "      <th>derogatory_marks</th>\n",
       "      <th>total_debt</th>\n",
       "      <th>loan_status</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10700.0</td>\n",
       "      <td>7.672</td>\n",
       "      <td>52800</td>\n",
       "      <td>0.431818</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>22800</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8400.0</td>\n",
       "      <td>6.692</td>\n",
       "      <td>43600</td>\n",
       "      <td>0.311927</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>13600</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>9000.0</td>\n",
       "      <td>6.963</td>\n",
       "      <td>46100</td>\n",
       "      <td>0.349241</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>16100</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10700.0</td>\n",
       "      <td>7.664</td>\n",
       "      <td>52700</td>\n",
       "      <td>0.430740</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>22700</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10800.0</td>\n",
       "      <td>7.698</td>\n",
       "      <td>53000</td>\n",
       "      <td>0.433962</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>23000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   loan_size  interest_rate  borrower_income  debt_to_income  num_of_accounts  \\\n",
       "0    10700.0          7.672            52800        0.431818                5   \n",
       "1     8400.0          6.692            43600        0.311927                3   \n",
       "2     9000.0          6.963            46100        0.349241                3   \n",
       "3    10700.0          7.664            52700        0.430740                5   \n",
       "4    10800.0          7.698            53000        0.433962                5   \n",
       "\n",
       "   derogatory_marks  total_debt  loan_status  \n",
       "0                 1       22800            0  \n",
       "1                 0       13600            0  \n",
       "2                 0       16100            0  \n",
       "3                 1       22700            0  \n",
       "4                 1       23000            0  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>loan_size</th>\n",
       "      <th>interest_rate</th>\n",
       "      <th>borrower_income</th>\n",
       "      <th>debt_to_income</th>\n",
       "      <th>num_of_accounts</th>\n",
       "      <th>derogatory_marks</th>\n",
       "      <th>total_debt</th>\n",
       "      <th>loan_status</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>77531</th>\n",
       "      <td>19100.0</td>\n",
       "      <td>11.261</td>\n",
       "      <td>86600</td>\n",
       "      <td>0.653580</td>\n",
       "      <td>12</td>\n",
       "      <td>2</td>\n",
       "      <td>56600</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77532</th>\n",
       "      <td>17700.0</td>\n",
       "      <td>10.662</td>\n",
       "      <td>80900</td>\n",
       "      <td>0.629172</td>\n",
       "      <td>11</td>\n",
       "      <td>2</td>\n",
       "      <td>50900</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77533</th>\n",
       "      <td>17600.0</td>\n",
       "      <td>10.595</td>\n",
       "      <td>80300</td>\n",
       "      <td>0.626401</td>\n",
       "      <td>11</td>\n",
       "      <td>2</td>\n",
       "      <td>50300</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77534</th>\n",
       "      <td>16300.0</td>\n",
       "      <td>10.068</td>\n",
       "      <td>75300</td>\n",
       "      <td>0.601594</td>\n",
       "      <td>10</td>\n",
       "      <td>2</td>\n",
       "      <td>45300</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77535</th>\n",
       "      <td>15600.0</td>\n",
       "      <td>9.742</td>\n",
       "      <td>72300</td>\n",
       "      <td>0.585062</td>\n",
       "      <td>9</td>\n",
       "      <td>2</td>\n",
       "      <td>42300</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       loan_size  interest_rate  borrower_income  debt_to_income  \\\n",
       "77531    19100.0         11.261            86600        0.653580   \n",
       "77532    17700.0         10.662            80900        0.629172   \n",
       "77533    17600.0         10.595            80300        0.626401   \n",
       "77534    16300.0         10.068            75300        0.601594   \n",
       "77535    15600.0          9.742            72300        0.585062   \n",
       "\n",
       "       num_of_accounts  derogatory_marks  total_debt  loan_status  \n",
       "77531               12                 2       56600            1  \n",
       "77532               11                 2       50900            1  \n",
       "77533               11                 2       50300            1  \n",
       "77534               10                 2       45300            1  \n",
       "77535                9                 2       42300            1  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Read the CSV file from the Resources folder into a Pandas DataFrame\n",
    "lending_data_df = pd.read_csv(\n",
    "    Path('Resources/lending_data.csv')\n",
    ")\n",
    "\n",
    "# Review the DataFrame\n",
    "display(lending_data_df.head())\n",
    "display(lending_data_df.tail())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Create the labels set (`y`)  from the “loan_status” column, and then create the features (`X`) DataFrame from the remaining columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Separate the data into labels and features\n",
    "\n",
    "# Separate the y variable, the labels\n",
    "y = lending_data_df['loan_status']\n",
    "\n",
    "# Separate the X variable, the features\n",
    "X = lending_data_df.drop(columns = 'loan_status')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    0\n",
       "1    0\n",
       "2    0\n",
       "3    0\n",
       "4    0\n",
       "Name: loan_status, dtype: int64"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "77531    1\n",
       "77532    1\n",
       "77533    1\n",
       "77534    1\n",
       "77535    1\n",
       "Name: loan_status, dtype: int64"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Review the y variable Series\n",
    "display(y.head())\n",
    "display(y.tail())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>loan_size</th>\n",
       "      <th>interest_rate</th>\n",
       "      <th>borrower_income</th>\n",
       "      <th>debt_to_income</th>\n",
       "      <th>num_of_accounts</th>\n",
       "      <th>derogatory_marks</th>\n",
       "      <th>total_debt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10700.0</td>\n",
       "      <td>7.672</td>\n",
       "      <td>52800</td>\n",
       "      <td>0.431818</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>22800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8400.0</td>\n",
       "      <td>6.692</td>\n",
       "      <td>43600</td>\n",
       "      <td>0.311927</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>13600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>9000.0</td>\n",
       "      <td>6.963</td>\n",
       "      <td>46100</td>\n",
       "      <td>0.349241</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>16100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10700.0</td>\n",
       "      <td>7.664</td>\n",
       "      <td>52700</td>\n",
       "      <td>0.430740</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>22700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10800.0</td>\n",
       "      <td>7.698</td>\n",
       "      <td>53000</td>\n",
       "      <td>0.433962</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>23000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   loan_size  interest_rate  borrower_income  debt_to_income  num_of_accounts  \\\n",
       "0    10700.0          7.672            52800        0.431818                5   \n",
       "1     8400.0          6.692            43600        0.311927                3   \n",
       "2     9000.0          6.963            46100        0.349241                3   \n",
       "3    10700.0          7.664            52700        0.430740                5   \n",
       "4    10800.0          7.698            53000        0.433962                5   \n",
       "\n",
       "   derogatory_marks  total_debt  \n",
       "0                 1       22800  \n",
       "1                 0       13600  \n",
       "2                 0       16100  \n",
       "3                 1       22700  \n",
       "4                 1       23000  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>loan_size</th>\n",
       "      <th>interest_rate</th>\n",
       "      <th>borrower_income</th>\n",
       "      <th>debt_to_income</th>\n",
       "      <th>num_of_accounts</th>\n",
       "      <th>derogatory_marks</th>\n",
       "      <th>total_debt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>77531</th>\n",
       "      <td>19100.0</td>\n",
       "      <td>11.261</td>\n",
       "      <td>86600</td>\n",
       "      <td>0.653580</td>\n",
       "      <td>12</td>\n",
       "      <td>2</td>\n",
       "      <td>56600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77532</th>\n",
       "      <td>17700.0</td>\n",
       "      <td>10.662</td>\n",
       "      <td>80900</td>\n",
       "      <td>0.629172</td>\n",
       "      <td>11</td>\n",
       "      <td>2</td>\n",
       "      <td>50900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77533</th>\n",
       "      <td>17600.0</td>\n",
       "      <td>10.595</td>\n",
       "      <td>80300</td>\n",
       "      <td>0.626401</td>\n",
       "      <td>11</td>\n",
       "      <td>2</td>\n",
       "      <td>50300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77534</th>\n",
       "      <td>16300.0</td>\n",
       "      <td>10.068</td>\n",
       "      <td>75300</td>\n",
       "      <td>0.601594</td>\n",
       "      <td>10</td>\n",
       "      <td>2</td>\n",
       "      <td>45300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77535</th>\n",
       "      <td>15600.0</td>\n",
       "      <td>9.742</td>\n",
       "      <td>72300</td>\n",
       "      <td>0.585062</td>\n",
       "      <td>9</td>\n",
       "      <td>2</td>\n",
       "      <td>42300</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       loan_size  interest_rate  borrower_income  debt_to_income  \\\n",
       "77531    19100.0         11.261            86600        0.653580   \n",
       "77532    17700.0         10.662            80900        0.629172   \n",
       "77533    17600.0         10.595            80300        0.626401   \n",
       "77534    16300.0         10.068            75300        0.601594   \n",
       "77535    15600.0          9.742            72300        0.585062   \n",
       "\n",
       "       num_of_accounts  derogatory_marks  total_debt  \n",
       "77531               12                 2       56600  \n",
       "77532               11                 2       50900  \n",
       "77533               11                 2       50300  \n",
       "77534               10                 2       45300  \n",
       "77535                9                 2       42300  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Review the X variable DataFrame\n",
    "display(X.head())\n",
    "display(X.tail())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Check the balance of the labels variable (`y`) by using the `value_counts` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    75036\n",
       "1     2500\n",
       "Name: loan_status, dtype: int64"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check the balance of our target values\n",
    "y.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4: Split the data into training and testing datasets by using `train_test_split`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the train_test_learn module\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# Split the data using train_test_split\n",
    "# Assign a random_state of 1 to the function\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X, \n",
    "    y, \n",
    "    random_state = 1\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a Logistic Regression Model with the Original Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Step 1: Fit a logistic regression model by using the training data (`X_train` and `y_train`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(random_state=1)"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Import the LogisticRegression module from SKLearn\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "# Instantiate the Logistic Regression model\n",
    "# Assign a random_state parameter of 1 to the model\n",
    "LR_model = LogisticRegression(random_state = 1)\n",
    "\n",
    "# Fit the model using training data\n",
    "LR_model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Save the predictions on the testing data labels by using the testing feature data (`X_test`) and the fitted model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Make a prediction using the testing data\n",
    "LR_predictions = LR_model.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Evaluate the model’s performance by doing the following:\n",
    "\n",
    "* Calculate the accuracy score of the model.\n",
    "\n",
    "* Generate a confusion matrix.\n",
    "\n",
    "* Print the classification report."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9520479254722232"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print the balanced_accuracy score of the model\n",
    "balanced_accuracy_score(y_test, LR_predictions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Predicted Healthy Loans (low-risk)</th>\n",
       "      <th>Predicted Non-Healthy Loans (high-risk)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Actual Healthy Loans (low-risk)</th>\n",
       "      <td>18663</td>\n",
       "      <td>102</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Actual Non-Healthy Loans (high-risk)</th>\n",
       "      <td>56</td>\n",
       "      <td>563</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                      Predicted Healthy Loans (low-risk)  \\\n",
       "Actual Healthy Loans (low-risk)                                    18663   \n",
       "Actual Non-Healthy Loans (high-risk)                                  56   \n",
       "\n",
       "                                      Predicted Non-Healthy Loans (high-risk)  \n",
       "Actual Healthy Loans (low-risk)                                           102  \n",
       "Actual Non-Healthy Loans (high-risk)                                      563  "
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Generate a confusion matrix for the model\n",
    "'''\n",
    "-- Out of the 18,765 loan status's that are healthy, the model predicted 18,663 as \n",
    "   healthy correctly and 102 as healthy incorrectly \n",
    "\n",
    "-- Out of the 619 loan status's that are non-healthy (high-risk), the model \n",
    "   predicted 563 as non-healthy correctly and 56 as non-healthy incorrectly\n",
    "'''\n",
    "\n",
    "cm_imbalanced = confusion_matrix(y_test, LR_predictions)\n",
    "cm_imbalanced_df = pd.DataFrame(cm_imbalanced, \n",
    "                                index = ['Actual Healthy Loans (low-risk)', \n",
    "                                'Actual Non-Healthy Loans (high-risk)'], \n",
    "                                columns = ['Predicted Healthy Loans (low-risk)', 'Predicted Non-Healthy Loans (high-risk)']\n",
    "                              )\n",
    "cm_imbalanced_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                   pre       rec       spe        f1       geo       iba       sup\n",
      "\n",
      "          0       1.00      0.99      0.91      1.00      0.95      0.91     18765\n",
      "          1       0.85      0.91      0.99      0.88      0.95      0.90       619\n",
      "\n",
      "avg / total       0.99      0.99      0.91      0.99      0.95      0.91     19384\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Print the classification report for the model\n",
    "print(classification_report_imbalanced(y_test, LR_predictions))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4: Answer the following question."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question:** How well does the logistic regression model predict both the `0` (healthy loan) and `1` (high-risk loan) labels?\n",
    "\n",
    "**Answer:** \n",
    "* The model performs well according to the balanced accuracy score (95%), however this is due to the data being imbalanced. The number of healthy loans (low-risk) highly outweighs the number of non-healthy (high-risk) loans indicating that the model would predict loan status's as healthy better than being able to predict loan status's as non-healthy. \n",
    "\n",
    "* According to the imbalanced classification report, the model predicted healthy loans [`0`] 100% of the time and predicted non-healthy loans [`1`] 85% of the time.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Predict a Logistic Regression Model with Resampled Training Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: Use the `RandomOverSampler` module from the imbalanced-learn library to resample the data. Be sure to confirm that the labels have an equal number of data points. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the RandomOverSampler module form imbalanced-learn\n",
    "from imblearn.over_sampling import RandomOverSampler\n",
    "\n",
    "# Instantiate the random oversampler model\n",
    "# # Assign a random_state parameter of 1 to the model\n",
    "ROS_model = RandomOverSampler(random_state = 1)\n",
    "\n",
    "# Fit the original training data to the random_oversampler model\n",
    "X_oversampled, y_oversampled = ROS_model.fit_resample(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    56271\n",
       "1    56271\n",
       "Name: loan_status, dtype: int64"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Count the distinct values of the resampled labels data\n",
    "y_oversampled.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Use the `LogisticRegression` classifier and the resampled data to fit the model and make predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Instantiate the Logistic Regression model\n",
    "# Assign a random_state parameter of 1 to the model\n",
    "LR_oversampled_model = LogisticRegression(random_state = 1)\n",
    "\n",
    "# Fit the model using the resampled training data\n",
    "LR_oversampled_model.fit(X_oversampled, y_oversampled)\n",
    "\n",
    "# Make a prediction using the testing data\n",
    "LR_oversampled_pred = LR_oversampled_model.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Evaluate the model’s performance by doing the following:\n",
    "\n",
    "* Calculate the accuracy score of the model.\n",
    "\n",
    "* Generate a confusion matrix.\n",
    "\n",
    "* Print the classification report."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9936781215845847"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print the balanced_accuracy score of the model \n",
    "balanced_accuracy_score(y_test, LR_oversampled_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Predicted Healthy Loans (low-risk)</th>\n",
       "      <th>Predicted Non-Healthy Loans (high-risk)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Actual Healthy Loans (low-risk)</th>\n",
       "      <td>18649</td>\n",
       "      <td>116</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Actual Non-Healthy Loans (high-risk)</th>\n",
       "      <td>4</td>\n",
       "      <td>615</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                      Predicted Healthy Loans (low-risk)  \\\n",
       "Actual Healthy Loans (low-risk)                                    18649   \n",
       "Actual Non-Healthy Loans (high-risk)                                   4   \n",
       "\n",
       "                                      Predicted Non-Healthy Loans (high-risk)  \n",
       "Actual Healthy Loans (low-risk)                                           116  \n",
       "Actual Non-Healthy Loans (high-risk)                                      615  "
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Generate a confusion matrix for the model\n",
    "'''\n",
    "-- Out of the 18,765 loan status's that are healthy, the model predicted 18,649 as \n",
    "   healthy correctly and 116 as healthy incorrectly \n",
    "\n",
    "-- Out of the 619 loan status's that are non-healthy (high-risk), the model \n",
    "   predicted 615 as non-healthy correctly and 4 as non-healthy incorrectly\n",
    "'''\n",
    "\n",
    "cm_oversampled = confusion_matrix(y_test, LR_oversampled_pred)\n",
    "cm_oversampled_df = pd.DataFrame(cm_oversampled, \n",
    "                                index = ['Actual Healthy Loans (low-risk)', \n",
    "                                'Actual Non-Healthy Loans (high-risk)'], \n",
    "                                columns = ['Predicted Healthy Loans (low-risk)', 'Predicted Non-Healthy Loans (high-risk)']\n",
    "                              )\n",
    "cm_oversampled_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       1.00      0.99      1.00     18765\n",
      "           1       0.84      0.99      0.91       619\n",
      "\n",
      "    accuracy                           0.99     19384\n",
      "   macro avg       0.92      0.99      0.95     19384\n",
      "weighted avg       0.99      0.99      0.99     19384\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Print the classification report for the model\n",
    "print(classification_report(y_test, LR_oversampled_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4: Answer the following question"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question:** How well does the logistic regression model, fit with oversampled data, predict both the `0` (healthy loan) and `1` (high-risk loan) labels?\n",
    "\n",
    "**Answer:** \n",
    "\n",
    "* The oversampled model generated an accuracy score of 99% which turned out to be higher than the model fitted with imbalanced data. The oversampled model performs better because it does a exceptional job in catching mistakes such as labeling non-healthy (high-risk) loans as healthy (low-risk). This is analyzed based off of the recall score increasing from the imbalanced model to the oversampled model 0.91 --> 0.99\n",
    "\n",
    "* A lending company might want a model that requires a higher recall because: \n",
    "\n",
    "    * healthy loans being identified as a non-healthy loan might be more costly for a lending company since it might cause the loss of customers but it would not be as big of a deal since they have not provided any funds to the customer indicating no loss in terms of money\n",
    "  \n",
    "    * non-healthy loans being identified as a healthy loan might surely be more costly for a lending company due to the loss of funds being provided by the lender"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.7.13 ('dev')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "6a7ae136a53a24601e63a177d18d4029f938b6714f24b844ae865dcffb9f6766"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}