week1/week1-learn-pandas.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "db115dbb",
   "metadata": {},
   "source": [
    "# Machine learning zoomcamp\n",
    "\n",
    "* https://github.com/alexeygrigorev/mlbookcamp-code/tree/master/course-zoomcamp"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb584611",
   "metadata": {},
   "source": [
    "## Pandas\n",
    "\n",
    "* Video: https://www.youtube.com/watch?v=0j3XK5PsnxA\n",
    "* Notebook from Video: https://github.com/alexeygrigorev/mlbookcamp-code/blob/master/course-zoomcamp/01-intro/notebooks/09-pandas.ipynb\n",
    "* Notebook for exercise: https://github.com/alexeygrigorev/mlbookcamp-code/blob/master/appendix-d-pandas.ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b94227ee",
   "metadata": {},
   "source": [
    "## My Exercise and Notes\n",
    "\n",
    "Pandas is python library for data analysis and manipulation. Pandas data is in tabular format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "d0df16f7",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f813c7e1",
   "metadata": {},
   "source": [
    "Sample data for analysis and manipulation. Defined as list of lists."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "ef593d8c",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = [\n",
    "    ['Nissan', 'Stanza', 1991, 138, 4, 'MANUAL', 'sedan', 2000],\n",
    "    ['Hyundai', 'Sonata', 2017, None, 4, 'AUTOMATIC', 'Sedan', 27150],\n",
    "    ['Lotus', 'Elise', 2010, 218, 4, 'MANUAL', 'convertible', 54990],\n",
    "    ['GMC', 'Acadia',  2017, 194, 4, 'AUTOMATIC', '4dr SUV', 34450],\n",
    "    ['Nissan', 'Frontier', 2017, 261, 6, 'MANUAL', 'Pickup', 32340],\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f43ba94f",
   "metadata": {},
   "source": [
    "### DataFrames\n",
    "\n",
    "Load data into a dataframe. Pandas dataframe is a table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "1f011175",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>Sedan</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>218.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>194.0</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>4dr SUV</td>\n",
       "      <td>34450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>Pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         0         1     2      3  4          5            6      7\n",
       "0   Nissan    Stanza  1991  138.0  4     MANUAL        sedan   2000\n",
       "1  Hyundai    Sonata  2017    NaN  4  AUTOMATIC        Sedan  27150\n",
       "2    Lotus     Elise  2010  218.0  4     MANUAL  convertible  54990\n",
       "3      GMC    Acadia  2017  194.0  4  AUTOMATIC      4dr SUV  34450\n",
       "4   Nissan  Frontier  2017  261.0  6     MANUAL       Pickup  32340"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39aff428",
   "metadata": {},
   "source": [
    "Let us define some meaningful names to the columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "819e2c6c",
   "metadata": {},
   "outputs": [],
   "source": [
    "column_names = [\n",
    "    'Make', 'Model', 'Year', 'Engine HP', 'Engine Cylinders',\n",
    "    'Transmission Type', 'Vehicle_Style', 'MSRP'\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "4c22d237",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>Sedan</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>218.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>194.0</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>4dr SUV</td>\n",
       "      <td>34450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>Pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Make     Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "0   Nissan    Stanza  1991      138.0                 4            MANUAL   \n",
       "1  Hyundai    Sonata  2017        NaN                 4         AUTOMATIC   \n",
       "2    Lotus     Elise  2010      218.0                 4            MANUAL   \n",
       "3      GMC    Acadia  2017      194.0                 4         AUTOMATIC   \n",
       "4   Nissan  Frontier  2017      261.0                 6            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "0         sedan   2000  \n",
       "1         Sedan  27150  \n",
       "2   convertible  54990  \n",
       "3       4dr SUV  34450  \n",
       "4        Pickup  32340  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(data, columns=column_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "c2b978dc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>Sedan</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>218.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>194.0</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>4dr SUV</td>\n",
       "      <td>34450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>Pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Make     Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "0   Nissan    Stanza  1991      138.0                 4            MANUAL   \n",
       "1  Hyundai    Sonata  2017        NaN                 4         AUTOMATIC   \n",
       "2    Lotus     Elise  2010      218.0                 4            MANUAL   \n",
       "3      GMC    Acadia  2017      194.0                 4         AUTOMATIC   \n",
       "4   Nissan  Frontier  2017      261.0                 6            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "0         sedan   2000  \n",
       "1         Sedan  27150  \n",
       "2   convertible  54990  \n",
       "3       4dr SUV  34450  \n",
       "4        Pickup  32340  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(data, columns=column_names)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "446cd18f",
   "metadata": {},
   "source": [
    "Alternatively, we can use a list of dictionaries to create a dataframe. Here we have a dictionary for every row of the data with column name as the key and the column value as the value of the dictionary. When using such a format, there is no need to separately define columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "390ec888",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = [\n",
    "    {\n",
    "        \"Make\": \"Nissan\",\n",
    "        \"Model\": \"Stanza\",\n",
    "        \"Year\": 1991,\n",
    "        \"Engine HP\": 138.0,\n",
    "        \"Engine Cylinders\": 4,\n",
    "        \"Transmission Type\": \"MANUAL\",\n",
    "        \"Vehicle_Style\": \"sedan\",\n",
    "        \"MSRP\": 2000\n",
    "    },\n",
    "    {\n",
    "        \"Make\": \"Hyundai\",\n",
    "        \"Model\": \"Sonata\",\n",
    "        \"Year\": 2017,\n",
    "        \"Engine HP\": None,\n",
    "        \"Engine Cylinders\": 4,\n",
    "        \"Transmission Type\": \"AUTOMATIC\",\n",
    "        \"Vehicle_Style\": \"Sedan\",\n",
    "        \"MSRP\": 27150\n",
    "    },\n",
    "    {\n",
    "        \"Make\": \"Lotus\",\n",
    "        \"Model\": \"Elise\",\n",
    "        \"Year\": 2010,\n",
    "        \"Engine HP\": 218.0,\n",
    "        \"Engine Cylinders\": 4,\n",
    "        \"Transmission Type\": \"MANUAL\",\n",
    "        \"Vehicle_Style\": \"convertible\",\n",
    "        \"MSRP\": 54990\n",
    "    },\n",
    "    {\n",
    "        \"Make\": \"GMC\",\n",
    "        \"Model\": \"Acadia\",\n",
    "        \"Year\": 2017,\n",
    "        \"Engine HP\": 194.0,\n",
    "        \"Engine Cylinders\": 4,\n",
    "        \"Transmission Type\": \"AUTOMATIC\",\n",
    "        \"Vehicle_Style\": \"4dr SUV\",\n",
    "        \"MSRP\": 34450\n",
    "    },\n",
    "    {\n",
    "        \"Make\": \"Nissan\",\n",
    "        \"Model\": \"Frontier\",\n",
    "        \"Year\": 2017,\n",
    "        \"Engine HP\": 261.0,\n",
    "        \"Engine Cylinders\": 6,\n",
    "        \"Transmission Type\": \"MANUAL\",\n",
    "        \"Vehicle_Style\": \"Pickup\",\n",
    "        \"MSRP\": 32340\n",
    "    }\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "12976d51",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>Sedan</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>218.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>194.0</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>4dr SUV</td>\n",
       "      <td>34450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>Pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Make     Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "0   Nissan    Stanza  1991      138.0                 4            MANUAL   \n",
       "1  Hyundai    Sonata  2017        NaN                 4         AUTOMATIC   \n",
       "2    Lotus     Elise  2010      218.0                 4            MANUAL   \n",
       "3      GMC    Acadia  2017      194.0                 4         AUTOMATIC   \n",
       "4   Nissan  Frontier  2017      261.0                 6            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "0         sedan   2000  \n",
       "1         Sedan  27150  \n",
       "2   convertible  54990  \n",
       "3       4dr SUV  34450  \n",
       "4        Pickup  32340  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(data)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdcf51d2",
   "metadata": {},
   "source": [
    "Display the first few entries of the dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "04baec6e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>Sedan</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>218.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>194.0</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>4dr SUV</td>\n",
       "      <td>34450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>Pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Make     Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "0   Nissan    Stanza  1991      138.0                 4            MANUAL   \n",
       "1  Hyundai    Sonata  2017        NaN                 4         AUTOMATIC   \n",
       "2    Lotus     Elise  2010      218.0                 4            MANUAL   \n",
       "3      GMC    Acadia  2017      194.0                 4         AUTOMATIC   \n",
       "4   Nissan  Frontier  2017      261.0                 6            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "0         sedan   2000  \n",
       "1         Sedan  27150  \n",
       "2   convertible  54990  \n",
       "3       4dr SUV  34450  \n",
       "4        Pickup  32340  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea796147",
   "metadata": {},
   "source": [
    "Display first *n* entries of the dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "34b82c25",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>Sedan</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Make   Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "0   Nissan  Stanza  1991      138.0                 4            MANUAL   \n",
       "1  Hyundai  Sonata  2017        NaN                 4         AUTOMATIC   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "0         sedan   2000  \n",
       "1         Sedan  27150  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head(2)  # or df.head(n=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf86f031",
   "metadata": {},
   "source": [
    "## Series\n",
    "\n",
    "A table consists of columns. In Pandas dataframe is a table and columns of the table are series. Thus dataframe may consist of multiple series.\n",
    "\n",
    "To access it, use dot or brackets. When column names have spaces, use the brackets notation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "27ef5f14",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     Nissan\n",
       "1    Hyundai\n",
       "2      Lotus\n",
       "3        GMC\n",
       "4     Nissan\n",
       "Name: Make, dtype: object"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.Make"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "f1ab9801",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     Nissan\n",
       "1    Hyundai\n",
       "2      Lotus\n",
       "3        GMC\n",
       "4     Nissan\n",
       "Name: Make, dtype: object"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Make']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "59371259",
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (1897567212.py, line 1)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m\"/tmp/ipykernel_2080/1897567212.py\"\u001b[0;36m, line \u001b[0;32m1\u001b[0m\n\u001b[0;31m    df.Engine HP\u001b[0m\n\u001b[0m              ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    "df.Engine HP"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "8361bbbc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    138.0\n",
       "1      NaN\n",
       "2    218.0\n",
       "3    194.0\n",
       "4    261.0\n",
       "Name: Engine HP, dtype: float64"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Engine HP']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "e7b5c57d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    138.0\n",
       "1      NaN\n",
       "2    218.0\n",
       "3    194.0\n",
       "4    261.0\n",
       "Name: Engine HP, dtype: float64"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "col_name = 'Engine HP'\n",
    "df[col_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "634fb4f4",
   "metadata": {},
   "source": [
    "Use a list to select a subset of the columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "bb0ae89b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>34450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Make     Model  Year   MSRP\n",
       "0   Nissan    Stanza  1991   2000\n",
       "1  Hyundai    Sonata  2017  27150\n",
       "2    Lotus     Elise  2010  54990\n",
       "3      GMC    Acadia  2017  34450\n",
       "4   Nissan  Frontier  2017  32340"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "columns = ['Make', 'Model', 'Year', 'MSRP']\n",
    "df[columns]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4b565a0",
   "metadata": {},
   "source": [
    "Add more columns to dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "0ec585de",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "      <th>id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>Sedan</td>\n",
       "      <td>27150</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>218.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>194.0</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>4dr SUV</td>\n",
       "      <td>34450</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>Pickup</td>\n",
       "      <td>32340</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Make     Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "0   Nissan    Stanza  1991      138.0                 4            MANUAL   \n",
       "1  Hyundai    Sonata  2017        NaN                 4         AUTOMATIC   \n",
       "2    Lotus     Elise  2010      218.0                 4            MANUAL   \n",
       "3      GMC    Acadia  2017      194.0                 4         AUTOMATIC   \n",
       "4   Nissan  Frontier  2017      261.0                 6            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  id  \n",
       "0         sedan   2000   1  \n",
       "1         Sedan  27150   2  \n",
       "2   convertible  54990   3  \n",
       "3       4dr SUV  34450   4  \n",
       "4        Pickup  32340   5  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['id'] = [1, 2, 3, 4, 5]\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc839702",
   "metadata": {},
   "source": [
    "Delete a column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "1cf03d5c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>Sedan</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>218.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>194.0</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>4dr SUV</td>\n",
       "      <td>34450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>Pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Make     Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "0   Nissan    Stanza  1991      138.0                 4            MANUAL   \n",
       "1  Hyundai    Sonata  2017        NaN                 4         AUTOMATIC   \n",
       "2    Lotus     Elise  2010      218.0                 4            MANUAL   \n",
       "3      GMC    Acadia  2017      194.0                 4         AUTOMATIC   \n",
       "4   Nissan  Frontier  2017      261.0                 6            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "0         sedan   2000  \n",
       "1         Sedan  27150  \n",
       "2   convertible  54990  \n",
       "3       4dr SUV  34450  \n",
       "4        Pickup  32340  "
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "del df['id']\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e659c1e",
   "metadata": {},
   "source": [
    "## Index\n",
    "\n",
    "Rows of a dataframe are called indices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "125a62fe",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RangeIndex(start=0, stop=5, step=1)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.index"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00683f3b",
   "metadata": {},
   "source": [
    "We can define different ids to the indices (instead of the default 0, 1, 2 ...)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "ff905256",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>Sedan</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>218.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>194.0</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>4dr SUV</td>\n",
       "      <td>34450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>e</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>Pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Make     Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "a   Nissan    Stanza  1991      138.0                 4            MANUAL   \n",
       "b  Hyundai    Sonata  2017        NaN                 4         AUTOMATIC   \n",
       "c    Lotus     Elise  2010      218.0                 4            MANUAL   \n",
       "d      GMC    Acadia  2017      194.0                 4         AUTOMATIC   \n",
       "e   Nissan  Frontier  2017      261.0                 6            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "a         sedan   2000  \n",
       "b         Sedan  27150  \n",
       "c   convertible  54990  \n",
       "d       4dr SUV  34450  \n",
       "e        Pickup  32340  "
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.index = ['a', 'b', 'c', 'd', 'e']\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79f660fd",
   "metadata": {},
   "source": [
    "## Accessing elements\n",
    "\n",
    "To get data in a particular row, access the data using index id and df.loc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "0286a629",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Make                 Nissan\n",
       "Model                Stanza\n",
       "Year                   1991\n",
       "Engine HP             138.0\n",
       "Engine Cylinders          4\n",
       "Transmission Type    MANUAL\n",
       "Vehicle_Style         sedan\n",
       "MSRP                   2000\n",
       "Name: a, dtype: object"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.loc['a']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b7a4666",
   "metadata": {},
   "source": [
    "You can also access the data using positional index and df.iloc "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "06c9805a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Make                 Nissan\n",
       "Model                Stanza\n",
       "Year                   1991\n",
       "Engine HP             138.0\n",
       "Engine Cylinders          4\n",
       "Transmission Type    MANUAL\n",
       "Vehicle_Style         sedan\n",
       "MSRP                   2000\n",
       "Name: a, dtype: object"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.iloc[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22563a2c",
   "metadata": {},
   "source": [
    "Access multiple rows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "ad28d16c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>Sedan</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>218.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Make   Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "b  Hyundai  Sonata  2017        NaN                 4         AUTOMATIC   \n",
       "c    Lotus   Elise  2010      218.0                 4            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "b         Sedan  27150  \n",
       "c   convertible  54990  "
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.loc[['b', 'c']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "6df41617",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>Sedan</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>218.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Make   Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "b  Hyundai  Sonata  2017        NaN                 4         AUTOMATIC   \n",
       "c    Lotus   Elise  2010      218.0                 4            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "b         Sedan  27150  \n",
       "c   convertible  54990  "
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.iloc[[1,2]]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4430bdeb",
   "metadata": {},
   "source": [
    "To reset the index to the default of numerical 0, 1, ... use df.reset_index. This will add a numerical index and maintain the existing index as a new column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "62a35935",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>a</td>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>b</td>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>Sedan</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>c</td>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>218.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>d</td>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>194.0</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>4dr SUV</td>\n",
       "      <td>34450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>e</td>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>Pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  index     Make     Model  Year  Engine HP  Engine Cylinders  \\\n",
       "0     a   Nissan    Stanza  1991      138.0                 4   \n",
       "1     b  Hyundai    Sonata  2017        NaN                 4   \n",
       "2     c    Lotus     Elise  2010      218.0                 4   \n",
       "3     d      GMC    Acadia  2017      194.0                 4   \n",
       "4     e   Nissan  Frontier  2017      261.0                 6   \n",
       "\n",
       "  Transmission Type Vehicle_Style   MSRP  \n",
       "0            MANUAL         sedan   2000  \n",
       "1         AUTOMATIC         Sedan  27150  \n",
       "2            MANUAL   convertible  54990  \n",
       "3         AUTOMATIC       4dr SUV  34450  \n",
       "4            MANUAL        Pickup  32340  "
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.reset_index()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3efdb83f",
   "metadata": {},
   "source": [
    "If you want to not have the previous index saved as a new columns use drop=True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "43aa87fa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>Sedan</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>218.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>194.0</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>4dr SUV</td>\n",
       "      <td>34450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>Pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Make     Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "0   Nissan    Stanza  1991      138.0                 4            MANUAL   \n",
       "1  Hyundai    Sonata  2017        NaN                 4         AUTOMATIC   \n",
       "2    Lotus     Elise  2010      218.0                 4            MANUAL   \n",
       "3      GMC    Acadia  2017      194.0                 4         AUTOMATIC   \n",
       "4   Nissan  Frontier  2017      261.0                 6            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "0         sedan   2000  \n",
       "1         Sedan  27150  \n",
       "2   convertible  54990  \n",
       "3       4dr SUV  34450  \n",
       "4        Pickup  32340  "
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "3829564b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>Sedan</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>218.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>d</th>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>194.0</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>4dr SUV</td>\n",
       "      <td>34450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>e</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>Pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Make     Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "a   Nissan    Stanza  1991      138.0                 4            MANUAL   \n",
       "b  Hyundai    Sonata  2017        NaN                 4         AUTOMATIC   \n",
       "c    Lotus     Elise  2010      218.0                 4            MANUAL   \n",
       "d      GMC    Acadia  2017      194.0                 4         AUTOMATIC   \n",
       "e   Nissan  Frontier  2017      261.0                 6            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "a         sedan   2000  \n",
       "b         Sedan  27150  \n",
       "c   convertible  54990  \n",
       "d       4dr SUV  34450  \n",
       "e        Pickup  32340  "
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd38da59",
   "metadata": {},
   "source": [
    "Note above, the original dataframe is still the same. To save the new dataframe, assign it to the original datadframe itself"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "4a10cad3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>Sedan</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>218.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>194.0</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>4dr SUV</td>\n",
       "      <td>34450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>Pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Make     Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "0   Nissan    Stanza  1991      138.0                 4            MANUAL   \n",
       "1  Hyundai    Sonata  2017        NaN                 4         AUTOMATIC   \n",
       "2    Lotus     Elise  2010      218.0                 4            MANUAL   \n",
       "3      GMC    Acadia  2017      194.0                 4         AUTOMATIC   \n",
       "4   Nissan  Frontier  2017      261.0                 6            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "0         sedan   2000  \n",
       "1         Sedan  27150  \n",
       "2   convertible  54990  \n",
       "3       4dr SUV  34450  \n",
       "4        Pickup  32340  "
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = df.reset_index(drop=True)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fd0caee",
   "metadata": {},
   "source": [
    "## Element wise operations\n",
    "\n",
    "Under the hood for operations, pandas uses numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "07e9ef6f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     2000\n",
       "1    27150\n",
       "2    54990\n",
       "3    34450\n",
       "4    32340\n",
       "Name: MSRP, dtype: int64"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['MSRP']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b4fd655",
   "metadata": {},
   "source": [
    "Perform operations on a single series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "e0143035",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     2.00\n",
       "1    27.15\n",
       "2    54.99\n",
       "3    34.45\n",
       "4    32.34\n",
       "Name: MSRP, dtype: float64"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['MSRP'] / 1000"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b30cf18f",
   "metadata": {},
   "source": [
    "Or perform on multiple series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "a1dd5cd1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.38</td>\n",
       "      <td>20.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NaN</td>\n",
       "      <td>271.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2.18</td>\n",
       "      <td>549.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.94</td>\n",
       "      <td>344.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2.61</td>\n",
       "      <td>323.4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Engine HP   MSRP\n",
       "0       1.38   20.0\n",
       "1        NaN  271.5\n",
       "2       2.18  549.9\n",
       "3       1.94  344.5\n",
       "4       2.61  323.4"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['Engine HP', 'MSRP']] / 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "dcf08188",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1991\n",
       "1    2017\n",
       "2    2010\n",
       "3    2017\n",
       "4    2017\n",
       "Name: Year, dtype: int64"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Year']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0c1fe3b",
   "metadata": {},
   "source": [
    "Logical operations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "7548f1f6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1     True\n",
       "2    False\n",
       "3     True\n",
       "4     True\n",
       "Name: Year, dtype: bool"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Year'] >= 2015"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e4f8f4e",
   "metadata": {},
   "source": [
    "## Filtering"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "671ad976",
   "metadata": {},
   "source": [
    "To look at all records where Year is greater than or equal to 2015 as an example, we can apply filter as follows. It wil display all record where condition is True.\n",
    "\n",
    "dataframe[condition]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "9f09f012",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>Sedan</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>194.0</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>4dr SUV</td>\n",
       "      <td>34450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>Pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Make     Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "1  Hyundai    Sonata  2017        NaN                 4         AUTOMATIC   \n",
       "3      GMC    Acadia  2017      194.0                 4         AUTOMATIC   \n",
       "4   Nissan  Frontier  2017      261.0                 6            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "1         Sedan  27150  \n",
       "3       4dr SUV  34450  \n",
       "4        Pickup  32340  "
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df['Year'] >= 2015]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "ce16a8bd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>Pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Make     Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "0  Nissan    Stanza  1991      138.0                 4            MANUAL   \n",
       "4  Nissan  Frontier  2017      261.0                 6            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "0         sedan   2000  \n",
       "4        Pickup  32340  "
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df [\n",
    "    df['Make'] == 'Nissan'\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "d1b41475",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>Pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Make     Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "4  Nissan  Frontier  2017      261.0                 6            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "4        Pickup  32340  "
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df [\n",
    "    (df['Year'] >= 2015) & (df['Make'] == 'Nissan')\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27e4de0b",
   "metadata": {},
   "source": [
    "## String operations\n",
    "\n",
    "Unlike Numpy which can only do numeric operations, pandas can also do string operations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "67080754",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0          sedan\n",
       "1          Sedan\n",
       "2    convertible\n",
       "3        4dr SUV\n",
       "4         Pickup\n",
       "Name: Vehicle_Style, dtype: object"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Vehicle_Style']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca13ccde",
   "metadata": {},
   "source": [
    "Using the str method of pandas allows us to use any of the standard string methods like lower (to convert all upper case to lower case), replace (to replace a character in the string with another character) and so on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "16baa034",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0          sedan\n",
       "1          sedan\n",
       "2    convertible\n",
       "3        4dr suv\n",
       "4         pickup\n",
       "Name: Vehicle_Style, dtype: object"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Vehicle_Style'].str.lower()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "5fc247b3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0          sedan\n",
       "1          Sedan\n",
       "2    convertible\n",
       "3        4dr_SUV\n",
       "4         Pickup\n",
       "Name: Vehicle_Style, dtype: object"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Vehicle_Style'].str.replace(' ', '_')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "158423ff",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0          sedan\n",
       "1          sedan\n",
       "2    convertible\n",
       "3        4dr_suv\n",
       "4         pickup\n",
       "Name: Vehicle_Style, dtype: object"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Vehicle_Style'].str.lower().str.replace(' ', '_')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "98bda94c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>sedan</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>218.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>194.0</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>4dr_suv</td>\n",
       "      <td>34450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Make     Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "0   Nissan    Stanza  1991      138.0                 4            MANUAL   \n",
       "1  Hyundai    Sonata  2017        NaN                 4         AUTOMATIC   \n",
       "2    Lotus     Elise  2010      218.0                 4            MANUAL   \n",
       "3      GMC    Acadia  2017      194.0                 4         AUTOMATIC   \n",
       "4   Nissan  Frontier  2017      261.0                 6            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "0         sedan   2000  \n",
       "1         sedan  27150  \n",
       "2   convertible  54990  \n",
       "3       4dr_suv  34450  \n",
       "4        pickup  32340  "
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Vehicle_Style'] = df['Vehicle_Style'].str.lower().str.replace(' ', '_')\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6cfa548",
   "metadata": {},
   "source": [
    "## Sumarizing operations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "902afe3d",
   "metadata": {},
   "source": [
    "Operations on columns with numerical values. These are termed as numerical columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "4bf1f77c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2000\n",
      "54990\n",
      "30186.0\n"
     ]
    }
   ],
   "source": [
    "print(df['MSRP'].min())\n",
    "print(df['MSRP'].max())\n",
    "print(df['MSRP'].mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "aa889acc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count        5.000000\n",
       "mean     30186.000000\n",
       "std      18985.044904\n",
       "min       2000.000000\n",
       "25%      27150.000000\n",
       "50%      32340.000000\n",
       "75%      34450.000000\n",
       "max      54990.000000\n",
       "Name: MSRP, dtype: float64"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['MSRP'].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "df53a76e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>5.000000</td>\n",
       "      <td>4.00000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>5.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>2010.400000</td>\n",
       "      <td>202.75000</td>\n",
       "      <td>4.400000</td>\n",
       "      <td>30186.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>11.260551</td>\n",
       "      <td>51.29896</td>\n",
       "      <td>0.894427</td>\n",
       "      <td>18985.044904</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1991.000000</td>\n",
       "      <td>138.00000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>2000.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>2010.000000</td>\n",
       "      <td>180.00000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>27150.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>2017.000000</td>\n",
       "      <td>206.00000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>32340.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>2017.000000</td>\n",
       "      <td>228.75000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>34450.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>2017.000000</td>\n",
       "      <td>261.00000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>54990.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              Year  Engine HP  Engine Cylinders          MSRP\n",
       "count     5.000000    4.00000          5.000000      5.000000\n",
       "mean   2010.400000  202.75000          4.400000  30186.000000\n",
       "std      11.260551   51.29896          0.894427  18985.044904\n",
       "min    1991.000000  138.00000          4.000000   2000.000000\n",
       "25%    2010.000000  180.00000          4.000000  27150.000000\n",
       "50%    2017.000000  206.00000          4.000000  32340.000000\n",
       "75%    2017.000000  228.75000          4.000000  34450.000000\n",
       "max    2017.000000  261.00000          6.000000  54990.000000"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "6f4c2f75",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>5.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>2010.0</td>\n",
       "      <td>203.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>30186.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>11.0</td>\n",
       "      <td>51.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>18985.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1991.0</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>2010.0</td>\n",
       "      <td>180.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>27150.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>2017.0</td>\n",
       "      <td>206.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>32340.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>2017.0</td>\n",
       "      <td>229.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>34450.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>2017.0</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>54990.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Year  Engine HP  Engine Cylinders     MSRP\n",
       "count     5.0        4.0               5.0      5.0\n",
       "mean   2010.0      203.0               4.0  30186.0\n",
       "std      11.0       51.0               1.0  18985.0\n",
       "min    1991.0      138.0               4.0   2000.0\n",
       "25%    2010.0      180.0               4.0  27150.0\n",
       "50%    2017.0      206.0               4.0  32340.0\n",
       "75%    2017.0      229.0               4.0  34450.0\n",
       "max    2017.0      261.0               6.0  54990.0"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe().round()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "689543b6",
   "metadata": {},
   "source": [
    "Operations on columns with string values. These are termed as Categorical columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "b51df23e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     Nissan\n",
       "1    Hyundai\n",
       "2      Lotus\n",
       "3        GMC\n",
       "4     Nissan\n",
       "Name: Make, dtype: object"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Make']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "589af59d",
   "metadata": {},
   "source": [
    "Find number of unique values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "id": "69e537fc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Make'].nunique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "b95c1bcb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Make                 4\n",
       "Model                5\n",
       "Year                 3\n",
       "Engine HP            4\n",
       "Engine Cylinders     2\n",
       "Transmission Type    2\n",
       "Vehicle_Style        4\n",
       "MSRP                 5\n",
       "dtype: int64"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.nunique()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07b5e520",
   "metadata": {},
   "source": [
    "## Missing values\n",
    "\n",
    "Typically when working with machine learning, we do not want any missing values in the data. And if there are any, we need to process/transform them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "id": "3aa0b9f7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    138.0\n",
       "1      NaN\n",
       "2    218.0\n",
       "3    194.0\n",
       "4    261.0\n",
       "Name: Engine HP, dtype: float64"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Engine HP']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "id": "00669afb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1     True\n",
       "2    False\n",
       "3    False\n",
       "4    False\n",
       "Name: Engine HP, dtype: bool"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Engine HP'].isnull()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "id": "81759bd1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Make  Model   Year  Engine HP  Engine Cylinders  Transmission Type  \\\n",
       "0  False  False  False      False             False              False   \n",
       "1  False  False  False       True             False              False   \n",
       "2  False  False  False      False             False              False   \n",
       "3  False  False  False      False             False              False   \n",
       "4  False  False  False      False             False              False   \n",
       "\n",
       "   Vehicle_Style   MSRP  \n",
       "0          False  False  \n",
       "1          False  False  \n",
       "2          False  False  \n",
       "3          False  False  \n",
       "4          False  False  "
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "id": "e8b0cdad",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Make                 0\n",
       "Model                0\n",
       "Year                 0\n",
       "Engine HP            1\n",
       "Engine Cylinders     0\n",
       "Transmission Type    0\n",
       "Vehicle_Style        0\n",
       "MSRP                 0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95cae800",
   "metadata": {},
   "source": [
    "Fill missing values with 0. Although in ML, usually it is not preferred to fil with zero, since this will cause the data to change such that the behavior is different (e.g. now the mean is different)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "id": "8e0c1856",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    138.0\n",
       "1      0.0\n",
       "2    218.0\n",
       "3    194.0\n",
       "4    261.0\n",
       "Name: Engine HP, dtype: float64"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Engine HP'].fillna(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "913f6624",
   "metadata": {},
   "source": [
    "Typically in ML, we fill missing values with the mean value, so that the overall mean of the data is still the same. But it depends on case to case basis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "0c03cf17",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    138.00\n",
       "1    202.75\n",
       "2    218.00\n",
       "3    194.00\n",
       "4    261.00\n",
       "Name: Engine HP, dtype: float64"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Engine HP'].fillna(df['Engine HP'].mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2640be4",
   "metadata": {},
   "source": [
    "You can drop the data having missing values. Again, this is a conscious decision to be taken considering the effect of dropping data like this - whether there is enough data to afford chuking out data like this, whether removing data in this way affects the distribution of the data [example whether it will cause certain samples of data to be more than certain other samples] and so on."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0c10905",
   "metadata": {},
   "source": [
    "Drop entire row where there is any missing values. This can be done by usin axis=0 (which is the default behavior). In Pandas axis=0 means rows and axis=1 means columns (where as in Numpy its other way round - so remember this)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "id": "50de4423",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>218.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>194.0</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>4dr_suv</td>\n",
       "      <td>34450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Make     Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "0  Nissan    Stanza  1991      138.0                 4            MANUAL   \n",
       "2   Lotus     Elise  2010      218.0                 4            MANUAL   \n",
       "3     GMC    Acadia  2017      194.0                 4         AUTOMATIC   \n",
       "4  Nissan  Frontier  2017      261.0                 6            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "0         sedan   2000  \n",
       "2   convertible  54990  \n",
       "3       4dr_suv  34450  \n",
       "4        pickup  32340  "
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.dropna(axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "id": "cbf5677a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>sedan</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>4dr_suv</td>\n",
       "      <td>34450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Make     Model  Year  Engine Cylinders Transmission Type Vehicle_Style  \\\n",
       "0   Nissan    Stanza  1991                 4            MANUAL         sedan   \n",
       "1  Hyundai    Sonata  2017                 4         AUTOMATIC         sedan   \n",
       "2    Lotus     Elise  2010                 4            MANUAL   convertible   \n",
       "3      GMC    Acadia  2017                 4         AUTOMATIC       4dr_suv   \n",
       "4   Nissan  Frontier  2017                 6            MANUAL        pickup   \n",
       "\n",
       "    MSRP  \n",
       "0   2000  \n",
       "1  27150  \n",
       "2  54990  \n",
       "3  34450  \n",
       "4  32340  "
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.dropna(axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b160957f",
   "metadata": {},
   "source": [
    "To remove rows/columns having missing values when the entire row/column has missing values or even if there is a single missing value. Control this using how='all' or how='any'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "id": "86115555",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hyundai</td>\n",
       "      <td>Sonata</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>sedan</td>\n",
       "      <td>27150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>218.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>194.0</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>4dr_suv</td>\n",
       "      <td>34450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Make     Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "0   Nissan    Stanza  1991      138.0                 4            MANUAL   \n",
       "1  Hyundai    Sonata  2017        NaN                 4         AUTOMATIC   \n",
       "2    Lotus     Elise  2010      218.0                 4            MANUAL   \n",
       "3      GMC    Acadia  2017      194.0                 4         AUTOMATIC   \n",
       "4   Nissan  Frontier  2017      261.0                 6            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "0         sedan   2000  \n",
       "1         sedan  27150  \n",
       "2   convertible  54990  \n",
       "3       4dr_suv  34450  \n",
       "4        pickup  32340  "
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.dropna(how='all')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "id": "28cb45e8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Make</th>\n",
       "      <th>Model</th>\n",
       "      <th>Year</th>\n",
       "      <th>Engine HP</th>\n",
       "      <th>Engine Cylinders</th>\n",
       "      <th>Transmission Type</th>\n",
       "      <th>Vehicle_Style</th>\n",
       "      <th>MSRP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Stanza</td>\n",
       "      <td>1991</td>\n",
       "      <td>138.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>sedan</td>\n",
       "      <td>2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lotus</td>\n",
       "      <td>Elise</td>\n",
       "      <td>2010</td>\n",
       "      <td>218.0</td>\n",
       "      <td>4</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>convertible</td>\n",
       "      <td>54990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GMC</td>\n",
       "      <td>Acadia</td>\n",
       "      <td>2017</td>\n",
       "      <td>194.0</td>\n",
       "      <td>4</td>\n",
       "      <td>AUTOMATIC</td>\n",
       "      <td>4dr_suv</td>\n",
       "      <td>34450</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Nissan</td>\n",
       "      <td>Frontier</td>\n",
       "      <td>2017</td>\n",
       "      <td>261.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MANUAL</td>\n",
       "      <td>pickup</td>\n",
       "      <td>32340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Make     Model  Year  Engine HP  Engine Cylinders Transmission Type  \\\n",
       "0  Nissan    Stanza  1991      138.0                 4            MANUAL   \n",
       "2   Lotus     Elise  2010      218.0                 4            MANUAL   \n",
       "3     GMC    Acadia  2017      194.0                 4         AUTOMATIC   \n",
       "4  Nissan  Frontier  2017      261.0                 6            MANUAL   \n",
       "\n",
       "  Vehicle_Style   MSRP  \n",
       "0         sedan   2000  \n",
       "2   convertible  54990  \n",
       "3       4dr_suv  34450  \n",
       "4        pickup  32340  "
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.dropna(how='any')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "768b8fc5",
   "metadata": {},
   "source": [
    "## Grouping"
   ]
  },
  {
   "cell_type": "raw",
   "id": "b82a5123",
   "metadata": {},
   "source": [
    "e.g. you want to group cars by their transmission_type and then want to see the average MSRP (price) for every group. An SQL query on a database would look something like this:\n",
    "\n",
    "SELECT\n",
    "    tranmission_type,\n",
    "    AVG(msrp)\n",
    "FROM\n",
    "    cars\n",
    "GROUP BY\n",
    "    transmission_type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "id": "538a610d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Transmission Type\n",
       "AUTOMATIC    30800.000000\n",
       "MANUAL       29776.666667\n",
       "Name: MSRP, dtype: float64"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('Transmission Type').MSRP.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7cef9da",
   "metadata": {},
   "source": [
    "## Get the underlying Numpy array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "id": "6c2f0222",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     2000\n",
       "1    27150\n",
       "2    54990\n",
       "3    34450\n",
       "4    32340\n",
       "Name: MSRP, dtype: int64"
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['MSRP']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "id": "64730709",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 2000, 27150, 54990, 34450, 32340])"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['MSRP'].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "id": "cda75ffd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 7.60140233, 10.20916916, 10.91492481, 10.4472933 , 10.38409105])"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.log1p(df['MSRP'].values)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a94dc491",
   "metadata": {},
   "source": [
    "Get the list of dictionaries representation of the datafram data [Remember we started with this to populate the dataframe, we can convert back to this format]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "id": "d7fcb407",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'Make': 'Nissan',\n",
       "  'Model': 'Stanza',\n",
       "  'Year': 1991,\n",
       "  'Engine HP': 138.0,\n",
       "  'Engine Cylinders': 4,\n",
       "  'Transmission Type': 'MANUAL',\n",
       "  'Vehicle_Style': 'sedan',\n",
       "  'MSRP': 2000},\n",
       " {'Make': 'Hyundai',\n",
       "  'Model': 'Sonata',\n",
       "  'Year': 2017,\n",
       "  'Engine HP': nan,\n",
       "  'Engine Cylinders': 4,\n",
       "  'Transmission Type': 'AUTOMATIC',\n",
       "  'Vehicle_Style': 'sedan',\n",
       "  'MSRP': 27150},\n",
       " {'Make': 'Lotus',\n",
       "  'Model': 'Elise',\n",
       "  'Year': 2010,\n",
       "  'Engine HP': 218.0,\n",
       "  'Engine Cylinders': 4,\n",
       "  'Transmission Type': 'MANUAL',\n",
       "  'Vehicle_Style': 'convertible',\n",
       "  'MSRP': 54990},\n",
       " {'Make': 'GMC',\n",
       "  'Model': 'Acadia',\n",
       "  'Year': 2017,\n",
       "  'Engine HP': 194.0,\n",
       "  'Engine Cylinders': 4,\n",
       "  'Transmission Type': 'AUTOMATIC',\n",
       "  'Vehicle_Style': '4dr_suv',\n",
       "  'MSRP': 34450},\n",
       " {'Make': 'Nissan',\n",
       "  'Model': 'Frontier',\n",
       "  'Year': 2017,\n",
       "  'Engine HP': 261.0,\n",
       "  'Engine Cylinders': 6,\n",
       "  'Transmission Type': 'MANUAL',\n",
       "  'Vehicle_Style': 'pickup',\n",
       "  'MSRP': 32340}]"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.to_dict(orient='records')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9cbfec95",
   "metadata": {},
   "source": [
    "To add\n",
    "\n",
    "From https://github.com/alexeygrigorev/mlbookcamp-code/blob/master/appendix-d-pandas.ipynb\n",
    "* Splitting data\n",
    "* dtypes\n",
    "* sorting and reordering\n",
    "* agg"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}