02_nonlinear_regression_part1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nonlinear Regression Part 1\n",
    "========================="
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"overview\">\n",
    "<p class=\"overview-title\">Overview</p>\n",
    "<p>Questions</p>\n",
    "    <ul>\n",
    "        <li>How can I analyze enzyme kinetics data in Python?</li>\n",
    "        <li>What is the process for non-linear least squares curve fitting in Python?</li>\n",
    "    </ul>\n",
    "<p>Objectives:</p>\n",
    "    <ul>\n",
    "        <li> Create a pandas dataframe with enzyme kinetics data from a .csv file</li>\n",
    "        <li> Add velocity calculations to the dataframe</li>\n",
    "        <li> Perform the non-linear regression calculations</li>\n",
    "    </ul>\n",
    "    \n",
    "<p>In this module, we will calculate initial rates from the raw data (&Delta;A<sub>405</sub>) in an enzyme kinetics experiment with alkaline phosphatase. In the process, we will import the raw data into a pandas dataframe, use some pandas tools to reorganize the data, produce a second pandas dataframe that contains the substrate concentrations and initial rates at each concentration. Finally, we will export this information to a csv file to use in the next module, where we will explore nonlinear curve fitting in python.\n",
    "    </p>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is pandas and why do we use it?\n",
    "\n",
    "*Please note: this section about pandas is included for those who have not completed the Working with Pandas notebook already.* Pandas is a python library that is designed to work with two dimensional data arrays. It is built on numpy, another python library that specializes in numerical analysis. Numpy also has the ability to create and analyze data arrays. If you are not familiar with arrays, here are some simple examples.\n",
    "\n",
    "### 1D arrays\n",
    "A one dimensional array is simply a list of items, for example, a list of the elements: H, He, Li, Be, B, etc.\n",
    "\n",
    "### 2D arrays\n",
    "A two dimensional array is an array which has rows and columns. It can have any number of columns and rows. A spreadsheet with rows and columns is analogous to a 2D array.\n",
    "\n",
    "### 3D arrays\n",
    "A three-dimensional array would be a collection of two dimensional arrays. For example, this might be a collection of x, y, and z coordinates for a structure as a function of time. \n",
    "\n",
    "The numpy library has functions that will manage n-dimensional arrays, while pandas only works on 2D arrays. You may need numpy for nD arrays at some point, but for this workshop, we will learn to use pandas as we learn how to perform linear and nonlinear regression of laboratory data based on data in 2D arrays.\n",
    "\n",
    "The pandas library contains powerful tools for working with 2D data arrays, including the ability to identify the rows and columns of data by unique identifiers: things like \"Substrate Concentration (mM)\" and \"initial velocity\" rather than \"row 1\" or \"column C\"). Pandas also has many more functions that we will not explore in this workshop, but here are three excellent free online resources for learning more about pandas:\n",
    "\n",
    "+ [Using pandas for data analysis](https://education.molssi.org/python-data-analysis/02-pandas/index.html) from MolSSI\n",
    "+ Charlie Weiss's excellent online textbook, *Scientific Computing for Chemists*, which you can find on his GitHub site, [SciCompforChemists](https://weisscharlesj.github.io/SciCompforChemists/intro.html)\n",
    "+ Corey Schafer's [Pandas Tutorials on YouTube](https://www.youtube.com/watch?v=ZyhVh-qRZPA&list=PL-osiE80TeTsWmV9i9c58mdDCSskIFdDS)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importing the Data\n",
    "We start by importing data from a csv file as we did earlier with the data for linear regression. These data represent the rate of p-nitrophenol appearance for a series of p-nitrophenol phosphate concentrations in the presence of alkaline phosphatase. We will import the libraries we need, import the data and set up a pandas dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating the pandas dataframe\n",
    "The filehandle, `datafile`, points to a csv file that contains the raw kinetics data. As we saw in the `Working with Pandas` module, the pandas library has a tool for creating a dataframe from an existing csv file. Notice that the variable for the dataframe is called AP_kin_df. The `_df` at the end of the variable name is a reminder that this is a pandas dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you look at this dataframe, notice that the index (the item at the far left of each row) is an integer. In this case, we want to use the 'Time (min)' values that are found in the first series as the index. There is a simple fix - the set_index function. \n",
    "\n",
    "```python\n",
    "AP_kin_df.set_index('Time (min)', inplace=True)\n",
    "```\n",
    "\n",
    "By making the time values the index for each row, we can easily omit them from our initial rate calculations. We use the `inplace=True` option to make the change to the dataframe permanent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Datatype\n",
    "Before we calculate the slopes to get initial velocities, we need to check for the datatypes on the numbers. We must ensure that the numbers are floats, rather than strings, so we can do calculations on them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Calculating the initial velocity\n",
    "The index for each row is the time, which will be the x values to get the slope of each line. The values in each column are the absorbance values at each time point, so those will be our y values. Now we need to follow these steps to calculate the initial velocity at each substrate concentration. \n",
    "1. Inspect the data.\n",
    "1. Create a second dataframe with the substrate concentration as the first series.\n",
    "1. Calculate the slopes from the first dataframe and add those as a column to the second dataframe.\n",
    "1. Calculate the initial velocity by dividing the slope by the extinction coefficient for p-nitrophenol under these buffer conditions, 15.0 mM<sup>-1</sup>cm<sup>-1</sup>.\n",
    "1. Export the second dataframe to a csv file that we will use in the next module."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inspect the data\n",
    "In an earlier module, we used pyplot from matplotlib to create a well-annotated plot of our linear regression data. We could do that here, but we only want to inspect the data to make sure we are on the right track. To do that we can use the [plot command from pandas](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.plot.html), which builds the plot using tools from matplotlib. The syntax is\n",
    "\n",
    "```python\n",
    "dataframe.plot()\n",
    "```\n",
    "In our case, the only argument we will pass is \"marker = 'o' so that the individual data points will appear."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 1. Inspect the data using the plot command that is available with the dataframe\n",
    "# We use the plot function that is built into pandas for this simple data display\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2. Create a second dataframe with the substrate concentration as the first series.\n",
    "# Note the syntax for this pandas function - the D and F are capitalized.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Calculate the Slopes\n",
    "\n",
    "There is some scatter in the data, but generally, the slopes of the curves increase with increasing substrate concentration. Now we need to calculate the slopes for each of the lines in the plot above. To do so, we can use the linregress function from scipy.stats that we used for the least squares linear regression analysis of the protein assay data from an earlier module. We will use the index from our data frame, `Time (min)`, AP_kin_df. The y values will be taken from each of the series from the same dataframe. \n",
    "\n",
    "Remember that linregress provides five outputs: slope, intercept, r-value, p-value and standard error. We need only the slope, so we will use this format\n",
    "\n",
    "```python\n",
    "slope, _, _, _, _ = stats.linregress(xdata, ydata)\n",
    "```\n",
    "\n",
    "where _ is just a placeholder that we will ignore.\n",
    "\n",
    "To get the slopes for each series we will use a for loop. First, we'll create an empty list to contain the slopes that are generated as the for loop cycles through the series in the AP_kin_df dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 3. Calculate the slopes from each column in the AP_kin_df dataframe \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Populate the new dataframe with the slopes\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Calculate the initial velocity\n",
    "The initial velocity can be calculated by dividing the slope by the micromolar extinction coefficient under the experimental conditions, 0.015 $\\mu$M$^{-1}$cm$^{-1}$. It is possible to complete the calculation and add it to the dataframe with a single line of code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate initial velocities and place those in a new column in the dataframe\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The final step before creating the csv file is to make pNPP (mM) the index for the dataframe.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use this dataframe now to perform the nonlinear regression fit using the SciPy library in part 2 of this lesson. To save this data for part 2, we need to write it to a csv file in our data directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check your understanding\n",
    "\n",
    "You will find an Excel file in your data folder, chymotrypsin_kinetics.xlsx, with some kinetic data from a chymotrypsin experiment. Apply the principles above to create dataframes and a .csv file for creating a Michaelis-Menten plot with these data. Under these assay conditions the extinction coefficient for p-nitrophenol is 18,320 M<sup>-1</sup>cm<sup>-1</sup>.</p>\n",
    "\n",
    "**Hint***\n",
    ":class: dropdown\n",
    "You will need to get the data into a layout and file format that is easily read by pandas. \n",
    "Delete the first seven lines of the Excel file.\n",
    "Delete the first column of the Excel file.\n",
    "Save the file as chymotrypsin_kinetics.csv.\n",
    "Your data will should look something like this: \n",
    "\n",
    "![csv image](images/csv_image.png \"csv image\")\n",
    "    \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}