CC_vs_R2/cc_vs_r2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pearson Correlation Coefficient (CC)\n",
    "\n",
    "Pearson Correlation Coefficient is defined as $r = \\frac{cov(X, Y)}{\\sigma_X \\sigma_Y}$, \n",
    "which measures the linear correlation between two sets of data.\n",
    "\n",
    "When we have n samples of the data, we will use sample covariance to calculate the correlation\n",
    "coefficient:\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "r = \\frac{\\sum{(x_i - \\bar{x})(y_i - \\bar{y})}}{\\sqrt{\\sum{(x_i - \\bar{x})^2}\\sum{(y_i - \\bar{y})^2}}}\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "$r$ can take a value between -1 and 1. The higher the absolute value of the $r$ is, the stronger\n",
    "the correlation between x and y is. \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def pearson_correlation(x, y):\n",
    "    if x.shape != y.shape:\n",
    "        raise ValueError(\"x and y shape should be the same\")\n",
    "    \n",
    "    x_diffs = x - x.mean()\n",
    "    y_diffs = y - y.mean()\n",
    "    \n",
    "    if np.all(x_diffs == 0) or np.all(y_diffs == 0):\n",
    "        raise ValueError(\"x or y has constant value, cc is not defined\")\n",
    "    \n",
    "    r = (x_diffs * y_diffs).sum() / np.sqrt((x_diffs ** 2).sum() * (y_diffs ** 2).sum())\n",
    "    return r"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "np.float64(1.0)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = np.array([1, 2, 3, 4, 5])\n",
    "y = np.array([10, 20, 30, 40, 50])\n",
    "pearson_correlation(x, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "np.float64(0.006683847076446301)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = np.array([1, 2, 3, 4, 5])\n",
    "y = np.random.rand(5)\n",
    "pearson_correlation(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Coefficient of Determination (R2 score)\n",
    "\n",
    "Coefficient of determination is a measure of how much variation of a dependent variable (e.g. output of a model)\n",
    "is predictable from the independent variable (e.g. actual observed data).\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "R^2 = 1 - \\frac{\\sum{(y_i - \\hat{y}_i)^2}}{\\sum{(y_i - \\bar{y})^2}}\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "$\\sum{(y_i - \\hat{y}_i)^2} = \\sum{e_i ^ 2}$ is the residual sum of squares, $SS_{res}$, and \n",
    "$\\sum{(y_i - \\bar{y})^2}$ is the total sum of squares, $SS_{tot}$, so we can also write $R^2$\n",
    "as $1 - \\frac{SS_{res}}{SS_{tot}} = 1 - FVU$.\n",
    "\n",
    "where $FVU$ is Fraction of Variance Unexplained, which compares the unexplained variance (the model's errors) with the \n",
    "total variance of the data.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "def r2_score(y, y_hat):\n",
    "    if y.shape != y_hat.shape:\n",
    "        raise ValueError(\"x and y shape should be the same\")\n",
    "    \n",
    "    ss_res = ((y - y_hat)**2).sum()\n",
    "    ss_tot = ((y - y.mean())**2).sum()\n",
    "    \n",
    "    r2 = 1 - ss_res / ss_tot\n",
    "    return r2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "np.float64(1.0)"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = np.array([1, 2, 3, 4, 5])\n",
    "y = np.array([1, 2, 3, 4, 5])\n",
    "r2_score(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An R2 of 1 indicates perfect fit to the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "np.float64(0.974)"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = np.array([1, 2, 3, 4, 5])\n",
    "y = np.array([1.4, 2.3, 2.9, 4, 5])\n",
    "r2_score(x, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "np.float64(0.37764462899758466)"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = np.array([1, 2, 3, 4, 5])\n",
    "y = np.random.rand(5)\n",
    "pearson_correlation(x, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "np.float64(0.0)"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = np.array([1, 2, 3, 4, 5])\n",
    "y = np.array([3, 3, 3, 3, 3])\n",
    "r2_score(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An R2 of 0 indicates doing as good as the worst possible least-square predicotr, i.e. using the mean of the data as\n",
    "the prediction for all values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "np.float64(-444.5)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = np.array([1, 2, 3, 4, 5])\n",
    "y = np.array([10, 20, 30, 40, 50])\n",
    "r2_score(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Values of R2 outside the range of 0 to 1 means the model fits the data worse than just using the mean as the prediction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References\n",
    "\n",
    "- https://en.wikipedia.org/wiki/Pearson_correlation_coefficient\n",
    "- https://en.wikipedia.org/wiki/Coefficient_of_determination\n",
    "- https://machinelearningmastery.com/how-to-use-correlation-to-understand-the-relationship-between-variables/"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}