linear_algebra.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4,)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "a = np.array([1,2,3,4])\n",
    "a.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The concept of rows and columns applies when you have a 2D array. However, the array numpy.array([1,2,3,4]) is a 1D array and so has only one dimension, therefore shape rightly returns a single valued iterable."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">For a 2D version of the same array, consider the following instead:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1, 4)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a=np.array([[1,2,3,4]])\n",
    "a\n",
    "a.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1]\n",
      " [2]\n",
      " [3]\n",
      " [4]]\n",
      "(4, 1)\n",
      "[1]\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "# We will represent vector as a column vector,that is having multiple rows and columns\n",
    "x=np.array([[1],[2],[3],[4]])\n",
    "print(x)\n",
    "print(x.shape)#(4,1)--> 4 rows, 1 column\n",
    "print(x[0])\n",
    "print(x[0][0])\n",
    "# print(x.reshape((-1,)))#\n",
    "# x=x.reshape((-1,))\n",
    "# print(x)\n",
    "# print(x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 2]\n",
      " [3 4]]\n",
      "[[1]\n",
      " [2]\n",
      " [3]\n",
      " [4]]\n"
     ]
    }
   ],
   "source": [
    "print(x.reshape(2,2))\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "x=np.array(np.arange(10))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So  -1 notation, what it does is the notation for unknown dimension, meaning: let numpy fill the missing dimension with the correct value so my array remain with the same number of items."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 1],\n",
       "       [2, 3],\n",
       "       [4, 5],\n",
       "       [6, 7],\n",
       "       [8, 9]])"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.reshape(5,-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tensors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![tensors](https://miro.medium.com/max/875/1*V83L4ydCdk21tXjP22VjXQ.jpeg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###### A tensor is a container which can house data in N dimensions, along with its linear operations\n",
    "* Often and erroneously used interchangeably with the matrix (which is specifically a 2-dimensional tensor), tensors are generalizations of matrices to N-dimensional space.\n",
    ">Mathematically speaking, tensors are more than simply a data container, however. Aside from holding numeric data, tensors also include descriptions of the valid linear transformations between tensors. Examples of such transformations, or relations, include the cross product and the dot product. From a computer science perspective, it can be helpful to think of tensors as being objects in an object-oriented sense, as opposed to simply being a data structure."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Tensors often offer more natural representations of data, e.g., consider video, which consists of obviously correlated images over time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2, 2, 3)\n"
     ]
    }
   ],
   "source": [
    "#Here T is a Tensor\n",
    "X=np.array([[[4,5,7],[10,11,13]],[[1,2,3],[7,8,9]]])\n",
    "print(X.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### numpy.transpose"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 5]\n",
      " [2 6]\n",
      " [3 7]\n",
      " [4 8]]\n"
     ]
    }
   ],
   "source": [
    "x=np.array([[1,2,3,4],[5,6,7,8]])\n",
    "xt=np.transpose(x)\n",
    "print(xt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualizing matrix multiplication as a linear combination \n",
    "When multiplying two matrices, there's a manual procedure we all know how to go through. Each result cell is computed separately as the dot-product of a row in the first matrix with a column in the second matrix. While it's the easiest way to compute the result manually, it may obscure a very interesting property of the operation: multiplying A by B is the linear combination of A's columns using coefficients from B. Another way to look at it is that it's a linear combination of the rows of B using coefficients from A.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Right-multiplication: combination of columns\n",
    "- Let's begin by looking at the right-multiplication of matrix X by a column vector:<br>\n",
    "![title](https://eli.thegreenplace.net/images/math/ba570f74b122c0a20e4488a052b25fbda160c138.png)\n",
    "\n",
    "<br>\n",
    "- Representing the columns of X by colorful boxes will help visualize this:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![title](https://eli.thegreenplace.net/images/2015/veccomb.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sticking the white box with a in it to a vector just means: multiply this vector by the scalar a. The result is another column vector - a linear combination of X's columns, with a, b, c as the coefficients."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Right-multiplying X by a matrix is more of the same. Each resulting column is a different linear combination of X's columns:<br>\n",
    "![title](https://eli.thegreenplace.net/images/math/d6065791babbc5b967c06b57322711424097c83c.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![title](https://eli.thegreenplace.net/images/2015/matcomb.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Hadamard product\n",
    "- __Hadamard Product__ is elementwise multiplication and it outputs a vector.\n",
    "- Values that correspond positionally are multiplied to produce a new matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 6, 12],\n",
       "       [10, 18]])"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = np.array(\n",
    "[[2,3],\n",
    " [2,3]])\n",
    "b = np.array(\n",
    "[[3,4],\n",
    " [5,6]])\n",
    "\n",
    "# Uses python's multiply operator\n",
    "a * b #(Hadamard Operator)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Norms\n",
    "###### Defining L¹ & L² Norms (and Beyond!)\n",
    "- The L¹ and L² norms are commonly used for assigning a magnitude to a vector. For a vector x having N components, the L¹ just adds up the components. Since we would like our magnitude to always be positive, we take the absolute value of the components.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![title](https://cdn-images-1.medium.com/max/1200/1*UY2EEP_f7DC0BJsoM2Eqjg.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- The L² norm takes the sum of the squared values, taking the square root at the end. The L² norm is the same as a standard distance formula, which finds the shortest path from A to B."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![title](https://cdn-images-1.medium.com/max/1200/1*ccWm-HBA85UnnFiWrFwvsg.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- if you start to see a pattern, then you might ask: Why stop at 2? We can create a norm for all 1 ≤ p < ∞ in this way too!\n",
    "![title](https://cdn-images-1.medium.com/max/1200/1*raXsqlpSn8RFLhDGWOKQnA.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "   - Norms are 0 if and only if the vector is a zero vector\n",
    "   - Norms respect the triangle inequity.\n",
    "   -  ∥k⋅u∥=|k|⋅∥u∥. The norm of a vector multiplied by a scalar is equal to the absolute value of thisscalar multiplied by the norm of the vector."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Max Norm__: the largest absolute value of the element in the vector, which can be written as:\n",
    "![title](https://cdn-images-1.medium.com/max/1000/1*x5iqiil6pZWETrQP3VBa3Q.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "m=np.array([-4,2,6])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7.483314773547883"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lp2=np.linalg.norm(m)\n",
    "lp2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "12.0"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lp1=np.linalg.norm(m,ord=1)\n",
    "lp1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.0"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lpinf=np.linalg.norm(m,ord=np.inf)\n",
    "lpinf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/fnorm.png\" alt=\"Drawing\" style=\"width: 310px;\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/matrix norms.png\" alt=\"Drawing\" style=\"width: 510px;\"/>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "q2=np.array([[2,-2,1],[-1,2,-1],[2,-4,1]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 2, -2,  1],\n",
       "       [-1,  2, -1],\n",
       "       [ 2, -4,  1]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "q2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.9287221489704836"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.linalg.norm(q2,ord=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.0"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.linalg.norm(q2,'fro')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.0"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.linalg.norm(q2)# it calculates frobenius norm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### In case of matrices,np.linalg.norm(__matrix__) by default gives Frobenius norm\t"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "L2 norm and Frobenius norm\tare different for matrices. Frobenius norm\tis similar to L2 norm of a vector\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1 -3  4]\n",
      " [-2 10 -7]\n",
      " [ 0  1  0]\n",
      " [ 0  0  0]]\n",
      "[[ 5  0]\n",
      " [-2 11]\n",
      " [-3 12]]\n"
     ]
    }
   ],
   "source": [
    "A=np.array([[1,-3,4],[-2,10,-7],[0,1,0],[0,0,0]])\n",
    "B=np.array([[5,0],[-2,11],[-3,12]])\n",
    "print(A)\n",
    "print(B)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1, 15],\n",
       "       [-9, 26],\n",
       "       [-2, 11],\n",
       "       [ 0,  0]])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A.dot(B)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}