Chapter8Reduction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chapter 8 Dimension Reduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8.1 Overview \n",
    " \n",
    " We visit principal component analysis in this chapter as an example of dimension reduction methods. The idea of capture the major sources of variances, i.e., principal components, is a general approach to many statistical procedures. Note that we discuss dimension reduction under the unsupervised learning scenario, meaning that there are no responses in consideration. The principal components characterize the variability in the given data set with relatively small number of variables. When an outcome (response) is specified and we want to find small number of variables to maximize the explanability of the outcome, the corresponding method is known as [sufficient dimension reduction](https://en.wikipedia.org/wiki/Sufficient_dimension_reduction). \n",
    " \n",
    "Recall in Chapter 7, we discuss clustering methods that put observations into distinct clusters. If we think of the $K$ cluster centers (median, mean, etc.) as the representatives of the $n$ observations, we reduce the sample size $n$ to a small amount $K$. If instead, we run clustering methods on the **features** instead of observations. We will find $K$ \"features\" as representative features. When $K < p$, we have reduced the dimenion of the data set. \n",
    "\n",
    "Dimension reduction methods find a low-dimensional representation of the observations that contain a good fraction of information in the original data. The primary task for dimension reduction is to reduce the dimension, whereas the cluster analysis aims to find homogeneous subgroups. It is possible to use cluster analyses for dimension reduction, for instance, treat the cluster centers as representative features.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Typical  methods of dimension reduction includes \n",
    "- Principal Component Analysis (PCA)\n",
    "- Multi-Dimensional Scaling (MDS)\n",
    "- Kernel PCA, Sparse PCA, Manifold Learning, ...\n",
    "\n",
    "These can generally be thought of as soft clustering or as vector space embedding. We only discuss the principal component analysis in this chapter. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8.2 Principal Component Analysis\n",
    "\n",
    "\t"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "Suppose that we have a data set in an $n \\times p$ matrix $X$. We find the number of features $p$ too large for our purpose, e.g., to visualize the data set in a two-dimensional plot. As a result, we would like to find the best two-dimensional representation of the data. \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAAMFBMVEUAAABNTU1oaGh8fHyM\njIyampqnp6eysrK9vb3Hx8fQ0NDZ2dnh4eHp6enw8PD////QFLu4AAAACXBIWXMAABJ0AAAS\ndAHeZh94AAAb/klEQVR4nO3d2ULiSACG0QqbiILv/7Yj4IIdB2PyZ/Wci2mbkUoJfA0kKSwv\nQGdl7AnAEggJAoQEAUKCACFBgJAgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQI\nCQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJAgQEgQICQIEBIECAkChAQBQoIA\nIUGAkCBASBAgJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJAgQEgQ\nICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJ\nAoQEAUKCACFBgJAgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFCggAh\nQYCQIEBIECAkCBASBAwQUoGZafEoz4czwiYgSUgQICQIEBIECAkChAQBQoIAIUGAkCBASBAg\nJAgQEgQICQKEBAFCggAhQYCQIEBIECCkWWi1lJkBCWkGWn8qAIMR0gwIafqENH0dPqiGoQhp\n+oQ0A0KaPiHNgJBmQEfTJ6QZENL0CWkWZDR1QoIAIUGAkCBASBAgJAgQEgQICQKEBAFCggAh\nQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJAgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAg\nJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQ0vlL+1s+7SEIaW7kaexp0I6SxCWkR\nhDSyUpS0BEIamZCWQUgjE9IyCGlsOloEIY1NSIsgpPHJaAGEBAFCggAhQYCQIEBIECAkCBAS\nBAgJAoQEAUKCACFBgJAgQEgQICQIEFKSBRF/lpByLNH7w4SUI6Q/TEgxPsbkLxNSjJD+MiHF\nCOkvE1KOjv4wIeUI6Q8TUpKM/iwhQYCQIEBIECAkCBASBAgJAgYN6elhcznSstk99bUJGMWA\nIZ1W5dO6l03ASAYMaVeqx+fLV8dDVXZ9bAJGMmBIVXn++Pq5VH1sAkYyYEhfTp+5fy6NkJgZ\nz0gQMOx7pMPx8pX3SCzNkLu/1zd77VanXjYB4xj2ONLuchyp2jw4jsSyOLNhpix9mhYhzZLF\nuFMjpFkS0tSMFZLjSF34wKLJmU5I5VZiEwvmdpocL+3mSEiTI6RZ0tHUCGmWhDQ1o4T04yPA\nI+RHMpoWIUHAoMsoGu+YExIzM2BIT5WQWKohX9qdNmV9WUfhpR1LM+x7pMdSHl+ExPIMvLPh\nuC6bk5BYnMH32j2U6iAklmb43d/Pq58PgQiJmRnjONJWSCyNU4QgQEgQICQIEBIECAkChAQB\nQhqahUSLJKRhWdq6UEIalpAWSkiD8vE/SyWkQQlpqYQ0KCEtlZCGpaOFEtKwhLRQQhqajBZJ\nSBAgJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJA6GG5pkUVM/Wtw\nG9/5FiG1NtxiV8tq+9fgNr77LUJqTUhLIqSxDPeBQD56qH8NbuP73yKktoS0JEIajZCWREjj\n8R5pSbxHGo2QlkRIIxruwS2j/jW4je98i5AgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAg\nJAgQEgQICQKEBAFCggAhddBqBYulRYskpNZaram02HWhhNSakPgkpLZafe6MDwRaKiG1JSRu\nCKktIXFDSK15j8QnIbUmJD4JqQPHkXgnJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJ\nAoQEAUKCACFBgJAg4C+E1GoFkIVEEzXRu2H5IbVak2pp60RN9m4QUsMrTfYe/FsmezcsPqRW\nn9vj438marp3g5CaXWm69+CfMt27QUjNrjTde/BPme7dsPiQvEdalMneDUJqeKXJ3oN/y2Tv\nhuWH5DjSskz0bvgLIUHvhAQBQoIAIUGAkCBASBAwfEj7VSmbQ6+bgKENGNJ1///6ekRt18sm\nYCRDh7Qru9PLy3FX9n1sAkYydEhVOZ2/PpVVH5uAkQwd0vv5HT/87tWWm4CRDB3S9j2kqo9N\nwEgGDWnzsD+Ux9cvT7v7exuExMwMGtLHKfClVKc+NgEjGfI40vPzfr/ZXHY57O52JCTmxpkN\njTVYodRurcxEV9jwG0JqqMGa2XarNye75pPfEFJDQuKesUKa23GkBp8r1O4Tbqb7uTj8xnRC\nKrcSm4gSEnd5adeMkLhLSA15j8Q9QmpISNwzQkj7qqzuLqKYZEiOI3HPoGc2bEq1f3m4/AO8\n7mcTMI4BQ3q+FLQr29PLcWNhH4syYEjb8xnfu+v6CQv7WJbBP7OhbG7+kt4EjGTwkB6vr+ks\n7GNRBn1pt31fPHHaWtjHogwY0qn6eD1X7j8hCYm5GfQ40u49n+r+x9oJiblxZgMECAkChAQB\nQoIAIUGAkCBASBDQNaTzrw3Lr0zLhtRgck3mH1qPNODqo1Y/uNVRrXQM6eHjIwcmG1KD6TX5\nCUIrZAdcD9vqB7det6WOIVX31xW1JaQAIQ2pY0g93eTBURs8YzZ5Uq1/T+2SdsP0ptUPPuD8\nFqZjSLty/9PwWxJSd0IaVNedDZv1U2oq/7eJjkMJ6RffIqS2OoRUvhp5Vv8/1s/Ta/IT1L+n\ndkm7YXrT6gfXUUtCavYtQuKuv3BAtsEDo8ljp/49tUvaDdObVj+4jFr5CyFB71K7v6v7a8c7\nbAJmIBTScbrvkWAAHUI6fNnXcPcDHweYFYypyzPS6raj6OEkITEziz9FCIZgrx0ELP+ALAxA\nSBDQ+aTV6vD636dqG5rPN5uA6eu8jOL58ufz/Q/F77IJmIHUXjsv7fjTOoZUfTwjOUWIv6zz\nS7vqfCT2UJWH1Iz+3QTMQNedDeu3fXab1ITqm4Dp63xA9nFzzugQms63m5iIP7J0J/RDLfK2\nucOZDQ39kcWkoR9qkbfNXUJqSEgjDDMjnc5s+HJ2w8iz6ln9x+zlBx9b6Ida5G1zn5CaEdII\nw8yJl3bNCGmEYeakQ0jVZv/Uy+esTjEk75HGGGZGup79XW0eDsfojG43MSFCGmGYGekQ0ulp\nv31bbb7ZPT6PPKv+1R8Yi3yohH6oRd42d3R+j/S8367/wM4GuCuzs+GwFhJ/WiCkp4eVZyT+\nuI4hHffnU+3K6iH7y12ExMx02dlw2FXn/XbbQ3wnuJCYmY67vzf7/L7vFyExO51C2vV0PFZI\nzI1nJAjo9B5p6z0SXNhrBwGOI0GAMxsgwLl2EODsbwiY+3qkJk+FofUPU18Y0Nv8erv9ape0\nuzcnYd4rZJusHwutyJv6UrXe5tfb7Ve7pN29ORHz/swGIX0Q0rhmHVIpP9+u9e9pcq1WmxpT\nb/Pr7farXdLu3pwKIeU2NSYhjUxIuU2NSUgjm3VI3iN96m1+vd1+tUva3ZsTIaTgpsYkpHHN\nOyTHkW70Nr/ebr/aJe3uzUnoekD208izgjEJCQLm/tIOJkFIEJAK6Sn625iFxMx0DWnnPRJ0\nDumzo+jvNRcSM9MxpKo8vqzL8bgu0U8/ERIz0zGk8yu6h9dno+eyjk3pRUjMTiCkQ9lfv8gR\nEjPTMaTN60u7Y1m9PAmJP61jSIdzQJdPEdrGpvQiJGan6+7vh/PftqXsQvP5ZhMwfc5sgAAh\nQUDnkB7Pn6K/jR6OTYfU4LSL0JKbZept9daSbuOuIa3fTmyInmoXDanBGUyhRaDL1Nt64mXd\nxp1PEarOT0aH6nwsKUdI0yGkRjqfInT9zO/nssrMp76JrkP9fFJt/Vt6ORN3nlrdFE2utLDb\nOHBmw9cvIoQ0GUJqpvNLu/dnpKmuRxJSN0JqpvMB2ct7pKcqes6q90gT0uqmaHKlZd3GnV/a\nfTHirP5/LCF1IqRGlh+S40hdtbopmlxpSbexMxsgQEgQ0Dmkw+b8/LzJ/vpLITEzkVOEXi+r\noiUJiZnpGNK+rE/nkPYW9vGndT5F6HQ9qWGyZzbAEAKnCDUP6elhc9lLvtn98OFdQmJmOoa0\nentGanLS6ml1c8Tp/pkQQmJmMu+RGi2j2JXq8Xpm3vH1++9+yIOQmJmue+02jZ5hLt6XXJw9\nlyo8KxhT5DhS2Tw2ud636y9Ss4IxDXhmg2cklmvAkM7L0q+Hbb1HYmm6hHTaXb58WpWq0Sc2\nrG/22q1O4VnBmLqEVF3e6Bya7mx4TW532TdRbR4cR2JZOoR03vX9+kdVPb+c1qXJ7obOs+pt\n2VCTgXvbVIPvabe6p9UwTTQYOHT7jXqP/2q89ldZl/M7nqfycPnvAL8fqbeFrE0G7m1TDb6n\n3XrTVsM00WDg0O036j3+yxHbX+W61d31d/UNca6dkH41sJCGHLhzSKty85dfDPL740il/PvD\n1i9ppcnAvW2qwfe0ulK7YZpoMHDo9hv1Hv/tkO2vsjq/tDte10+c7h8X+maQ2obLre+vMuLN\n2tumGnxPqysJadiBO4S0O+9s2F5/nfkQ65FGvVl721SD72l1JSENO3CHkE7VecOXnQz7cnPW\nQoD3SF++p9WVvEcadOAOIb2c3n9TXxnmN/YJ6VcDC2nIgbuE9HnJ5ocDrF/sq7L64UyI/5tV\n/SdNPDAaDtzbphp8T6srtRumiQYDh26/Ue/xX403yFWunjfnc4keLuVb2MeiDBjS86WgXdme\nXo6b+wsBhcTMDBjS9vxGanfdT366vzRdSMzMgCFdX4K+/f6XFgdkYboGD+nx+prOwj4WZdCX\ndtv3RUinrYV9LMqAIZ2qj9dz5YczioTEzAy5+/tl955P9cPxWyExM4OGNKVNQJKQIEBIECAk\nCBASBAgJAoQEAXMPKbXCZlpCC4lCy4YabHzUu2ESj4F5h5Ra8zktoaWtoYWsDTY+6t0wkceA\nkKZHSL8xkcfArEMqZSK3YlTtp2r3YzYYpt0lP29pSFN5DAhpcoT0G1N5DAhpcoT0G1N5DMw6\npKm8Pg6r/VTtfswGw7S75OctDWkijwEhTY+QfmMij4F5hzSRYwhxtZ+q3Y/ZYJh2l/y8pSFN\n4jEw95BgEoQEAUKCACFBgJAgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQ0hS1WkjU\nZtz+Vig12XhvRpifkKan1dLWNuP2t2a2ycZ7M8r8hDQ9QupGSENuYrpK+edOrV3Q7qNzmgzT\n7pJWG+/NOPMT0uQIqRshDbqJ6RJSN0IadBMTVrtL6/dxq3u9yTDtLmm18d6MMj8hTY+QuhHS\nkJuYtNo9Wr+LW93pTYZpd0mrjfdmhPkJCQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKC\nACFBgJAgQEgQICQIEFJSaslNg3GabKq3NUvUCCkntQi0wThNNtXbKlq+IaQcIf1hQoopJfOY\nazBOk021mk79SqmfaumEFCOkv0xIMUL6y4SUk3rENRinyaZaTad+JR01IqQcIf1hQkpKPeAa\njNNkU62mU7+SjBoQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJAgQEgQ\nIKTxtVp9FFprxIduN46QxtZqPWxo9Ssfut44QhqbkCZBSDPX4FN66t/S4EqttvR3db5xhDQy\nIU2CkOZOSJMgpNlrcAfWv6XVva6jO7reOEIam5AmQUjz1+D+q39LqztdRnd0u3GEBAFCggAh\nQYCQIEBIECAkCBg+pP2qlM2h103A0AYM6bqbfn098LXrZRMwkqFD2pXd6eXluCv7PjYBIxk6\npKqczl+fyqqPTcBIhg7p/TSMH04ua7kJGMnQIW3fQ6r62ASMZNCQNg/7Q3l8/fK0u7+3QUjM\nzKAhfZypXkp16mMTMJIhjyM9P+/3m81ll8PubkdCYm6c2fB/U2iwOiW0vKfBYqMBFxL19kMt\nm5C+n0CD9ZKhBacNlr8OuLS1tx9q6YT0/QSENIVhZmSskKZ9HOl9v8i9eTT5nlabql0S2lK7\n2Yw5zJxMJ6RyK7GJDoQ0iWHmxEu7b7cvpCkMMydC+n4CDR4HocdKfZjaJQM+Knv7oZZOSN9P\nQEhTGGZGhgzptC1l/bakb9o7G14cR5rIMLMxYEin6vKv1OY6yNRDgt8YMKTLYr7TvlpfBhES\nSzJgSNX1isdqdRQSCzP0eqRXp/VaSCzMgCGtyvsp36u1kFiWAUPal+3bV8eyFhKLMuTu791H\nPYcfdo4KiZkZ9IDs8+b9q+NWSCyJMxsgQEgQICQIEBIECAkChAQBQoIAIf3fFGa3AogxCen7\nCcxuTSrjEtL3ExASvyKkb7c/u8/tYWRC+nb7QuJ3hPTt9oXE7wjp+wl4j8SvCOn7CQiJXxHS\n/03BcSR+QUgQICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFCgoDlhdTbooT5\nDdzKtGYzG0sLqbdlcvMbuJVpzWZGhLTYgVuZ1mxmZGEh9fZRIvMbuJVpzWZOhLTUgVuZ1mzm\nREhLHbiVac1mThYW0gzfykzrkTut2cyIkBY7cCvTms2MLC2kOR7umdYDd1qzmY3lhQQjEBIE\nCAkChAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFCggAhQYCQIEBIC9ZgaZHVRyFCWqwGi12t\nh40R0mIJaUhCWqoGHwjkM4NyhLRUQhqUkJZKSIMS0mJ5jzQkIS2WkIYkpAVzHGk4QoIAIUGA\nkCBASBAgJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJAgQEgQICQI\nEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQMGtLTw+by+3g2u6e+NgGjGDCk06p8WveyCRjJ\ngCHtSvX4fPnqeKjKro9NwEgGDKkqzx9fP5eqj03ASAYM6csvWfzhd5u23ASMxDMSBAz7Hulw\nvHzlPRJLM+Tu7/XNXrvVqZdNwDiGPY60uxxHqjYPjiOxLM5saKzc30HCnyakht5eko49DSZK\nSA0JiXvGCmlux5E+9pKMPRGmaTohlVuJTURNd2ZMgpd2zQiJu4TUkI64R0gNCYl7hgzptC1l\nfXgbZGY7G14cR+KeIRf2VdflsddB5hcS/L9BT1rdv9a0ry6LY4XEogy6jOLyx7FaHYXEwoyw\nsO+0XguJhRkwpFV5XzqxWguJZRkwpH3Zvn11LGshsShD7v7efdRz+GFXspCYmUEPyD5v3r86\nboXEkjizAQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEARMNCWamxaM8H05Ub/MzcO8Dz27C\nXQYWkoGFFBhYSAYWUmBgIRlYSIGBhWRgIQUGFpKBhRQYWEgGFlJgYCEZWEiBgYVkYCEFBhaS\ngYUUGFhIBhZSYGAhGVhIgYGnHhLMgpAgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQ\nEgQICQKEBAFCgoCphvR0O7HWn2z+j/o4u6pUu1N+4NSMX563pWyPNxeEZlwbODPjbz6FPjPh\n+sCpm/hUm2CrGU80pFN1M7Hn0G1WH2d9+fsqPnBqxi+HyzDV570amnFt4NCM3x/u1ccloQnX\nBk7dxMfqOu7nvyntZjzRkDblS0ibyKC1cZ5K9fzyXJWn9MCpGb9UrxM8bcru/e+pGdcGjs34\n7PA5v9iE/x04NeHt5TbYle37BS1nPM2QHr/8U7MvD5FRa+PsyuGysa7D1wZOzfjxci+fPv8d\nTs24NnBqxmen6vMxnppwbeDUhN8eaJ+Pt5YznmRIx7L+GtI+MmxtnE05P6F3/7etNnBqxtvy\n/PWC1IxrA6dmfLYpp5uvMxOuDZya8Nt7iM9/UlrOeJIhrcvxNqRNOWxf3/51HrY2Tu1fo9TA\nqRmvystDVbafD5/UjGsDp2b8cn4M9nAT1wZOTfjh7aXdxxNQyxlPMaSH8vjyNaSLdddxa+Pk\nQvpn4NSMS9l8feuemnFt4NSMX74+b0RD+jJwbML7896G6vPpbTkhXZ5Vv+5Lfjzvpez8XF4b\nJ/ewrA2cmvF5n8C28z+XTQbOzPh8921v/hYM6d+BQxN+uAT5+Y5oOSGtzjtl6z/HKbDT959x\nkv9cfjPB7jMul7cyx/yMawNfJW7j65v1zw3d/hEc+Kr7hPfnl3av/6R0/dd1eiFtL7fXNz9H\n6PF+M04VDak+TuDx/s84qRn/36MlcFNU5Zu/JW7i6rsxAu8Wz68Xb4JsOePphfRxDLuHO/mf\nca47aI6pQyj5GW/+vVNTM64N/CZx1PvL3HI38ff70fL/pLSc8RxCqi7/aHS/M2rjPFye/A6l\n676f2sCpGV8nePx8S52acW3g1Iz/3S2dmnBt4NyD4vzfmyNqLWc8vZCuvpytdXkZ+91L5N+p\njZM67F4bODXj1zcxp/ML+Mf3C1Izrg2cmvHrP+hfjlDlzmz4Z+Dgg+L0NtrVos5s+PqMe7qe\nDtX5H7Xbca7jrzK7UGsDp2b8tktp/TFwasa1gWMzXn3so85O+N+BYxNeZ27iOYR0OUF3FTiQ\nfTPO58CJo5DfDpyY8cth/T7B7Iy/HTgx48+XEuEJfzdw5Cb+nGCXGU81JJgVIUGAkCBASBAg\nJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJAgQEgQICQIEBIECAkC\nhAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFCggAhTcBptypl3ey3z938ct1f/Ebvwy+/n99y\n247v7dehlur08/e2C2lVfvf9/Jrbdnzbsj6+vBzXjX6zcKuQipD65rYdX7n+vu5Towe6kKbJ\nbTu+rw/w/apU+7eLd++/XvuwKW9f/k9IN9c6bkr1cLlwV70+yb1+1+WF4/uAD73+LH+WkMa3\nK9vjx182l0f9+uX8uH94//Lh+iZq9/J/Id1e6/KO65zL+vzF9jaky7c126nB7whpAl4f8avd\n0+XLQ1mfXk7rcrgk8fzyXJXH85ev/3m8hPNtSF+u9frlvqzOF16vXz5f2r3/L+KENAWH7Xmn\n3Xkf9ebyfulUNufH/fmCw/nLq/8P6cu1nt7+1+bt+jchPb14p9QTt+pEPD1U58d5effxiL/+\ncTw8rP8/pNq1/rn+7c4GIfXCrToZz+cXXf8T0vr9QiFNlFt1dB+P7JtH/83l5z+2ZbU/HO+F\n9M+FQhqaW3V0m7f9aKdSvb+xubi+pTmU7duD/05IX671/t9v3iN9vRZBbtXRPZWyP73+sT4H\n9Xje1fayv+42uO51O1yber7zHunLtd7/15e9dscXIfXKrTq+3dsbnPNhoLd3Q9Xxsrf6/OXm\n5jue/gnp453T7bWu/+vjwvOXq3J+thNSj9yqE/C8fX3eWD9e/7J/fdRv355ANq/vjS4Xbl//\n/9Ph8xnn4iakL9f6+O+uer3W+cunlZB65ladrNAj/vpER8+ENFldQ7qcDnHaNDqnnK6ENFld\nQ3o7Qa/KzIb7hDRZnV/a7S/n8EXmwk+EBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFB\ngJAgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFCggAhQcB/ccSfgT3t\nKigAAAAASUVORK5CYII=",
      "text/plain": [
       "plot without title"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 420,
       "width": 420
      }
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAAMFBMVEUAAABNTU1oaGh8fHyM\njIyampqnp6eysrK9vb3Hx8fQ0NDZ2dnh4eHp6enw8PD////QFLu4AAAACXBIWXMAABJ0AAAS\ndAHeZh94AAAZA0lEQVR4nO3dDVeq2gKGUVAzM7X//29Pah/UYUvKywJ0zjHuvVwTlxnPNoFF\n1RvQWzX2E4B7ICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFCggAhQYCQIEBI\nECAkCBASBAgJAoQEAUKCACFBgJAgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQI\nCQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJAgQEgQICQIEBIECAkChAQBQoIA\nIUGAkCBASBAgJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJAgQEgQ\nICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJ\nAoQEAUKCACFBgJAgQEgQUCCkCmbmhq08H84IQ0CSkCBASBAgJAgQEgQICQKEBAFCggAhQUD5\nkDaLqlptBx0CSisY0vksiuX5hIr1IEPASEqHtK7Wh7e3/braDDEEjKR0SHV1OC4fqsUQQ8BI\nSof0eZbs5bNlhcTMlA7p6TOkeoghYCRFQ1o9b7bVy/viYX15b4OQmJmiIX3NgKqq+jDEEDCS\nkseRdrvNZrU67XJYX+xISMyNMxuYi8753I073DT3uw8hMQ+dl0Zo3OHmyyjcTkjMg5DaH8Rx\nJK7RebWexh16XNnn9udXZJWWB/nfo/S8thH3TUi3EBK/COkWQuI3n5FuICR+E9KXw1NVLT+m\n9NnZwLU606ge4zjSoT79M7E6P4iQuCcFQzpN5jts6uXpQYTEPSkYUn1ecV8v9kLizhS/ZsP7\nm9JyKSTuTMGQFtXnKd+LpZC4LwVD2lRPH0v7aikk7krJ3d/rr3q2HXsnhcTMFD0gu1t9Lu2f\nhMQ9cWYDBAgJAoQEAUKCACFBgJAgQEgQICQKuW2K0Fwu4CEkirht0up8roUjJIoQUmKVCQ5B\nUbdd2GdGl2cTEiUIKbLKBIegKCFFVpngEJTlM1JilQkOQVlCSqwywSEozXGk/qtMcAhIEhIE\nCAkChAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFCggAhQYCQIEBIFNKYWdS52PUAN97a14VH\nFRJFNOa6di52PcCNt/Z18VGFRBFCSqwywSEoqnE9oM7Frge48dbgt9D21RsesOcTmsYQFCWk\nyCoTHIKihBRZZYJDUFZjI+xc7HqAG2/t6+KjCokihJRYZYJDUFpjG+xc7HqAG2/t68KjCgkC\nhAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFCggAhQYCQIEBIECAkJq3kLIk+IwiJCSs5b6/f\nCEJiwoTUj5A4ar/eSPutw497eZUbRrl+lQkOwQwIqSchcSSknoTEic9I/QiJEyH1IyQ+OI7U\nh5CYGSFBgJAgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQICQKExFQ0pgB9L7be\n+I+1ks/g6lWLrDLBIZiYxqTU78XWG/+xVvIZ3LBykVUmOAQTI6QBCOnhNC7c873YeuM/1ko+\ng1vWLrLKBIdgWoQ0BCE9HCENQUiPp7EVfy+23viPtZLP4IaVi6wywSGYGCENQEiPqLERfy+2\n3viPtZLP4OpVi6wywSEgSUgQICQIEBIECAkChAQBRUN6fV6d9tSv1q9DDQGjKBjSYVF9Ww4y\nBIykYEjrqn7ZnZb227paDzEEjKRgSHW1+1reVfUQQ8BICob04/SLy+diCImZ8Y4EAWU/I233\npyWfkbg3JXd/Lxt77RaHQYaAcZQ9jrQ+HUeqV8+OI3FfnNlAXtd0ouQcott0PoNrn6KQSOua\n4Jqc1dr3Gd56h/+vccOzuH6VCQ7BYIQ02CotD+I40r362qH0j4sANRZHf4a33qFllRuexvWr\ntDzI/x6lakoMwTiENNwqExyCoQhpuFUmOASD+d4IG5tj++Loz/DWO/x/jRuexfWrTHAIBiOk\nwVY5OzxV1XL78SB2Ntyx722wsTW2L46k8xlc+xRLTuyrT5Wvzg8iJO5J0ZNWN+81berT5Fgh\ncVeKTqM4/c++XuyFxJ0ZYWLfYbkUEnemYEiL6nPqxGIpJO5LwZA21dPH0r5aCom7UnL39/qr\nnm3HzkUhMTNFD8juVp9L+ychcU+c2QABQoIAIUGAkCBASBAgJAgQEgQIiSF1TkIaaGpS8RlP\nQmI4ndNiB5osO8IcXCExHCGFV5ngEAyvcTGe9uvytN+aHLcYITEYIaVXmeAQDE9I6VUmOAQF\n+IwUXmWCQ1CAkMKrTHAIiqgcR0quMsEhIElIECAkCBASBAgJAoQEAUKCACFBgJAgQEgQICQI\nEBIECAkChAQBQoIAIRHSNfXomilC3/e9ZhZT1whDTlISEhFdk2GvmbT6fd9r5tV2jTDstFkh\nESGkEqtMcAiiquprO20stn79isdqX+uaW297BjcQEglCKrLKBIcgSkhFVpngEGQ1ttLWDfaa\nrfj7vu1rXXPrbc/gekIiQkglVpngEKQ1NtLW7fWajfj7vu1rXXPrbc/gWkKCACFBgJAgQEgQ\nICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQICQKExN/cNplnyClAkyIk/uK26aXDTkqd\nFCHxF0LqICT+4LZL8Ax84Z5JERJ/IKQuQuIPhNRFSPyFz0gdhMRfCKmDkPgbx5EuEhIECAkC\nhAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFCggAhQYCQIEBIECAk/qZ9ZtEV842SU5OmN81J\nSPxF+1zXK2bAJifLTnHirZD4CyF1EBJ/0H49oCuuEpS8oNAkL04kJP5ASF2ExB8IqYuQ+Auf\nkToIib8QUgch8TftW+4V23Ny059aRkKCCCFBgJAgQEgQICQIEBIElA9ps6iq1XbQIaC0giGd\nd/0vzwfT1oMMASMpHdK6Wh/e3vbrajPEEDCS0iHV1eG4fKgWQwwBIykd0uepHZdP8RASM1M6\npKfPkOohhoCRFA1p9bzZVi/vi4f15b0NQmJmiob0dfZ7VdWHIYaAkZQ8jrTbbTar1WmXw/pi\nR0JibpzZwFnXFJ/G19sXW+/bedd7ISSOuiadNr7evth638673o++IR1P+Mm/RPf7ek+VkHrq\nGdJz9eXKB3EcaUq6foqNr7cvtt638653pGdI9eVTfS48yP8Grppue1BuJaS+eoY00Atzty/3\nVAmpr54hravL+7FvdLcv92R1beaNr7cvtt638673o+/OhtXyNfVU/jUEBQippx4hVT+N/Kzo\nqetH2Ph6+2LrfTvvei+EBAEOyELACCet/uEdTEjMTGr3d31xftHJRkjcrVBI+798RtrVyxuG\ngBnoEdL2xxvMxWswfNh1XDyoz7OCMfV5R1o0O/rT4aRNtRvqWcGYnCIEAXZ/Q4ADshAgJAjo\nfdJqfbwe/mv9FHo+LUPA9PWeRnHeDffnHdvXDwEzkNpr51c7Hlrvqeaf70jdpwjdOATMQO9f\n7erjkdhtXT2nntHvIYi4bXdQ13Sj4ecbzWUWU9+dDR9/N6xapZ7Q/4egv9v2rLav1bi1fTFo\nmEcdQu8Dsi+rqvtPWfYbgt6ENDRnNjyC2w72ta/VuLV9MWiYRx2EkB6BkAbX68yGH2c3jPys\nuEBIgxPSQ7jtZ9S+VuPW9sWg2XTkV7vHIKSh9QipXm1eB7nOqpAGcNvm2L5W49b2xaB5ZNT/\n7O969bzdR59RcwiYiR4hHV43Tx+zzVfrl7/NIR/uWcGYen9G2m2elnY28OgyOxu2SyHx0AIh\nvT4vvCPx4HqGtN8cT7WrFs/ZP+4iJGamz86G7bo+7rd72sZ3gguJmem5+3u1ye/7fhMSs9Mr\npPVAx2OFxNx4R4KAXp+RnnxGghN77SDAcSQIcGYDBDjX7k5c8RPomgXROUtiLlMbSnL29124\nYgJc17y8znl785lsV5L5SHdBSGMzQ/YeVNWft+7GXVsXu74+pwuSlOSaDfdASKMT0j0Q0uiE\ndBeu2LYbd21d7Pq6z0ithHQXhDQ2Id2JKzbtxl1bF7u+7jhSCyFBgJAgoO8B2W8jPysYk5Ag\nwK92ECAkCEiF9Br9a8xCYmb6hrT2GQl6h/TdUfTvmguJmekZUl29vC2r/X5ZRa9+IiRmpmdI\nx9/ont/fjXbVMvaU3oTE7ARC2lab80KOkJiZniGt3n+121eLt1ch8dB6hrQ9BnS6itBT7Cm9\nCYnZ6bv7+/n4/56qah16Pi1DwPQ5s+FBdE4nar218/igqUkfhPQQOie4tt7aeajdZNkvgb12\nJ3WdeDZtQxAgpKGFQtrbazdljfO42k/par218+yvQU4Pm6keIW1/TEdajPysuEBIg+vzjrRo\nduQUoQkT0uBSn5Gy/GTCGht8+7bfemtnJjr6Yq/dQxDS0HqHtF0dX8hV9i9S+NHENbb39k2/\n9dbOSmT0oW9Iy4/fvOtoSX44zEzPkDbV8nAMaeNcOx5a74l9h7ePK0anntHvIWAGAnvthAQ9\nQ1p8vCPtHJDloWU+I23r4yzZHCExM3332q0+DiVEL9kgJOYmchypWr2Enk7rEDB5zmyAACFB\nQJ+Q9uu6qteH5NP5PQTMQ4+Q9vVpN0P25KCfQ8BM9AjpqVoe3g7L7MlBP4eAmegR0vH0oOMk\n8+jVGn4OATPRI6SPs4KGOI9eSMyMkCZhoGk9nZOQSBHSBAw00bRzWiw5QpoAIc1fr5B+GPlZ\nzdhAF+NpPOxAI/BNSOMT0h1witD4hHQHhDQBPiPNn5AmQEjzJ6RJGGgjrxxHKkVIECAkCBAS\nBAgJAoQEAUKCgKIhvT6fL4O3Wnf8fT8hMTMFQzo0/1Tm5QtKComZKRjSuqpfdqel/bau1kMM\nASMpGFJd7b6Wd5ev9CAkZqZgSD/OUbl8woqQmBnvSBBQ9jPS9nwxSZ+RuDcld38vG3vtFhev\ndCwkZqbscaT16ThSvXp2HIn74syGeWifTtR6a+fMI7OUBiCkOWif4Np6a+dcWPNmByGkORDS\n5I0VkuNIV2i/CFDrre13bb9D5335u+mEVDUlhrgfQpo+v9rNgJCmT0hz0L7Bt97a2UbjDjrK\nEdIcCGnyhDQP7dt7662daTTuIKMUIUGAkCCg6HykH4YYAkZSMKSNkLhbJX+129WXL3kSGALG\nUfQz0u7ydL7EEDCKsjsbNo3Z5gMNAWOw1w4ChAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFC\nggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJAgQEgQICQIEBIECAkChAQBQoIAIUGAkCBA\nSBAgJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJAgQEgQICQIEBIE\nCAkChAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKC\nACFBgJAgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQ8WkhVJVIG8FghVWcDPToP\nTEgQ8FAhVZWSGIaQIEBIEPBQIfmMxFCEBAGPFZLjSAzk0UKCQQgJAoQEAUKCACFBgJAgoHxI\nm0VVrbaDDgGlFQzpfARneT4muh5kCBhJ6ZDW1frw9rZfV5shhoCRlA6prg7H5UO1GGIIGEnp\nkD7P0Ll8po6QmJnSIT19hlQPMQSMpGhIq+fNtnp5XzysL+9tEBIzUzSkr0kMVVUfhhgCRlLy\nONJut9msVqddDuuLHQmJuXncMxvMTCLoUUMyV5YoIUHAWCGNfBzJ9YTImk5IVVNiiMvDC4mk\nB/3VTkhkPWhIPiORJSQIKBrS6/PqtPmu1q9DDfF3MiKoYEiHRWNvwnKQIWAkBUNaV/XL7rS0\n39ZOWuWuFAyprnZfyzvTKLgrxa/Z0PZ/YkPASLwjQUDZz0jb/WnJZyTuTcnd38vGXruFiX3c\nk7LHkdan40j16nkCx5Eg6FHPbIAoIUGAkCBASBAgJAgQEgTcWUjtcyMat34vdt4V/u6uQmqf\nrde4tXmt1467wjWE1P0A0OmeQmq/oknj1u/FzrvCVYTU+QDQTUidDwDd7ikkn5EYjZC6HwA6\n3VVIjiMxljsLCcYhJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBwJ2F\n1DrzqH2SkZlHBN1VSK1zYdunvZoLS5SQIOCeQmq9XlD7pYFcL4gsIUGAkCDgnkLyGYnRCAkC\n7iokx5EYy52FBOMQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJAgYF4h\ntc8h6pyE1LrY+Vjwd3MKqX1Wa+e02NbFzseCawipewToNKOQqqptO2/cesVi52PBVYTUOQJ0\nE1LnCNBtRiH5jMR0Cal7BOg0p5AcR2Ky5hUSTJSQIEBIECAkCBASBAgJAoQEAUKCACFBgJAg\nQEgQICQIEBIECAkChAQBsw2pc+JQ+3wkGMRMQ+qcyto+QxYGIiQImGdInZf7adyh877Qn5Ag\nQEgQMM+QfEZiYoQEATMNyXEkpmW2IcGUCAkChAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFC\nggAhQYCQIEBIECAkCBASBAgJAiYaEszMDVt5Ppxpjl1yMN/Y3AbrP5aQZj6Wb2waYwlp5mP5\nxqYxlpBmPpZvbBpjCWnmY/nGpjGWkGY+lm9sGmMJaeZj+camMZaQZj6Wb2waYwlp5mP5xqYx\nlpBmPpZvbBpjCWnmY/nGpjGWkGY+lm9sGmM5NRsChAQBQoIAIUGAkCBASBAgJAgQEgQICQKE\nBAFCggAhQYCQIEBIECAkCBASBIwY0qbc2JtFVa8PRYY6PFXV067IUGevhV7Gmy8vf5Pd8WXc\nlxipx4XzfzxM5tncYFfsh/K2Pr1QdZGS6tNY5Uo61GVexl3RkLblfmKfHdU9HybzbK63q8v9\n61Y9HY5vgE8FxlofR1lXqwJDna0KvYy7gt/T+z9H9e7tsKrWxQbcVq/9HmCskDbVslhIq/NA\nRcarq0OpoU5eSr1JbKrnIuOcvJwSOvR9l/i7Q933n4mxQnp/pcptbR9Dlhuv2BawL/bv0aba\nFBnn5Kng78Ynq6rvb5FjhbQrumEfHaplqaHWxTa6ZbUv9DKuqu1TVRf6XWtRvT3Xp1/Iy9j1\n/yVyxL12hUPaVNsyA73/tlXqd/vn6qXUy7g6fyQv849RVa0Cn///rv8b0uOEtO/9W/BfbVZ1\noc8Tpx0AhV7G6r3Zt0Oh99r3hnbHAwmFPpbtAvuhHiWkQ13sF7u34+/4Rba3xXH/cNmXsVqU\nGOZ8/GBfZrDjr+L9f1l5lJCWhX4mZ2X2Nz2dfv5l39jLjFYV3M/6dtzV2v8xHiOk/WJZ5Cj5\nlyLfW5+/Zn/7mCVGKXnAInSI7CFC2pbbYXc+jlTml5KiIX1+Y0U+aT6f3mv3hX5skT37jxBS\nqR/I0enMhsOq4EGXQi/j+rgr8pD4NPEH7/8QHY47G15KDPb+/hc4avUIIT2V/A2oLriX+KTQ\ny3g4f2OFduw/l3wVF/13fj9GSGU/SqzralHwJIBiL+Oh6De2XRY7+pt5Bc1HggAhQYCQIEBI\nECAkCBASBAgJAoQEAUKCACFBgJAgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQI\nCQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFNwvkvCtZPv//2+u+/2Nr443JX/J25\n7ZX353pe3Un4/OOc9c+SFr9/PDeFdH4UIQ3KqzsJ5638sPz1x47/t/HfFFIlpOF5dSfhYys/\nVHXbza03CGlSvLqT8LmVn/93s6jqzdvHL3zv/7tdVR9/4/sfIX2u8H7jflXVz6cb1/X7+9v7\nvT4e5f0/688vESekSfjxjrQ6bfrLr5Cez5+f1m//Culrhfcb6+PiMZflceGpGdLpbpui39fj\nENIknKvYnz4jbavl4fhxafv9S9nL29vLabk1pB8rvC9uqsXxxnr3tqs/3oveml9iAEKahK+9\ndofj+8v7f72/Oa1+/vr275B+rPD68aXVsav3nBohvf58AJK8rpPQPI70GVX1vdnvt8/Lf4f0\nvxUaq/68UUiD8bpOws+3nt8hLT//v5Amy+s6Cf/7Ha65+FQtNtv9pZB+3Sik8ryuk9Dcvs+f\nbho3n/77QkitK7R8Rvo9EEFe10lobt8vx/1tb5vzvoPzh6bXt92Fz0g/Vvj80o+9dvs3IQ3M\n6zoJP7bv5ddpd4vqeGBp/fER6PVXSF+fnL5XaNSy/Pr6+VGENCiv6yT83L4375v+aQfe6+J0\nhPapqpav21/7wxshfa/QrGVdv691XDw/ipAG5XW9b6fzHRiekO7U6XSIw+rX6eQMRUh36uME\nvbr7niQI6V5tllW18H5UipAgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQICQKE\nBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJAgQEgQICQI+A+h+QFTyPGFOQAAAABJ\nRU5ErkJggg==",
      "text/plain": [
       "plot without title"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 420,
       "width": 420
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot(x=iris$Sepal.Length,y=iris$Sepal.Width, pch=16,xlab=\"Sepal Length\", ylab=\"Sepal Width\")\n",
    "plot(x=iris$Petal.Length,y=iris$Petal.Width, pch=16,xlab=\"Petal Length\", ylab=\"Petal Width\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can start with a good one-dimensional representation of the data. \n",
    "\n",
    "Some Possibilities:\n",
    "\n",
    "- Use <span style=\"color:red\"> one of the variables </span> (e.g. $X_1$).\n",
    "$$\n",
    " Z_1 = X_1.\n",
    "$$\n",
    "- Better idea: use a linear combination of the variables; i.e. a weighted average of the variables.\n",
    "$$\n",
    " Z_1 = \\sum_{i=1}^p w_{1,i} X_i,\n",
    "$$\n",
    "where $w_{1,i}$ is the weight for $X_i$. This is a better idea since it includes the case for one single variable as a special case (e.g., $w_{1,1}=1$ and $w_{1,i}=0$ for $i\\neq 1$).\n",
    "\n",
    "But how do we find the optimal choice of weights? "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There can be many criteria for choosing the weights, and thus many \"optimal\" weights under different criteria. \n",
    "\n",
    "For principal component analysis, the criterion is to find weights that help explain the maximum amount of variance. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A systematic way to find additional principal components (PC's) is to choose subsequent linear combinations  orthogonal/perpendicular to previous ones. \n",
    "\n",
    "Suppose that we are looking for the second direction \n",
    "$$\n",
    " Z_2 = \\sum_{i=1}^p w_{2,i} X_i,\n",
    "$$\n",
    "The orthogonality constraint means that we want to choose $(w_{2,1}, w_{2,2}, \\ldots, w_{2,p})^T$ to be orthogonal to $(w_{1,1}, w_{1,2}, \\ldots, w_{1,p})^T$. At the same time, the two direction $Z_1$ and $Z_2$ need to explains the maximum variation in the data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAAMFBMVEUAAABNTU1oaGh8fHyM\njIyampqnp6eysrK9vb3Hx8fQ0NDZ2dnh4eHp6enw8PD////QFLu4AAAACXBIWXMAABJ0AAAS\ndAHeZh94AAAbWklEQVR4nO3d60LiyBpA0SBqO7bi+7/ttBcUEEJIvqrUZa0f5zi2oaQqm4SA\nOrwBiw1rfwPQAiFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFB\nACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFB\nACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFB\nACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFB\nACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFBACFB\nACFBACFBACFBACFBACFBACFBACFBACFBACFBgAwhDVCZGXt5fDgrDAGRhAQBhAQBhAQBhAQB\nhAQBhAQBhAQBhAQBhAQBhAQBhAQBhAQBhAQBhAQBhAQBhAQBhAQBhNSEWT/qTCAhNWD2bw0g\njJAaIKT1Cal+C36RDVGEVD8hFUBI9RNSAYTUAB2tT0gNENL6hNQEGa1NSBBASBBASBBASBBA\nSBBASBBASBBASBBASBBASBBASBBASBBASBBASBBASBBASBBASBBASBBASBBASBBASBBASBBA\nSBBASBBASBBASMcD+42lzCKkw2H9Dm1mEtLhsEJiJiEdjOrvDDGXkA5GFRJzCelgVCExl5AO\nh9URMwnpcFghMZOQjgeWEbMICQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQII\nCQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQIIqR5+DWzBcoa0exiG7fPX\njYzeih3mN7+YvGgZQ9ptPvaE+88bEdKNhFS0jCE9Dk//anrabD9uREi38cebypYxpM3nhq+b\nu1ch3UxIZcsY0n4f2G2350IaDs0comGmpmwZQ7obdvuPto5IN9NR0TKG9DQ8fH30OmyFdCsh\nFS3n5e/H793g+coeYXc5R0YFy/qC7Mv9/qPXByHREu9sgABCggBCggBCggBCggBCggBCggBC\nggBCKpo3M9RCSAXz9rp6CKlgQqqHkMrlR5AqIqRyCakiQiqXkCoipILpqB5CKpiQ6iGkosmo\nFkLim2znE1LHjsNxIrmEkLp1Go6QlhBSt07CcbF9ESH16jQcIS0ipF4JKZSQevUrHB0tIaRu\npbzY0F+QQurW73ACM+ru2CakjqXa2YWUapMChyCZLi9bCIloQkq2SYFDkIyQkm1S4BCk02FH\nQiKekFJtUuAQpNRbRkIiXH8RvRMSoXo8rXsnJEIJKeUmBQ5BEl1e+n4nJCIJKekmBQ7RshX3\nYyEl3aTAIdp1tCdf3KW9YTWWkJpzsCtf3KvT7e5CSrlJgUPU48bd8vDkaoWQvI6UcpMCh6jF\nzbv8QUgXn7B0+0wmHSEVTkh1EFLZZuzyv87shJSBkMq2KKR1niP1SUhlm3XsGA4ufgspDyEV\nbuEuf3FTGcUSUuEcO+ogpOLJqAZCggBCggBCggBCggBCggBCggBCggBCggBCYgmvFn8REvN5\n/9I3ITGfkL4Jidn8fOAPITGbkH4IidmE9ENIfYnd63X0TUg9id7xhfRNSD2J3/Fl9EVI7Tv9\nXSj2/QSE1Lrfv51LSAkIqXVCykJIjTuKR0fJCKlxQspDSI07OZ2TUSJCap2jUBZCap2QshBS\n+2SUgZAggJAggJAggJAggJAggJAggJBIoL8r7kIiXI+vAQuJcEJKtUmBQ5BMlz/2JCSiCSnZ\nJisN0dlaTpFjSoSUbJNVhuhwNa/JNCU9zryQeiKkZNoNqcsTjHH5pqS/aRdSR0xJOkLqiClJ\np92QejxTv3Z3O5ySXITUjut3uLspyafhkLp7yjslk86mJJ+mQ+rL8GPtb6VDQmrGMEhpPUJq\nhpDWJKR2KGlFQmqHkFYkpJZMDUlo4TKGNAyTHzIt82xTZtcxK17GkJ7GQ3JiEkJI68h5avey\n2cYOYW84Z8J5nZLCZX2O9DI8Bg5hd5hHSCnkvdjwNLzEDWF3mEdIKdR71c7+MFfqeetyTYTU\nn7Tz1umqCKlHiY9HPa5KvSFdXLEOV7EgvT6+NRdSn8tYjmkhtbdCFYd0fjmEtK4pIbW4RlWH\ndG7LTs8syjFh/ltcIiERa/IBqa01ElJnMkzNhCdI7a1RayE1edoQp4jZEdLsTTIO0eAaBSpj\ndsr4LmI1F1KLl1bDFHIsKOKbCNZgSFxUSEgtPtgJqScZQmovkWmE1JXUHRVywFuBkJp3uGML\nKRUhNe5g1/74v9Tndb2WJKTGfe/ZOXZxISXepMAhOvGzawspKSG1bfglx3BJxyiTkNompEyE\n1LiTM7vkO3mfGXUUUrcLnPNiQ8c6Cen3XtTPPvVz8bvWkGr4tjsNqeK9ar5K73Ada9VHSL+e\nINSxOJVJNKF1rFWfIXX8ekcyqWa0krUSEjGElGGT1Ye40FHhi1OTZFNayVr1HVL0MB1LN6d1\nrFUnIR09E3ZASkBIOTYpawgdpZBwTmtYKiGxzH4eO5/THkM69/Js2gHbdTiVXU+jkHp/KF3G\n5H3pMqS8v8egHPH30lnyXushXV/jbvaFFCe03UzeVW2HNGWVu9kXUpzQdjN5Vwmpl33h5H4u\nvdND9T+cEavpkKY10sm+cDwZCx8+vjfuZPKuE1Iv+0KakDq/6P1DSG+97Avnzuxm3vFeTodv\n0HRIvRxspjmejIgDkpn9IaSOHE6FkGK1HVIvZ22zLJkaHZ1qPSSSENKp7kKy/DHM47HOQvJI\nShpthDS5DSHl1c9ctxDS5Dq+Lzb1sror62myOwppGISUV0+T3UBIU+sYBill1dVc9xPSIKTM\nfs11y9PebUhtLmlR9+tkqlue9yZCmrhCHYRU2j07/n5Ov7uSvtPl+gmpg3O7mffs1yZRs3Pu\ngNTq8amFkCaufC8d3Xjffm0TOUHD0RMkIS3dpJAhhrZ/prPAkH6PE/ODheVpM6TxBWpo+Y6M\n75wXPv9ro2S7+NiJXv1aDKmxJZpu5I5f/CchxRBSQ8oO6fig2NoiNRhScw92Nxg5r7tS0sgn\nkmhtjYTUhfJCau2passhtbROC41Oya9Pm7rbNRjS4QtG+/+O+Kaq5qElsfZDsgu9MwuJtRjS\n8dmdXeiTpz1JtRnSwXMCT5iy6XqihUQAx/5GQ/pZVCF9STgFw7FUwxSt+ZD6fpz8lnQWhNRu\nSD+PwD2v7o+UszAIqeWQDm6t07U9kHQn19FbHyEhpOSE1IW0e/nhlZ0kA1RASH3IcbEhzY1X\nQkitOt6zb93Xb+yi94yE1Krf4dyY0fTsRPSun5D6Wu9lZ1s3bO207lMvIXW23tcuLoxPxS2X\nJjqb2IuE1KTxFK5Nxg0Xs2MvB1a8Qp2E1NuLHOMh3BLSlSmLnNifW6pwoYTUprE7fH0y1g2p\nyqUSUpvGOhj/p+Otp5YU+i3XuFRLQto9DMP2+euToffbc6TF5oR08OkCQqprsRaEtNt83N37\nz08KqSxju+Olfzn+/NQdOmpauw3pcXj6V9PTZvvxycJDqvIJ7CIje+P4Aem0pKTf5dnxOwtp\n8/nB6+butYaQejO6N579h0JCqvPsYUFI+7u6226FVKJbd8bfh4Lcu/NQ7880LwjpbtjtP9oK\nqQXF7MFFfBO3WRDS0/Dw9dHrsBVSA4oJqUJLLn8/fs/5c/D0W8uVBK1jhzUuekH25X7/0euD\nkPjS5YGtk3c2cMnIHj83BiEl26TAIXh3+4tNk2+zs5KE1LWMITVelpB6NnLwmH9cObtl8wcp\nIfVsdkijTZzbUEghmxQ4BG+zQ7pSxeUDUsMlCalr854jXY3i3BMkIQVs8uXvn/uP2bx//Jtq\nCG4yq5YZVQhpdJPXh6f3/9vdPU3Zbnc3/NhGf1fXtbyM842fol36/M1VtN7RopBeN58/1fc8\nDJvX69s9Dpv/Xj43fN4Mj8Hf1TXNr2Sca9PUb0hjjzozbm3/wd3w8Pn+77/b4e76dpvh5fvj\nl2EzaYg4baxkBtcmapjT0VsLJwSjd3tBSM/Dn+/P3Q//Tfg+Lv3H12eGuWs0Qfsn6VFShVS/\nVCE9fP880scPUlzdLs8Rafypc3+rf6trE9XvRI7f8wUhXTnC/PLvOdLz51OpdM+RLt7Vftf/\nRkK6JFlImxtDetsenLvd7ca+Mj4kz5EmuimkriY0WUgPw/P3556H+/NffOTv48frSJv7P4le\nRxq5r0Ka6MpEHUxxb1M6en8XhPTyc9H7dTPhYsOMIW7ebuxBY+qa97RvnHEtj59/F9LhP864\nvf0H/570/Hm/fvDyZzPhWsOsIW7dbvkJfG97xxnjd//0gNTTXI0dqWfc2vdHf77n8mHk62dI\n8Bwp3y007+cJkrn6tuy9dq+P7xcQ7v9MeF/D3CFu3DDogGTvuM5UHWru3d8LF9beMZ2ZOtBc\nSMe3c/MyC2k6M3Vg0ZtWHzfD5nH0BaGZYhbncKEnL7i94wYm6tuCkF4//6zLlDd+zx1i4a3M\nuVArJOZY9ILsdve220ZfsTscYtmN/Jyl3VSHjLjdorcIvZ/VvY6//XSe4JA88SG15W9aTbF7\nConKNBxS16/Bk1kXIbmCQGoth/Rz3UBIJLYopCMrf1fXblFGpNRJSDoirbbfIrS/OWd2JNZx\nSNIiTg8hnT37dJSaYeGEtTzfQmKqhVOWacZXWtS+QvqZZK/R3i4mpLRzvtqq9hDSyfXF40/F\njtSyhVOW7BLvuVGS3f7lgbNssvIQQoqQM6S5o6y3rF2E9LkwZ0uKHqhhkSGN38b8cYSUYQgh\nLbRwyoS0fJMihjhzdpdimHblCmlJDas9PnYUUp7LRk2bMXWHm9x2QBLScqlDUlImv6Y7dUhe\nR8oyhJAyOzPdE2a/xkUSEunMnO8aF6mvkKpcoorNfuCqb42ERDodnQF0FlKNj3U1m9tRfavU\nXUjcZOEePS+kGg9jQuKygD165vFISCHWeEGtsoXLYp09OvMzq5iBhPQ54M/KKepbwB694IA0\necul32BIs0L6HHA/nTWeVSSzOKRFT5EmbrfKt3jmdrJssuYQU2bpZ+mEdGClkN5uGnbZtxh2\nGtl6SNOmafgl7BuoWkxHSQ9JC1es05Bmr8m0rxLSCSFNv6Esm8QMMeMuT5qng3yEdGLRVFQQ\nUpfPkdKEdHwc0lGkubN5y3YxR825m//cTpZNQoaY89hz2MiFR9ejLxFSqApC6vB1pFkH8Z+N\nLmz861ZlFGnubN6yXREr1k1Ilzaedau9Mk0XVRTSvIP4tZKENJmJGtF9SHaPyc7OlKn7VFNI\nC9+5NX5Iuvl2u3NuAk3eXl0hzbqtr7W+fA5nT5hESGPaD2kfyuWTOyY5M3+m9FsPIe1v1Kov\nc/GAFD6lU26xsIUUElPlCmnKbRa3lB2FVN7kV+fX7CWZUiGFmXVtbsrXlDX59Ut4QBq91fLO\nLtoIaeqsljTzVbg+YUmeIAkpSKqQuMk60yqkOHPfvZDmu+nW3FlduBJnL2oEfWvJCIlLZk7r\n4rU4vYFzN1jcgguJS9YK6fQIdP4GC1vuJkL6PdWFzXKd5oUU/qhWx8NkkyHVMPE1WHJAElKK\nTdIPcXo8mjDxpS/N+oQ0XSshHW09ZeZrWJz1zZmhiTM78iUn/1TFUgmJYJNmduSLfv1TFUvV\na0h1nC9U6jSD819yJaTDf61gnVoMacpDmJDyuDDLI9M/HMrxLcYQEikJKXST7ENcX4PqVqpK\nl4qYGlI969NoSBOGqGyh6nQxiEnPkWpan25DquIZbPWEFLtJgUOQxeUeRiqpsCMhkdTcIIQU\noqIJ5Iq5OVSVkZC4orL9eTVCYkR1Z1irERIjhDSVkLiswqtnEyS5P0LishZDSnSPhMRlQpp+\ns1k2KXAIpmi2o/hfa5llkwKHYAohTb7dLJsUOATvru9QbWUkJBII26Vqqi3RG/mE1LGgHSp6\nvzx7W2EDDEOSlITUr6j9KXavPHtroUMIiVBB+1Pwbpk+pCRPlITUryJDOntr0Tu+kIgUemYn\npAybFDgEZV5syBJSgtfH+gkpdt4aETIpSY4WKYcQ0oJbDJ85fiTfxxPs+F5HmneLQqrG73VK\n8JwmWichVbASXFbB8gmJ8lWwfEKiAuWvXichVbASjCh/+TKGNBxLMcTVwcNvl0xKX7yMIT2t\nHlL4rcKXnKd2L5tt6iEu3Z7j0XoWTXw1q5b1OdLL8Jh6iAu3J6S1TJn6i/9e0brlvdjwNLyk\nHuLszblmt5rrMz/yFRUtWzlX7SY/gVpw07E3y3UTpv7yF9S0buWElHCIdIlyxfV5H/mKmlat\ni5Aq/SNwLQgKqfxVWyOk67MipGZcmPSfT42tS0Wr1kdIdT24NeXsnB99cmRVKlq1XkJ6q+up\na0vOH48mhVTR0yQhkd3pSsx8ClWUjkKq6VWJtt0URyWrJiSyE9LcTbIPcfnJ69JbJsBtcVSx\nak2GVM0z1F41uDqNh3RmsRpbwEo1twothjSMhNTgYyElaD6ksZcCIUpnIXnyRBothjRybick\n0mg+pAv/smwAONFkSCMXwHVEEo2G9Hkr5y9+C4l4LYd04bZlRLz+QoIEhAQBhAQBhAQBhAQB\nhAQBhAQBhAQBhAQBhAQBhAQBhAQBhAQBhAQBhAQBhAQBhAQBhAQBhAQBhAQBhPQzqN+KwmxC\n2g/p93SxgJD2QwqJBYT0NaLfZcwSQvoaUUgsIaSvEYXEEkLaD6kjFhDSfkghsUA/IV2tREbM\n10tIpwcc1RCqz5CcxxGsk5BOLsoJiWBdhuRaN9GEBAE6Cen4ZE5IROsyJM+RiNZLSMcXvIVE\nsH5COhlBRkTqNSQIJSQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQI\nICQIICQIICQIICQIICQIICQIICQIICQIUHlIfj0dZag6JL8wlVIICQLUHJI/KkExhAQBhAQB\nag7JcySK0VFIkiOdqkO6JQ5HL1KqPKQbblJIJNRLSC5MkJSQIICQIEAvIXmORFJCombFLGk3\nIRU050Qp6NGxo5BojpDWH4L6lXQFSUhUq9eQdg/DsH3+upHRWylhZihepyHtNh93+v7zRoTE\nYuV0lDOkx+HpX01Pm+3HjQiJxfoMafO54evm7lVIxCgko6wh7e/ybrsVEo3JGNLdsNt/tBUS\nbckY0tPw8PXR67AVEk3Jefn78bue5yuntkKiMllfkH2533/0+vDrVoZDs4eAVXhnAwQQEgRY\nI6TrZ25CojJCggBCggBCggBCggBCggAuf0MAIUEAIUGA5kLyRj3W0FhI3vPKOoQEAdoKyY9h\nsBIhQQAhQYC2QvIciZUICQI0FpLXkVhHcyHBGoQEAYQEAYQEAYQEAYQEAYQEAYQEAYQEAYQE\nAYQEAYQEAYQEAYQEAYQEAYQEAYQEAYQEAYQEAYQEAYQEAYQEAYQEAYQEAYQEAYQEAYQEAYQE\nAYQEAYQEAYQEAYQEAYQEAYQEAYREU9b606dCoiHr/TFuIdEQIWUfggYNw2olCYl2CCn/EDRI\nSPmHoEWeI2UfghYJKfsQtOMwHa8jZR6CVqx3FDr6LrJsUuAQtEJIqw5BI1a8Unf0bWTZpMAh\naISQ1h2CRggp2xCrTzIpFdFRByGVMc8kU8YCC4nqlbC8zYdUyCk0jRMSBBASBGg+JM+RyEFI\nEKD9kMq4qEPjeggJkhMSBBASBBASBBASBBASBBASBBASBBASBBASBBASBBASBBASBBASBBAS\nBBASBBASBBASBCg0JKjMjL08PpxVxsg7kLtUw0BZz4WEVPZI7lIVIwmp9JHcpSpGElLpI7lL\nVYwkpNJHcpeqGElIpY/kLlUxkpBKH8ldqmIkIZU+krtUxUhCKn0kd6mKkYRU+kjuUhUjCan0\nkdylKkYSUukjuUtVjOSt2RBCSBBASBBASBBASBBASBBASBBASBBASBBASBBASBBASBBASBBA\nSBBASBBASBAgR0i7h2F4eMkw0NvT3bB53OUY6d9Y6afucdPU3fkYJtcK5dvpPuWYvs3HL/jP\ncKcePwbaZNn1Xub8yYLbbD/uzl3qYT5kuDvv8q1Qtp3uS4bpexwe3v/nPvlAL8PD7v2h9SH5\nSP/G2iTf8/4Om5f3cf4mHuddhrvzMUy2Fcq20+1lmL7N8P4AlGGh7j+HyLFLPA3b5MM8Ds//\n/ve/4U/icd7y3J13+VYo2063l+9XXmyyjZThPg2P6Ye5H17f3h/EMzyq5rg7h8PlGivfTpct\npMfhKdNIu2GbfpCXDHvDkO8Am+Pu/MiyQu/y7XS5Qvpv+PeYl8nTxxlRei2FlG+Yd5lWKOdO\nlyukp/tNjlP9d6+bTE8whTRTrhXKuNO9ZXyO9JDnMLvbZDptENJM+VYo2073LuH0Hf996F26\nJ36HA22TvuxyOFLyPW/TZkhpV+hYwp3uVLaQEq7Uz0Cvd9vXVKMcj5Rhz/u8avea6bWQTCGl\nXqET+U5Ys72O9JrhJfrnXJeD3iVfoz8fz8mfMz1jzrPLZVuhfDvdl1zvbNjdpz9dfc3ZUfo9\nL+c7GzKFlG+Fsu10e/nea5d+Ch+G4eR0MqX0w9xlmrcPWWYt4wrl2un28rxVcTPcZXhoGBoL\naffx7u/Uo3zJMms5VyjTTrfn55EggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAg\ngJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAg\ngJAggJAggJAggJDq8PmH7jYPX38S/OVhMzw8n3zNk8Vcj7mvw/5vRm4+Snr8/I+718Mvecn0\nNz85x9zX4TOS3XZ4/5uyf4bNv6PR7s9XVp9eNkJakbmvw1cku2Hz9va6D+hhePj+gqdhK6QV\nmfs67CN5///H4c/nf+zuf/5s979DlZBWZO7rcHhE2g4vv7/g5U1IazL3dfiM5PXjOdKlYIS0\nInNfh++rdjshFcnc1+HwdSQhFcjc1+Ewkvvv50jPu0tfQ2bmvg6HkfzZX7X7O9xd+hoyM/d1\nOIzk+3Wk7fB06WvIzNzX4SiSh493Nrzev18Lv/Q15GXu63Acyfbce+2EtCZzX4eTSP67H4bt\nf+NfQ07mHgIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQII\nCQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQIICQL8D8JNriMT/Baq\nAAAAAElFTkSuQmCC",
      "text/plain": [
       "plot without title"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 420,
       "width": 420
      }
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAAMFBMVEUAAABNTU1oaGh8fHyM\njIyampqnp6eysrK9vb3Hx8fQ0NDZ2dnh4eHp6enw8PD////QFLu4AAAACXBIWXMAABJ0AAAS\ndAHeZh94AAAb/klEQVR4nO3d2ULiSACG0QqbiILv/7Yj4IIdB2PyZ/Wci2mbkUoJfA0kKSwv\nQGdl7AnAEggJAoQEAUKCACFBgJAgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQI\nCQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJAgQEgQICQIEBIECAkChAQBQoIA\nIUGAkCBASBAgJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJAgQEgQ\nICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJ\nAoQEAUKCACFBgJAgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFCggAh\nQYCQIEBIECAkCBASBAwQUoGZafEoz4czwiYgSUgQICQIEBIECAkChAQBQoIAIUGAkCBASBAg\nJAgQEgQICQKEBAFCggAhQYCQIEBIECCkWWi1lJkBCWkGWn8qAIMR0gwIafqENH0dPqiGoQhp\n+oQ0A0KaPiHNgJBmQEfTJ6QZENL0CWkWZDR1QoIAIUGAkCBASBAgJAgQEgQICQKEBAFCggAh\nQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJAgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAg\nJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQ0vlL+1s+7SEIaW7kaexp0I6SxCWkR\nhDSyUpS0BEIamZCWQUgjE9IyCGlsOloEIY1NSIsgpPHJaAGEBAFCggAhQYCQIEBIECAkCBAS\nBAgJAoQEAUKCACFBgJAgQEgQICQIEFKSBRF/lpByLNH7w4SUI6Q/TEgxPsbkLxNSjJD+MiHF\nCOkvE1KOjv4wIeUI6Q8TUpKM/iwhQYCQIEBIECAkCBASBAgJAgYN6elhcznSstk99bUJGMWA\nIZ1W5dO6l03ASAYMaVeqx+fLV8dDVXZ9bAJGMmBIVXn++Pq5VH1sAkYyYEhfTp+5fy6NkJgZ\nz0gQMOx7pMPx8pX3SCzNkLu/1zd77VanXjYB4xj2ONLuchyp2jw4jsSyOLNhpix9mhYhzZLF\nuFMjpFkS0tSMFZLjSF34wKLJmU5I5VZiEwvmdpocL+3mSEiTI6RZ0tHUCGmWhDQ1o4T04yPA\nI+RHMpoWIUHAoMsoGu+YExIzM2BIT5WQWKohX9qdNmV9WUfhpR1LM+x7pMdSHl+ExPIMvLPh\nuC6bk5BYnMH32j2U6iAklmb43d/Pq58PgQiJmRnjONJWSCyNU4QgQEgQICQIEBIECAkChAQB\nQhqahUSLJKRhWdq6UEIalpAWSkiD8vE/SyWkQQlpqYQ0KCEtlZCGpaOFEtKwhLRQQhqajBZJ\nSBAgJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJA6GG5pkUVM/Wtw\nG9/5FiG1NtxiV8tq+9fgNr77LUJqTUhLIqSxDPeBQD56qH8NbuP73yKktoS0JEIajZCWREjj\n8R5pSbxHGo2QlkRIIxruwS2j/jW4je98i5AgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAg\nJAgQEgQICQKEBAFCggAhddBqBYulRYskpNZaram02HWhhNSakPgkpLZafe6MDwRaKiG1JSRu\nCKktIXFDSK15j8QnIbUmJD4JqQPHkXgnJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJ\nAoQEAUKCACFBgJAg4C+E1GoFkIVEEzXRu2H5IbVak2pp60RN9m4QUsMrTfYe/FsmezcsPqRW\nn9vj438marp3g5CaXWm69+CfMt27QUjNrjTde/BPme7dsPiQvEdalMneDUJqeKXJ3oN/y2Tv\nhuWH5DjSskz0bvgLIUHvhAQBQoIAIUGAkCBASBAwfEj7VSmbQ6+bgKENGNJ1///6ekRt18sm\nYCRDh7Qru9PLy3FX9n1sAkYydEhVOZ2/PpVVH5uAkQwd0vv5HT/87tWWm4CRDB3S9j2kqo9N\nwEgGDWnzsD+Ux9cvT7v7exuExMwMGtLHKfClVKc+NgEjGfI40vPzfr/ZXHY57O52JCTmxpkN\njTVYodRurcxEV9jwG0JqqMGa2XarNye75pPfEFJDQuKesUKa23GkBp8r1O4Tbqb7uTj8xnRC\nKrcSm4gSEnd5adeMkLhLSA15j8Q9QmpISNwzQkj7qqzuLqKYZEiOI3HPoGc2bEq1f3m4/AO8\n7mcTMI4BQ3q+FLQr29PLcWNhH4syYEjb8xnfu+v6CQv7WJbBP7OhbG7+kt4EjGTwkB6vr+ks\n7GNRBn1pt31fPHHaWtjHogwY0qn6eD1X7j8hCYm5GfQ40u49n+r+x9oJiblxZgMECAkChAQB\nQoIAIUGAkCBASBDQNaTzrw3Lr0zLhtRgck3mH1qPNODqo1Y/uNVRrXQM6eHjIwcmG1KD6TX5\nCUIrZAdcD9vqB7det6WOIVX31xW1JaQAIQ2pY0g93eTBURs8YzZ5Uq1/T+2SdsP0ptUPPuD8\nFqZjSLty/9PwWxJSd0IaVNedDZv1U2oq/7eJjkMJ6RffIqS2OoRUvhp5Vv8/1s/Ta/IT1L+n\ndkm7YXrT6gfXUUtCavYtQuKuv3BAtsEDo8ljp/49tUvaDdObVj+4jFr5CyFB71K7v6v7a8c7\nbAJmIBTScbrvkWAAHUI6fNnXcPcDHweYFYypyzPS6raj6OEkITEziz9FCIZgrx0ELP+ALAxA\nSBDQ+aTV6vD636dqG5rPN5uA6eu8jOL58ufz/Q/F77IJmIHUXjsv7fjTOoZUfTwjOUWIv6zz\nS7vqfCT2UJWH1Iz+3QTMQNedDeu3fXab1ITqm4Dp63xA9nFzzugQms63m5iIP7J0J/RDLfK2\nucOZDQ39kcWkoR9qkbfNXUJqSEgjDDMjnc5s+HJ2w8iz6ln9x+zlBx9b6Ida5G1zn5CaEdII\nw8yJl3bNCGmEYeakQ0jVZv/Uy+esTjEk75HGGGZGup79XW0eDsfojG43MSFCGmGYGekQ0ulp\nv31bbb7ZPT6PPKv+1R8Yi3yohH6oRd42d3R+j/S8367/wM4GuCuzs+GwFhJ/WiCkp4eVZyT+\nuI4hHffnU+3K6iH7y12ExMx02dlw2FXn/XbbQ3wnuJCYmY67vzf7/L7vFyExO51C2vV0PFZI\nzI1nJAjo9B5p6z0SXNhrBwGOI0GAMxsgwLl2EODsbwiY+3qkJk+FofUPU18Y0Nv8erv9ape0\nuzcnYd4rZJusHwutyJv6UrXe5tfb7Ve7pN29ORHz/swGIX0Q0rhmHVIpP9+u9e9pcq1WmxpT\nb/Pr7farXdLu3pwKIeU2NSYhjUxIuU2NSUgjm3VI3iN96m1+vd1+tUva3ZsTIaTgpsYkpHHN\nOyTHkW70Nr/ebr/aJe3uzUnoekD208izgjEJCQLm/tIOJkFIEJAK6Sn625iFxMx0DWnnPRJ0\nDumzo+jvNRcSM9MxpKo8vqzL8bgu0U8/ERIz0zGk8yu6h9dno+eyjk3pRUjMTiCkQ9lfv8gR\nEjPTMaTN60u7Y1m9PAmJP61jSIdzQJdPEdrGpvQiJGan6+7vh/PftqXsQvP5ZhMwfc5sgAAh\nQUDnkB7Pn6K/jR6OTYfU4LSL0JKbZept9daSbuOuIa3fTmyInmoXDanBGUyhRaDL1Nt64mXd\nxp1PEarOT0aH6nwsKUdI0yGkRjqfInT9zO/nssrMp76JrkP9fFJt/Vt6ORN3nlrdFE2utLDb\nOHBmw9cvIoQ0GUJqpvNLu/dnpKmuRxJSN0JqpvMB2ct7pKcqes6q90gT0uqmaHKlZd3GnV/a\nfTHirP5/LCF1IqRGlh+S40hdtbopmlxpSbexMxsgQEgQ0Dmkw+b8/LzJ/vpLITEzkVOEXi+r\noiUJiZnpGNK+rE/nkPYW9vGndT5F6HQ9qWGyZzbAEAKnCDUP6elhc9lLvtn98OFdQmJmOoa0\nentGanLS6ml1c8Tp/pkQQmJmMu+RGi2j2JXq8Xpm3vH1++9+yIOQmJmue+02jZ5hLt6XXJw9\nlyo8KxhT5DhS2Tw2ud636y9Ss4IxDXhmg2cklmvAkM7L0q+Hbb1HYmm6hHTaXb58WpWq0Sc2\nrG/22q1O4VnBmLqEVF3e6Bya7mx4TW532TdRbR4cR2JZOoR03vX9+kdVPb+c1qXJ7obOs+pt\n2VCTgXvbVIPvabe6p9UwTTQYOHT7jXqP/2q89ldZl/M7nqfycPnvAL8fqbeFrE0G7m1TDb6n\n3XrTVsM00WDg0O036j3+yxHbX+W61d31d/UNca6dkH41sJCGHLhzSKty85dfDPL740il/PvD\n1i9ppcnAvW2qwfe0ulK7YZpoMHDo9hv1Hv/tkO2vsjq/tDte10+c7h8X+maQ2obLre+vMuLN\n2tumGnxPqysJadiBO4S0O+9s2F5/nfkQ65FGvVl721SD72l1JSENO3CHkE7VecOXnQz7cnPW\nQoD3SF++p9WVvEcadOAOIb2c3n9TXxnmN/YJ6VcDC2nIgbuE9HnJ5ocDrF/sq7L64UyI/5tV\n/SdNPDAaDtzbphp8T6srtRumiQYDh26/Ue/xX403yFWunjfnc4keLuVb2MeiDBjS86WgXdme\nXo6b+wsBhcTMDBjS9vxGanfdT366vzRdSMzMgCFdX4K+/f6XFgdkYboGD+nx+prOwj4WZdCX\ndtv3RUinrYV9LMqAIZ2qj9dz5YczioTEzAy5+/tl955P9cPxWyExM4OGNKVNQJKQIEBIECAk\nCBASBAgJAoQEAXMPKbXCZlpCC4lCy4YabHzUu2ESj4F5h5Ra8zktoaWtoYWsDTY+6t0wkceA\nkKZHSL8xkcfArEMqZSK3YlTtp2r3YzYYpt0lP29pSFN5DAhpcoT0G1N5DAhpcoT0G1N5DMw6\npKm8Pg6r/VTtfswGw7S75OctDWkijwEhTY+QfmMij4F5hzSRYwhxtZ+q3Y/ZYJh2l/y8pSFN\n4jEw95BgEoQEAUKCACFBgJAgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQ0hS1WkjU\nZtz+Vig12XhvRpifkKan1dLWNuP2t2a2ycZ7M8r8hDQ9QupGSENuYrpK+edOrV3Q7qNzmgzT\n7pJWG+/NOPMT0uQIqRshDbqJ6RJSN0IadBMTVrtL6/dxq3u9yTDtLmm18d6MMj8hTY+QuhHS\nkJuYtNo9Wr+LW93pTYZpd0mrjfdmhPkJCQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKC\nACFBgJAgQEgQICQIEFJSaslNg3GabKq3NUvUCCkntQi0wThNNtXbKlq+IaQcIf1hQoopJfOY\nazBOk021mk79SqmfaumEFCOkv0xIMUL6y4SUk3rENRinyaZaTad+JR01IqQcIf1hQkpKPeAa\njNNkU62mU7+SjBoQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJAgQEgQ\nIKTxtVp9FFprxIduN46QxtZqPWxo9Ssfut44QhqbkCZBSDPX4FN66t/S4EqttvR3db5xhDQy\nIU2CkOZOSJMgpNlrcAfWv6XVva6jO7reOEIam5AmQUjz1+D+q39LqztdRnd0u3GEBAFCggAh\nQYCQIEBIECAkCBg+pP2qlM2h103A0AYM6bqbfn098LXrZRMwkqFD2pXd6eXluCv7PjYBIxk6\npKqczl+fyqqPTcBIhg7p/TSMH04ua7kJGMnQIW3fQ6r62ASMZNCQNg/7Q3l8/fK0u7+3QUjM\nzKAhfZypXkp16mMTMJIhjyM9P+/3m81ll8PubkdCYm6c2fB/U2iwOiW0vKfBYqMBFxL19kMt\nm5C+n0CD9ZKhBacNlr8OuLS1tx9q6YT0/QSENIVhZmSskKZ9HOl9v8i9eTT5nlabql0S2lK7\n2Yw5zJxMJ6RyK7GJDoQ0iWHmxEu7b7cvpCkMMydC+n4CDR4HocdKfZjaJQM+Knv7oZZOSN9P\nQEhTGGZGhgzptC1l/bakb9o7G14cR5rIMLMxYEin6vKv1OY6yNRDgt8YMKTLYr7TvlpfBhES\nSzJgSNX1isdqdRQSCzP0eqRXp/VaSCzMgCGtyvsp36u1kFiWAUPal+3bV8eyFhKLMuTu791H\nPYcfdo4KiZkZ9IDs8+b9q+NWSCyJMxsgQEgQICQIEBIECAkChAQBQoIAIf3fFGa3AogxCen7\nCcxuTSrjEtL3ExASvyKkb7c/u8/tYWRC+nb7QuJ3hPTt9oXE7wjp+wl4j8SvCOn7CQiJXxHS\n/03BcSR+QUgQICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFCgoDlhdTbooT5\nDdzKtGYzG0sLqbdlcvMbuJVpzWZGhLTYgVuZ1mxmZGEh9fZRIvMbuJVpzWZOhLTUgVuZ1mzm\nREhLHbiVac1mThYW0gzfykzrkTut2cyIkBY7cCvTms2MLC2kOR7umdYDd1qzmY3lhQQjEBIE\nCAkChAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFCggAhQYCQIEBIC9ZgaZHVRyFCWqwGi12t\nh40R0mIJaUhCWqoGHwjkM4NyhLRUQhqUkJZKSIMS0mJ5jzQkIS2WkIYkpAVzHGk4QoIAIUGA\nkCBASBAgJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJAgQEgQICQI\nEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQMGtLTw+by+3g2u6e+NgGjGDCk06p8WveyCRjJ\ngCHtSvX4fPnqeKjKro9NwEgGDKkqzx9fP5eqj03ASAYM6csvWfzhd5u23ASMxDMSBAz7Hulw\nvHzlPRJLM+Tu7/XNXrvVqZdNwDiGPY60uxxHqjYPjiOxLM5saKzc30HCnyakht5eko49DSZK\nSA0JiXvGCmlux5E+9pKMPRGmaTohlVuJTURNd2ZMgpd2zQiJu4TUkI64R0gNCYl7hgzptC1l\nfXgbZGY7G14cR+KeIRf2VdflsddB5hcS/L9BT1rdv9a0ry6LY4XEogy6jOLyx7FaHYXEwoyw\nsO+0XguJhRkwpFV5XzqxWguJZRkwpH3Zvn11LGshsShD7v7efdRz+GFXspCYmUEPyD5v3r86\nboXEkjizAQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEARMNCWamxaM8H05Ub/MzcO8Dz27C\nXQYWkoGFFBhYSAYWUmBgIRlYSIGBhWRgIQUGFpKBhRQYWEgGFlJgYCEZWEiBgYVkYCEFBhaS\ngYUUGFhIBhZSYGAhGVhIgYGnHhLMgpAgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQ\nEgQICQKEBAFCgoCphvR0O7HWn2z+j/o4u6pUu1N+4NSMX563pWyPNxeEZlwbODPjbz6FPjPh\n+sCpm/hUm2CrGU80pFN1M7Hn0G1WH2d9+fsqPnBqxi+HyzDV570amnFt4NCM3x/u1ccloQnX\nBk7dxMfqOu7nvyntZjzRkDblS0ibyKC1cZ5K9fzyXJWn9MCpGb9UrxM8bcru/e+pGdcGjs34\n7PA5v9iE/x04NeHt5TbYle37BS1nPM2QHr/8U7MvD5FRa+PsyuGysa7D1wZOzfjxci+fPv8d\nTs24NnBqxmen6vMxnppwbeDUhN8eaJ+Pt5YznmRIx7L+GtI+MmxtnE05P6F3/7etNnBqxtvy\n/PWC1IxrA6dmfLYpp5uvMxOuDZya8Nt7iM9/UlrOeJIhrcvxNqRNOWxf3/51HrY2Tu1fo9TA\nqRmvystDVbafD5/UjGsDp2b8cn4M9nAT1wZOTfjh7aXdxxNQyxlPMaSH8vjyNaSLdddxa+Pk\nQvpn4NSMS9l8feuemnFt4NSMX74+b0RD+jJwbML7896G6vPpbTkhXZ5Vv+5Lfjzvpez8XF4b\nJ/ewrA2cmvF5n8C28z+XTQbOzPh8921v/hYM6d+BQxN+uAT5+Y5oOSGtzjtl6z/HKbDT959x\nkv9cfjPB7jMul7cyx/yMawNfJW7j65v1zw3d/hEc+Kr7hPfnl3av/6R0/dd1eiFtL7fXNz9H\n6PF+M04VDak+TuDx/s84qRn/36MlcFNU5Zu/JW7i6rsxAu8Wz68Xb4JsOePphfRxDLuHO/mf\nca47aI6pQyj5GW/+vVNTM64N/CZx1PvL3HI38ff70fL/pLSc8RxCqi7/aHS/M2rjPFye/A6l\n676f2sCpGV8nePx8S52acW3g1Iz/3S2dmnBt4NyD4vzfmyNqLWc8vZCuvpytdXkZ+91L5N+p\njZM67F4bODXj1zcxp/ML+Mf3C1Izrg2cmvHrP+hfjlDlzmz4Z+Dgg+L0NtrVos5s+PqMe7qe\nDtX5H7Xbca7jrzK7UGsDp2b8tktp/TFwasa1gWMzXn3so85O+N+BYxNeZ27iOYR0OUF3FTiQ\nfTPO58CJo5DfDpyY8cth/T7B7Iy/HTgx48+XEuEJfzdw5Cb+nGCXGU81JJgVIUGAkCBASBAg\nJAgQEgQICQKEBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFBgJAgQEgQICQIEBIECAkC\nhAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFCggAhTcBptypl3ey3z938ct1f/Ebvwy+/n99y\n247v7dehlur08/e2C2lVfvf9/Jrbdnzbsj6+vBzXjX6zcKuQipD65rYdX7n+vu5Towe6kKbJ\nbTu+rw/w/apU+7eLd++/XvuwKW9f/k9IN9c6bkr1cLlwV70+yb1+1+WF4/uAD73+LH+WkMa3\nK9vjx182l0f9+uX8uH94//Lh+iZq9/J/Id1e6/KO65zL+vzF9jaky7c126nB7whpAl4f8avd\n0+XLQ1mfXk7rcrgk8fzyXJXH85ev/3m8hPNtSF+u9frlvqzOF16vXz5f2r3/L+KENAWH7Xmn\n3Xkf9ebyfulUNufH/fmCw/nLq/8P6cu1nt7+1+bt+jchPb14p9QTt+pEPD1U58d5effxiL/+\ncTw8rP8/pNq1/rn+7c4GIfXCrToZz+cXXf8T0vr9QiFNlFt1dB+P7JtH/83l5z+2ZbU/HO+F\n9M+FQhqaW3V0m7f9aKdSvb+xubi+pTmU7duD/05IX671/t9v3iN9vRZBbtXRPZWyP73+sT4H\n9Xje1fayv+42uO51O1yber7zHunLtd7/15e9dscXIfXKrTq+3dsbnPNhoLd3Q9Xxsrf6/OXm\n5jue/gnp453T7bWu/+vjwvOXq3J+thNSj9yqE/C8fX3eWD9e/7J/fdRv355ANq/vjS4Xbl//\n/9Ph8xnn4iakL9f6+O+uer3W+cunlZB65ladrNAj/vpER8+ENFldQ7qcDnHaNDqnnK6ENFld\nQ3o7Qa/KzIb7hDRZnV/a7S/n8EXmwk+EBAFCggAhQYCQIEBIECAkCBASBAgJAoQEAUKCACFB\ngJAgQEgQICQIEBIECAkChAQBQoIAIUGAkCBASBAgJAgQEgQICQKEBAFCggAhQcB/ccSfgT3t\nKigAAAAASUVORK5CYII=",
      "text/plain": [
       "plot without title"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 420,
       "width": 420
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "library(tidyverse)\n",
    "iris.pca <- iris[,-5] %>% prcomp(center = TRUE, scale = TRUE)\n",
    "plot(x=iris.pca$x[,1],y=iris.pca$x[,2], pch=16,xlab=\"PC 1\", ylab=\"PC 2\")\n",
    "plot(x=iris$Sepal.Length,y=iris$Sepal.Width, pch=16,xlab=\"Sepal Length\", ylab=\"Sepal Width\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**PC Loadings**\n",
    "\n",
    "- Observations can be plotted (\"projected\") in space of PC's \n",
    "- Roughly, equivalent to rotating the axes and plotting original data points in the new axes $Z_1,\\, Z_2$.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A matrix: 4 × 4 of type dbl</caption>\n",
       "<thead>\n",
       "\t<tr><th></th><th scope=col>PC1</th><th scope=col>PC2</th><th scope=col>PC3</th><th scope=col>PC4</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><th scope=row>Sepal.Length</th><td> 0.5210659</td><td>-0.37741762</td><td> 0.7195664</td><td> 0.2612863</td></tr>\n",
       "\t<tr><th scope=row>Sepal.Width</th><td>-0.2693474</td><td>-0.92329566</td><td>-0.2443818</td><td>-0.1235096</td></tr>\n",
       "\t<tr><th scope=row>Petal.Length</th><td> 0.5804131</td><td>-0.02449161</td><td>-0.1421264</td><td>-0.8014492</td></tr>\n",
       "\t<tr><th scope=row>Petal.Width</th><td> 0.5648565</td><td>-0.06694199</td><td>-0.6342727</td><td> 0.5235971</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A matrix: 4 × 4 of type dbl\n",
       "\\begin{tabular}{r|llll}\n",
       "  & PC1 & PC2 & PC3 & PC4\\\\\n",
       "\\hline\n",
       "\tSepal.Length &  0.5210659 & -0.37741762 &  0.7195664 &  0.2612863\\\\\n",
       "\tSepal.Width & -0.2693474 & -0.92329566 & -0.2443818 & -0.1235096\\\\\n",
       "\tPetal.Length &  0.5804131 & -0.02449161 & -0.1421264 & -0.8014492\\\\\n",
       "\tPetal.Width &  0.5648565 & -0.06694199 & -0.6342727 &  0.5235971\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A matrix: 4 × 4 of type dbl\n",
       "\n",
       "| <!--/--> | PC1 | PC2 | PC3 | PC4 |\n",
       "|---|---|---|---|---|\n",
       "| Sepal.Length |  0.5210659 | -0.37741762 |  0.7195664 |  0.2612863 |\n",
       "| Sepal.Width | -0.2693474 | -0.92329566 | -0.2443818 | -0.1235096 |\n",
       "| Petal.Length |  0.5804131 | -0.02449161 | -0.1421264 | -0.8014492 |\n",
       "| Petal.Width |  0.5648565 | -0.06694199 | -0.6342727 |  0.5235971 |\n",
       "\n"
      ],
      "text/plain": [
       "             PC1        PC2         PC3        PC4       \n",
       "Sepal.Length  0.5210659 -0.37741762  0.7195664  0.2612863\n",
       "Sepal.Width  -0.2693474 -0.92329566 -0.2443818 -0.1235096\n",
       "Petal.Length  0.5804131 -0.02449161 -0.1421264 -0.8014492\n",
       "Petal.Width   0.5648565 -0.06694199 -0.6342727  0.5235971"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "iris.pca$rotation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A lot of methods (PCA included) can be thought of as decomposing a data matrix $X_{n\\times p}$, into a mixture of $k$ archetypes\n",
    "$$\n",
    " X \\approx U V^{\\top},\n",
    "$$\n",
    "where $V_{p\\times k}$ is a matrix of archetypes and $U_{n\\times k}$ is a matrix of weights (for mixing archetypes to get $X$). One can also think of this as a \"vector space embedding\" (where our weights determine the embedding)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is not easy to choosing the number of principal components. There exist multiple proposals such as using gap statistic, universal threshold, clever cross-validations, etc. None of this methods dominate the rest. The right choice eventually comes down to the needs in a particular analysis project. For instance, the number of PCs to choose should be two if for visualization on a two-dimensional surface. \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAAM1BMVEUAAABNTU1oaGh8fHyM\njIyampqnp6eysrK9vb2+vr7Hx8fQ0NDZ2dnh4eHp6enw8PD////ojgWfAAAACXBIWXMAABJ0\nAAASdAHeZh94AAAWkUlEQVR4nO3d61riyhqF0XAQ2SLg/V/tBj61Qx9cklkSAmP8WA0LsNKm\n3gfIqbs3INaNvQBwD4QEDQgJGhASNCAkaEBI0ICQoAEhQQNCggaEBA0ICRoQEjQgJGhASNCA\nkKABIUEDQoIGhAQNCAkaEBI0ICRoQEjQgJCgASFBA0KCBoQEDQgJGhASNCAkaEBI0ICQoAEh\nQQNCggaEBA0ICRoQEjQgJGhASNCAkKABIUEDQoIGhHR7uq778j43yCq6PUKaIKvo9ghpgqwi\naEBIt+f9Hejwx27erT7v758Xh1vLl9+fup533dOu7u9XhzuLTd15WR4enq9211z2hyWk2/Mr\npGMVH/d3s64szp+6PP3P2SmXj+esjncW78/vXq//V3g8Qro9v0I6ePm4/3S6vT/ksT576rv5\n8e5Ha93hPWl9SG7/9rY6pDbK3+LBCOn2/ArpWMLH/eMnvcMf+/dmPp86O7zhbGYf7cy2p9YO\nT5nX822quA6/5NvzK6RN7/7x3eZp8+dTT9+ZNofH3t6W9Yr9/Hn3x0/jZ/kl355fIe1795/r\nQ9t5S5+VnN6Efm9m97I6fVG6wjI/PL/k2/MrpLP7q/fvP7O/vd2cbpw38zL/+MZ0laV+cH7J\nt+cfIb3tXxa/b7Y7f9c6a+bluAXiab0V0lX4Jd+ef4V0tHk6C6N734a3OW2bW/S/I83fv2IJ\n6Sr8km/P30Oav7/5HD7bnT31uLXhuNVu/dtWu/eXe0e6Dr/k2/P3kA5vOovdaZvDr4MdevuR\nTnV97kdan96eVpWYdXwFfsm35z82NizezkJa9bZAvPaObHj9KMyhDdcgpNvzr+9Ip+9Hi3X/\nfx3/fJl3s1Vtcnjbrw7dLGsL+fbw9NnTdufQhmsQ0qT53HYrrIdJE9KtsB4mTUi3wnqYNCHd\nCuth0oR0K6wHaEBI0ICQoAEhQQNCggaEBA0ICRoQEjQgJGhASNCAkKABIUEDQoIGhAQNCAka\nEBI0ICRoQEjQgJCgASFBA0KCBoQEDQgJGhASNCAkaEBI0ICQoAEhQQNCggaEBA0ICRoQEjQg\nJGhASNDAmCF1t2TE3wN3YNSQ/nc7hERESEVIRIRUhERESEVIRIRUhERESEVIRIRUhERESEVI\nRIRUhERESEVIRIRUhERESEVIRIRUhERESEVIRIRUhERESEVIRIRUhERESEVIRIRUhERESEVI\nRIRUhERESEVIRIRUhERESEVIRIRUhERESEVIRIRUhERESEVIRIRUhERESEVIRIRUhERESEVI\nRIRUhERESEVIRIRUhERESEVIRIRUhERESEVIRIRUhERESEVIRIRUhERESEVIRIRUhERESEVI\nRIRUhERESEVIRIRUhERESEVIRIRUhERESEVIRIRUhERESEVIRIRUhERESEVIRIRUhERESEVI\nRIRUhERESEVIRIRUhERESEVIRIRUhERESEVIRIRUhERESEVIRIRUhERESEVIRIRUhERESEVI\nRIRUhERESEVIRIRUhERESEVIRIRUhERESEVIRMIJtJ533XIzdOyx6+kREpGhE6g7vXDRnawG\n/oyx6+kREpEopFW32r+97VbdetjPGLueHiERiUKadfvj7X03H/Yzxq6nR0hEopC6rndnwM8Y\nu54eIRGJQnr6CGk27GeMXU+PkIgMD2n5vN50L4eb+9XArQ1C4m4MD6mcbs72w37G2PX0CInI\n4Am03a7Xy+Vpk8NqWEdC4n44sqEIiYiQipCICKkIiUiTCWQ/Eo/uh0Lq+v75srHr6RESkStM\nICFx/4RUhERESEVIRAZPoP1T1y3eT+n7emODkLh/QyfQfnbajrCsHyIkHtzQCXQ6mW+/ni1O\nP0RIPLihE2hWL9zN5jshQXbNhsOb0mIhJBg6gebdxyHf84WQeHhDJ9C6e3q/tesWQuLRDZ5A\nq896Nl8cBfTlEELibgyfQNvlx63dk5B4cI5sKEIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJC\nKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJC\nKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJC\nKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJC\nKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJC\nKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJC\nKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJCKkIiIqQiJCJC\nKkIiMnwCvT4vu6Pl6nXgEELibgydQPt598ti2BBC4m4MnUCrbvayPd3abWbdatAQQuJuDJ1A\ns277eXvbzQYNISTuxtAJ1HX/uvP9IYTE3fCOVIREJPiOtNmdbvmOBMM3fy96W+3m+0FDCIm7\nEexHWp32I82Wz/Yj8fAc2VCERERIRUhEhFSERKTJBLIfiUf3QyF1ff982dj19AiJiI92RUhE\nhFSERERIRUhEBk+g/VPXLTbvP8TGBh7c4BP7ZnV6bP0QIfHghh+0uj7UtJ6dTo4VEo9u+GkU\npz92s/lOSJCe2LdfLIQEQyfQvPs4dWK+EBIPb+gEWndP77d23UJIPLrBE2j1Wc/mi6OAvhxC\nSNyN4RNou/y4tXsSEg/OkQ1FSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhF\nSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhF\nSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhF\nSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhF\nSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhF\nSESEVIREREhFSESEVIRE5GwCredvb7t5N3/9uSHOHhi7nh4hEelPoE13uDfrDpqWJCTuX38C\nLbqXt203f3vpFj81xPkDY9fTIyQi/Ql0fEPadqu68TNDnD8wdj09QiLye0jLbiMkuNT5R7vt\nppu9+WgHl/ptY0PXPR/fkDY/NcT5A2PX0yMkIuebv2fHb0hv85efG+LsgbHr6RESETtki5CI\nCKkIicj5BNosT1vudj84RP+BsevpERKRswm0OG5tOPy/WdOShMT960+gdbfYH0Nad08/NcT5\nA2PX0yMkIv0JNOv2tS/WDlm4zO9HNggJBuhPoPn7O9LxwNUfGuL8gbHr6RESkb98R9rMuvVP\nDXH+wNj19AiJyNkEWnal6aF2QuIB/LkfqVu2PUJISDyAcAKt54fw/uMQVyFx/4ZOoNqwt6iP\ngqthQwiJu3E2gfar2eG/s9X+G687vnDVHZ+6W329cUJI3L/+BNrN3vcifeMQodMzj3twD/Zf\nby4XEvfv/AzZp2MY+1W3/O/X9Xfcfr0DV0jcv9+PbDi/8cXrjk95+ghp9t0hzh8Yu54eIRH5\n/Vi7o/23Qlo+rzfdcUv54R3sy60NQuL+9SfQqlscrwz5uviPzXCn17073Zx9uXVCSNy/P89H\n+uaRDdvter1cnjY5/MdWPiFx/84n0MvxyIZF0yPthMQjcM2GIiQiQipCItJkAtmPxKM7m0DP\n897GuEt+yB/P7/r++bKx6+kREpH+BHr+77mfDnH+wNj19AiJyPkO2cbb6/4c4vyBsevpERKR\nvx4i9HNDnD8wdj09QiLSn0DL7hvnT2RDnD8wdj09QiJyfhrFou0/w/znEOcPjF1Pj5CInH+0\ns7EBBhkaUnfuu0OcPzB2PT1CIjJ0Aq2FBL8MnkDb2Xevfick7t9fJ9Drf59qfryw8X+ftfTv\nIU4PjF1Pj5CInE2g1UUbG9bd9vIhzh4Yu54eIRE5P0P2g3/VHC5yfojQy9ui2+0WXdPdSULi\n/v1+iNDz4d1o2/Yq+kLi/v0e0uZ44KodsnCZ82PtXt523fztVUhwmf4E2hwDOl1JyD/GDBc5\nP0O2Lp/63R1EQ4Y4e2DsenqERMTFT4qQiAipCIlI/7r5TqOAgYRUhETER7siJCJn+5Habq37\n2xDnD4xdT4+QiLiKUBESkf4EmruKEAzTn0D7pasIwSCuIlSERERIRUhEbP4uQiIipCIkIsOv\nIpQNcXpg7Hp6hEQkuIrQoCHOHhi7nh4hEXEVoSIkIq4iVIRExFWEipCIuIpQERIRVxEqQiLi\nKkJFSERcRagIicjHBGq6wfvvQ/z5wNj19AiJyOc1G2ar3Q8P8ecDY9fTIyQiHxNofvhmtPiZ\ntyUhcf8+J9BuNTu0tPrevx02bIg/Hhi7nh4hEelPoNenQ0rzdevzzYXE/fttAr0ct34/tf2I\nJyTu3x8TaP98+Lo0+9EhPh8Yu54eIRH52wTaOLIBLuMdqQiJiO9IRUhEzo61s9UOhvmcQK/H\n/Ugz+5FgCEc2FCER+XWs3fOPXPj7TUg8go8J9BMX/f5tiD8fGLueHiERcYHIIiQiQipCIiKk\nIiQiQipCIiKkIiQiQipCIiKkIiQiQipCIiKkIiQiQipCIiKkIiQiQipCIiKkIiQiQipCIiKk\nIiQiQipCIiKkIiQiQipCIiKkIiQiQipCIiKkIiQiQipCIiKkIiQiQipCIiKkIiQiQipCIiKk\nIiQiQipCIiKkIiQiQipCIiKkIiQiQipCIiKkIiQiQipCIiKkIiQiQipCIjJ8Ar0+L7uj5eo/\n/iFnIXH/hk6g/bz7ZTFsCCFxN4ZOoFU3e9mebu02s241aAghcTeGTqBZt/28ve1mg4YQEndj\n6ATqun/d+f4QQuJueEcqQiISfEfa7E63fEeC4Zu/F72tdvP9oCGExN0I9iOtTvuRZstn+5F4\neI5sKEIiIqQiJCJCKkIi0mQC2Y/Eo/uhkLq+f75s7Hp6hETER7siJCJCKkIiIqQiJCJCKkIi\nIqQiJCJCKkIiMvx8pG9t4f5yCCFxN4ZOoLWQ4JfBE2g7+/qSJ98YQkjcjeETaPv16XzfGEJI\n3I1gAq17Z5sPGkJI3A1b7YqQiAipCImIkIqQiAipCImIkIqQiAipCImIkMoXv4fulvzECqIB\nIZWvQhp72XqEdKuEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhF\nSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhF\nSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhF\nSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhF\nSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhF\nSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhFSESEVIREREhF\nSESEVIREREhFSETCNbOed91yM3CIiczQiSwmoxq6ZrrTCxfdyWrYEBOZoRNZTEYVhbTqVvu3\nt92qWw8aYiIzdCKLyaiikGbd/nh7380HDTGRGTqRxWRUUUhd17tz+RATmaETWUxGFYX09BHS\nbNAQE5mhE1lMRjU8pOXzetO9HG7uV19vbRBSO0K6VcNDKqebs/2gISYyQyeymIxq8JrZbtfr\n5fK0yWH1ZUdCakhIt8qRDUVIRIRUhERESEVIRJqsGfuRrkVIt+qHQur6/vmysadlj5CI+GhX\nhERESEVIRIRUhERk+Jp5fV6evgEtV68Dh5jIDJ3IYjKqoWtmP+9tTVgMG2IiM3Qii8mohq6Z\nVTd72Z5u7TYzB61ei5Bu1dA1M+u2n7e3TqO4FiHdquyaDX+78/0hJjJDJ7KYjMo7UhESkeA7\n0mZ3uuU70hUJ6VYNXjOL3la7uRP7rkRItyrYj7Q67UeaLZ/tR7oaId0qRzYUIRERUhESESEV\nIRERUhESESEVIRERUhESESEVIRERUhESESEVIRERUhESESEVIRERUhESESEVIRERUhESESEV\nIRERUhESESEVIRERUhESESEVIRERUhESESEVIRERUhESESEVIRERUhESESEVIRERUhESESEV\nIRERUhESESEVIRERUhESESEVIRERUhESESEVIRERUhESESEVIRERUhESESEVIRERUhESESEV\nIRERUhESESEVIRERUhESESEVIRERUhESESEVIRERUhESESEVIRERUhESESEVIRERUhESESEV\nIRERUhESESEVIRERUhESESEVIRERUhESESEVIRERUhESESEVIRERUhESESEVIRERUpl+SN0t\nSebLNAmp3EFIYy9bj5CuOsREVr3FvJiQrjrERFa9xbyYkK46xERWvcW8mJCuOsREVr3FvJiQ\nrjrERFa9xbyYkK46xERWvcW8mJCuOsREVr3FvJiQrjrERFa9xbyYkK46xERWvcW8mJCuOsRE\nVr3FvJiQrjrERFa9xbyYkK46xERWvcW8mJCuOsREVr3FvJiQrjrERFa9xbyYkK46xERWvcW8\nmJCuOsREVr3FvJiQLvD6vDydDLlcvQ4cYiKr3mJeTEjftp/3TixeDBtiIqveYl5MSN+26mYv\n29Ot3WbWrQYNMZFVbzEvJqRvm3Xbz9vbbjZoiImseot5MSF9/3Xdv+58f4iJrHqLeTEhfZt3\npFFMfzHvVfAdabM73fId6Yqmv5j3avBfedHbajffDxpiIqveYl5MSBd4XZ32I82Wz/YjXc30\nF/NeObKhTH+GTmUxb8h3Zu93CancwQwde9l6prGYQvoB01j1FrOl2wvJfqRrsZgNTSGkb30U\nHemT8d999be7IRazoX8v5eUecPsKtCckaEBI0MAVTuyD+3eFE/vg/l3hxD64f1c4jQLu3xVO\n7IP75x0JGrjCiX1w/65wYh/cvyuc2Af3z2YCaEBI0ICQoAEhQQNCggaEBA0ICRoQEjQgJGhA\nSNCAkKABIUEDQoIGhAQNCAkaEBI0ICRoQEjQgJCgASFBA0KCBoQEDQgJGhASNCAkaEBI0ICQ\noAEhQQNCggaEBA0ICRoQEjQgJGhASNCAkKABIUEDQoIGhAQNCAkaEBI0ICRoQEjQgJCgASFB\nA0KCBoQEDQgJGhASNCAkaEBI0ICQoAEhQQNCggaEBA0ICRoQEjQgJGhASNCAkKABIUEDQoIG\nhAQNCAkaEBI0ICRoQEjQgJCgASFBA0KCBoQEDQgJGhASNCAkaEBI0ICQoAEhQQNCggaEBA0I\nCRoQEjQgJGhASNCAkKABIUEDQoIGhAQNCAkaEBI0ICRoQEjQgJCgASFBA0KCBoQEDQgJGhAS\nNCAkaEBI0ICQoAEhQQNCggaEBA0ICRoQEjQgJGhASNCAkKABIUEDQoIGhAQNCAkaEBI0ICRo\nQEjQgJCgASFBA/8HPpinGyoAG78AAAAASUVORK5CYII=",
      "text/plain": [
       "Plot with title \"iris.pca\""
      ]
     },
     "metadata": {
      "image/png": {
       "height": 420,
       "width": 420
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot(iris.pca)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAAMFBMVEUAAABNTU1oaGh8fHyM\njIyampqnp6eysrK9vb3Hx8fQ0NDZ2dnh4eHp6enw8PD////QFLu4AAAACXBIWXMAABJ0AAAS\ndAHeZh94AAAZ4UlEQVR4nO3d7WKquBqA0aCWWqv2/u92FNtu7bR+4AskYa0fM/acvU2CPIMi\n2vQBPC1NPQGogZAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAg\ngJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAg\ngJAggJAggJAggJAggJAggJAggJAggJDyk1K6+jMZ8hDlR0gF8hDlR0gF8hBBACHl5/MIdPjX\nbpHa75/3r8vDrdXbzz+6XqT0sjv9vG8PPyw3px/eVof/e9Huxpz7bAkpP/9COlbx9fOuSSfL\nyz+66v7Hpsvl68+0xx+Wn38+vY+/hPkRUn7+hXTw9vXzS3d7f8hjffFHPy2OP361lg7HpPUh\nuf3HR3tIbZJVzIyQ8vMvpGMJXz8fn+kd/rX/bOb7jzaHA86m+Wqn2XatHf7I4vTnnaoYh42c\nn38hbc5+Ph5tXjb//6Pda6bN4f/7+Fid/sZ+8br7370xLBs5P/9C2p/9/Hp60nbZ0ncl3UHo\nZzO7t7Z7oTTCnGfPRs7Pv5Aufm4/X/80vx1uuhuXzbwtvl4xjTLrmbOR8/NHSB/7t+XP03aX\nR62LZt6OZyBe1lshjcJGzs9fIR1tXi7CSJ/n8Dbdubnl+WukxedLLCGNwkbOz+8hLT4PPofn\ndhd/9Hi24XjWbv3jrN3nX3dEGoeNnJ/fQzocdJa77pzDv4sdzt5H6ur6fh9p3R2e2lNiHuMR\n2Mj5uXGyYflxEVJ7dgbi/ezKhvevwlzaMAYh5eev10jd66Pl+vx/Ov77bZGa9nTK4WPfHrpZ\nnc6Qbw9/vHnZ7lzaMAYhFc3ztlx4HIompFx4HIompFx4HIompFx4HIompFx4HCCAkCCAkCCA\nkCCAkCCAkCCAkCCAkCCAkCCAkCCAkCCAkCCAkCCAkCCAkCCAkCCAkCCAkCCAkCCAkCCAkCCA\nkCCAkCCAkCCAkCCAkCCAkCCAkCBA/5DeX1fdr1ZctX6zIrPXN6T94t/vAe5+qynMWd+Q2tS8\nbbtbu03T/fJfmLG+ITVp+317e/rd9DBffUO6+AVXftsVc+eIBAGeeI202XW3vEaC/qe/l2dn\n7Rb7yClBeZ54H6nt3kdqVq/eR2L2nCaAAEKCAEKCACEheR+JuRsopHQuYgjI2gh7uZCon5Ag\ngJAgwLN7+bpJi/WwQ0D+eu/l21Vq1h+vd3ywT0jUr+9evu0KatPL/mO3SlePSUKifn338pfj\nFd/t6fMT+7QYYggox3Mf7Eursx+ih4ByPBfS2+k53fUP9gmJ+vV/avfy9SGk/cv1D/YJifr1\n/jqu5vv5XLrxSXMhUb/+e3n7lU9z44Pmfw3hKjzqMdmVDV1FUqIS04U01vAwgqlCStf+TyiN\nkCCAkCCA10gQwFk7CDDhB/u8j0Q9fEIWAggJAggJAggJAggJAkwfks6owPQhKYkKCAkCZBCS\nkiifkCBADiEpieIJCQJkEZKSKJ2QIEAeISmJwgkJAmQSkpIom5AgQC4hKYmiZROSkiiZkCBA\nPiEpiYIJCQJkFJKSKJeQIEBOISmJYgkJAmQVkpIolZAgQF4hKYlCCQkCZBaSkihTbiEpiSIJ\nCQJkF5KSKJGQIEB+ISmJAgkJAmQYkpIoj5AgQI4hKYniCAkCZBmSkiiNkCBAniEpicIICQJk\nGpKSKEuuISmJoggJAmQbkpIoiZAgQL4hKYmCCAkCZBySkiiHkCBAziEpiWIICQJkHZKSKIWQ\nIEDeISmJQmQekpIog5AgQO4hKYkiCAkCZB+SkijBk7vpepHSajPoEEKiAL0vze7+4jJ12kGG\niPr7MLynQmpTu//42LVpPcQQUX8fhvdUSE3aH2/v02KIIeLuAIb2VEgpnf0QPkTcHcDQngrp\n5SukZoghIu8BhtU/pNXrepPeDjf37fWzDUKifv1DOuluNvshhoi9CxhS7110u12vV6vulEN7\ntaOQCpRE3vK/siHsPmA4hYSkJPImJAgQs5cP+z5S4L3AMAYKKZ2LGEJIZK2Up3ZKImtCggDl\nhKQkMtZ779y/pLT8/EjfGCcbhETO+u6d+6Y7j7A63ckoISmJfPXdObsP8+3XzbK7EyExc313\nzub0F3fNYjdaSEoiW899Z8PhoLRcjheSkshV311zkb4u+V4shcTs9d011+nl89YuLUcLSUlk\nqvee2X7Xs7lxFZCQqF//PXO7+rq1exktJCWRp4KubIi/M4hSWkhKIktCggDFhaQkciQkCFBe\nSEoiQ0KCAAWGpCTyIyQIUGJISiI7RYakJHIjJAhQZkhKIjNCggCFhqQk8iIkCFBqSEoiK0KC\nAMWGpCRyIiQIUG5ISiIjQoIABYekJPJRckhKIhtCggBFh6QkciEkCFB2SEoiE0KCAIWHpCTy\nICQIUHpISiILQoIAxYekJHIgJAhQfkhKIgMVhKQkpickCFBDSEpickKCAFWEpCSmVsdeLiQm\nVsleriSmJSQIUElISmJaQoIAtYSkJCYlJAhQTUhKYkr1hKQkJiQkCFBRSEpiOkKCADWFpCQm\nIyQIUFVISmIqQoIAdYWkJCYiJAhQWUhKYhpCggC1haQkJlFdSEpiCkKCAPWFpCQmICQIUGFI\nSmJ8QoIANYakJEYnJAhQZUhKYmxCggB1hqQkRtZ/j3t/XaWjVfs+1BD9CYlx9d3j9ov0z3KQ\nIZ6iJEbVd4drU/O27W7tNk1qhxjiKUJiVH13uCZtv29vUzPEEM9REmPqu7+l9NcPYUM8SUmM\nqNojkpAY0xOvkTa77laer5EmHJc56r23Lc/O2i32gwzxJCExnifeR2q795Ga1WuG7yNNOzDz\nU+mVDdMOzPzUHJKSGI2QIEDIvpbl+0gTD828DBRSOhcxRD9CYiRVP7VTEmMREgSoPCQlMY7e\nO9r+JaXl5vNOcj3ZMPnozEXvD/Y1p4/Hnu5ESMxc/4tW14ea1k334dicQ5p6eOah/8coun/t\nmsVOSPDsB/v2y2XmIU0+PnPQdy9bpK+PTiyWQmL2+u5l6/TyeWuXlnmHNP0EqF/vnaz9rmdz\n4yqgyffjySdA/frvZNvV163dS94hZTADalf7lQ2ZzIDazSKkHKZA3YQEAeYRUhZzoGYzCSmP\nSVAvIUGAuYSUySyolZAgwGxCymUa1ElIEGA+IWUzD2okJAgwo5DymQj1ERIEmFNIGc2E2ggJ\nAswqpJymQl3mFVJWc6EmQoIAMwspr8lQDyFBgLmFlNlsqIWQIMDsQsptOtRBSBBgfiFlNx9q\nICQIMMOQ8psQ5RMSBJhjSBnOiNLNMqQcp0TZhAQB5hlSlnOiZEKCADMNKc9JUS4hQYC5hpTp\nrCiVkCDAbEPKdVqUSUgQYL4hZTsvSiQkCDDjkPKdGOUREgSYc0gZz4zSzDqknKdGWYQEAeYd\nUtZzoyQXe9J68fGxW6TF+3BDZCbnuVGS8z1pkw4/NekgtKSsd9asJ0c5znekZXr72KbFx1ta\nDjVEdrKeHOU435GOB6Rtak83hhkiP3nPjlL8DGmVNkKCR10+tdtuUvMxq6d22U+PMvw42ZDS\n6/GAtBlqiAxlPj3KcHn6uzm+QvpYvA03RIZynx8lmPkbske5z48SCKmACZK/y51os+rO3O0G\nHCJH+c+Q3F3sQ8vj2YbD/9aElpT/bpr/DMnd+T60Tsv9MaR1ehlqiEwVMEXydr4LNWl/ei92\nTm/IdgqYInn7eWXDPEMqYo7k7HwPWnwekY4Xrg40RK5KmCM5++U10qZJ66GGyFYRkyRfFzvQ\nKp2EXmpXxj5axCTJ1//fR0qr2CuECtlHy5gluXJlw6cyZkmuhPSlkGmSp4vdZ982h3827X64\nIfJVyDTJ0/nus2s+30Wa2yVCJ6XMkxxdfkL25Xgs2rdpNdQQWStmouTn55UNlzduWi9SWt34\nPG0x+2cxEyU/P6+1O9rfEdLpjyxP7zu1dw+Rt3JmSm7O9502LY/fDPm+vFFG9/dS9xeO5yV2\n7fUrIcrZPcuZKbn5/+eR7ruyoQvp8xC2v35tXkG7Z0FTJS+Xu87b8cqG5T1X2l1cJX79qWBB\ne2dBUyUvfXedrp2Xr5CaIYaYQklzJSf9Q1q9rjfpeFnevr3+mqqknbOkuZKT/iGddDebq5dC\nFLVzFjVZ8nGx47wuzvq4Ybtdr1er7pTDjUuKito3i5os+TjfcV5Tuj+kfkPkr6zZkovLN2RD\nPxn72xD5K2u25OLXS4SGG6IAhU2XPJzvNqvU8/MT1byP1CltvuTg8mMUy36/PPb/IaVzfec2\nkdLmSw4un9o52XBU3ISZnpD+r7gJMz3f2fCL8mbM1IT0i/JmzNR+3Wfe5/lR838KnDLTuthl\nWq+RTgqcMtO6/ITsl9u/1TxduneIUpQ4Z6Z0eYnQ28cy7XbLdPvtpLWQ4J+flwi9Ho5G23s+\na75t7v2q/SJ3yiInzXR+hrQ5Xrh612uk7R1fkfK/IYpR5KSZzuW1dm8fu7T4eL/vZMM6bR8e\nohxlzpqpnO8vm2NA3TcJze2XMf+izFkzlctPyJ6+0eTe52x9hihHodNmGq5s+FOp82YKQvpT\nqfNmCuffm+/q70vFTpzxCelvxU6c8Xlqd0W5M2dsF+8jxZ6t+22IspQ7c8bmW4SuKXjqjOt8\nV1n0/Rah+4coTMFTZ1znu8p+1fNbhO4fojQlz50x+fKTq0qeO2MS0nVFT57xOP19XdGTZzxC\nuqHs2TMW3yJ0S+HTZxy+ReiWwqfPOPp+i1DPIUpU+vwZQ99vEeo5RIlKnz9j6P0tQv2GKFLx\nC2B4/b9FqNcQRSp+AQzviW8R6jNEmcpfAUPzLUJ3KH8FDM23CN2jgiUwrK9dJPSE9+9DFKyC\nJTCs7+9saNrdwEOUrIY1MKSvPWRxeGW0HOawVMNOWMMaGNL3HrJrm0NL7X1f591viJJVsQiG\nc76DvL8cUlqsoz9vXsc+WMcqGMqP/ePtePb7JfYpXh27YB2rYCj/2z/2r4eXS82gQ5SpkmUw\njN92j40rG35RyTIYhiPS3WpZB0PwGulutayDIVxca+es3VXVLIR43zvH+/F9pMb7SFdUsxDi\nubLhAfWshGj/rrV7HeSLvz9q2v3qWQnRvvaNIb70+8cQFahoKcTyBZGPqGgpxBLSQ2paC5GE\n9JiqFkMcIT2mqsUQR0gPqms1RBHSg+paDVGE9KjKlkMMIT2qsuUQQ0gPq209RBDSw2pbDxGE\n9LjqFsTzhPS46hbE84TUQ30r4llC6qG+FfEsIfVR4ZJ4jpB6qXFNPENIvdS4Jp4hpH6qXBT9\nCamfKhdFf0Lqqc5V0ZeQeqpzVfQlpL4qXRb9CKmvSpdFP0LqrdZ10YeQeqt1XfQhpP6qXRiP\nE1J/1S6MxwnpCfWujEcJ6Qn1roxHCekZFS+NxwjpKTWvjUcI6Sk1r41HCOk5VS+O+wnpOVUv\njvsJ6Ul1r4579d8P3l9X6WjV3vj9s3XvanWvjnv13Q/2i/TPcpAhClH58rhP392gTc3btru1\n2zSpHWKIQlS+PO7Tdzdo0vb79jY1QwxRitrXxz367gUp/fVD2BClqH193MMR6XnVL5DbnniN\ntNl1t+b+GmkGC+S23jvB8uys3WI/yBDFqH+F3PLE+0ht9z5Ss3qd9ftInRksketc2RBhBkvk\nOiGFmMMauUZIIeawRq4J2QPm/T5SZxaL5G8DhZTORQyRu1kskr95ahdkHqvkL0IKMo9V8hch\nRZnJMvmdkKLMZJn8Tkhh5rJOfiOkMHNZJ7/p/3mku89wz2YHm81C+b++D/5aSPBP771821z/\nypOAIaAY/ffy7fWP80UMAaV4Yi9fn33afKAhoBDO2kEAIQWbyUW6/CCkUF1FUpohIYVKZ/9k\nToQUKf34N7MhpEhCmi0hRRLSbAkplNdIcyWkUM7azZWQgnkfaZ6EBAGEBAGENJzZLnyOhDQg\nL5fmQ0iDmvPa50VIw5r14udESAPz9G4ehDS4ua9/HoQ0PAelGRDSGGyC6glpFLZB7YQ0Dk/v\nKieksdgMVRPSaByUaiakEdkS9RLSmGyKaglpVJ7e1UpII7M16iSksTkoVUlI47NBKiSkCdgi\n9RHSFDy9q46QpmGjVEZIE3FQqouQJmO71ERI07FhKiKkCXl6Vw8hTcq2qYWQpuWgVAkhTc3m\nqYKQJmf71EBI0/P0rgJCyoFNVDwhZcFBqXRCyoStVDYh5cJmKpqQsuHpXcmElBFbqlxCyomD\nUrGElBcbq1BCyoytVSYh5cbTuyIJKT82WIGElCEHpfIIKUu2WWmElCcbrTBCypSnd2URUrZs\nt5IIKV8OSgURUs5sumIIKWu2XSmElDdP7wohpNzZfEUQUvYclEogpALYgvkTUglswuwJqQie\n3uVOSIWwFfMmpFI4KGVNSOWwITMmpILYkvkSUkk8vcuWkMpiY2ZKSIVxUMrTkw/LepHSajPo\nEPxge+ao76Ny+g/jMnXaQYbgDzZohp4KqU3t/uNj16b1EEPwF0/v8vNUSE3aH2/v02KIIfib\nbZqbp0L6+i/j9f9CetAH4KCUmadCevkKqRliCK6yWbPSP6TV63qT3g439+31sw0e8WHYrjnp\nH9JJd7PZDzEEN3h6l5Hej8V2u16vVt0ph/ZqR0Iajk2bDVc2FM22zYWQyubpXSaEVDqbNwsh\nD4P3kabkoJSDgUJK5yKG4ApbeHqe2tXAJp6ckKrgsD81IVXCVp5W/+3//rrqXgGt2vehhuAB\nDkqT6rv194uzswnLQYbgQTb0hPpu/DY1b9vu1m7TuGg1D7b0dPpu+yZtv29vfYwiE57eTea5\n72z47YewIejBxp6II1JlHJSm8cRrpM2uu+U1UmZs7yn03urLs7N2Cx/sy4kNPoEn3kdqu/eR\nmtWr95Ey4+nd+FzZUCXbfGxCqpOD0siEVCubfVRCqpbtPiYh1cvTuxEJqWY2/WiEVDUHpbEI\nqXK2/jiEVDubfxRCqp6nd2MQ0gx4BIYnpDlwUBqckObBgzAwIc2ER2FYQpoLT+8GJaT58EAM\nSEgz4qA0HCHNisdiKEKaFw/GQIQ0M57eDUNIs+PxGIKQ5sdBaQBCmiMPSTghzZLHJJqQ5snT\nu2BCmisPSyghzZaDUiQhzZhHJo6Q5sxDE0ZIs+bpXRQhzZxHJ4aQ5s5BKYSQ8AAFEBIeoQBC\nwtO7AELiyIP0JCHRcVB6jpD45HF6hpD44oF6gpD45uldf0LijMeqLyFxzkGpJyFxycPVi5D4\nwePVh5D4ydO7HoTE/3nIHiYkfuGg9Cgh8SuP2mOExO88bA8REn/w9O4RQuJPHrn7CQkCCAkC\nCInbktdLtwiJW7qKpHSdkLglnf2TPwiJG9KPf/MbIXGDkO4hJG4Q0j2ExC0/XiOl5CTe/wmJ\nW347a5c+TTOjDAmJ2/4uJinqREiEmHtRQiLWTIMSEsOY2SFKSAxsHkUJibFUHZSQGFuVhygh\nMZmaihIS06sgKCGRj4IPUUIiQ+UFJSQyVs4hqv8M319X3RJX7ftQQ8Cn7IvqO7H9Iv2zHGQI\n+L9cg+o7oTY1b9vu1m7TpHaIIeBvuR2i+s6jSdvv29vUDDEE3CGTovoOfzHv64sQEmOYNihH\nJCpz7yEqtrknXiNtdt0tr5HI09Wior/1svc9Lc/O2i32gwwBIX4LKvpbL594H6nt3kdqVq/e\nR6IIZzWFf8eYKxuYIyFBACFBhHxeI53fifeRKEw2Z+0u7uR/95LORQwBwfJ4Hwk4IyQIICQI\nMMIH+6B+I3ywD+o3wgf7oH4jfIwC6jfCB/ugfo5IEGCED/ZB/Ub4YB/Ub4QP9kH9nCaAAEKC\nAEKCAEKCAEKCAEKCAFOGlGBKoTtz5J0VNPZwqlyVRY16ZwWNPZwqV2VRo95ZQWMPp8pVWdSo\nd1bQ2MOpclUWNeqdFTT2cKpclUWNemcFjT2cKldlUaPeWUFjD6fKVVnUqHdW0NjDqXJVFjXq\nnRU09nCqXJVFjXpnBY09nCpXZVGj3llBYw+nylVZ1Kh3VtDYw6lyVRY16p3BXAkJAggJAggJ\nAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAggJAkwY0rrCiNeL1LSV/Wrq/UtK\nL9upZzGA98gdcLqdeRv72wCy0Ha/5KCpq6SmW1R9Je2bKkLaNvWFtE0v++OR9mXqiURqj8tp\n02rqeYRbhe6AU+3M67SsL6TVaUV1LaxJxwNsXWs6eov9BUlTbZ/UVvjYfKpxYamZegbBdsH/\nJZ/qMd/Wub8d7dNy6imEa9N66ikEW6ZdFSF91BvSOm2mnkKww7Ogduo5BHtNb7E7oJCi7Zrq\nXpevV016nXoSobbHkydCytm+qe+J3cFLXc/tFse3KISUs+Vi6hkMYl/V2YaX7tm3kPK1Wyx3\nU89hGFU9Wulb3F2G3dPjQ9f00JxsKjxhd3ofaZdqOtIKKW+7Cjs6XdmwX9X1GqnjqV2uXsL/\nO5eD07V2Ff4nQki5in/CkIW2SYv6jkf1hAT1EBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIE\nEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIE\nEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIE\nEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIE\nEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIEEBIE+A9Pb5Hm7FMK\newAAAABJRU5ErkJggg==",
      "text/plain": [
       "Plot with title \"iris.pca\""
      ]
     },
     "metadata": {
      "image/png": {
       "height": 420,
       "width": 420
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot(iris.pca,type=\"l\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To summarize:\n",
    "\n",
    "- Data: $n$ observations living in a $p$-dimensional space\n",
    "- Not all $p$ dimensions are equally useful\n",
    "- Many are either completely redundant (correlated features) or uninformative (noise features).\n",
    "- Need  low-dimensional representation of the variables that captures most of the \"information\" in the data.\n",
    "- To maximize the information retained, we need to minimize the redundancy, and to do this, we look for low-dimensional representations that capture most of the variation in the data.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "4.0.3"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}