week_3/pricipal-component-analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Another explanation about PCA\n",
    "\n",
    "\n",
    "<img src = 'pca.jpeg' width=\"width\" height=\"height\"/>\n",
    "<sub>photo credit: Raunak Joshi</sub>\n",
    "\n",
    "\n",
    "In this lab, we are going to view another explanation about Principal Component Analysis(PCA). PCA is a statistical technique invented in 1901 by Karl Pearson that uses orthogonal transformations to map a set of variables into a set of linearly uncorrelated variables called Principal Components. \n",
    "\n",
    "PCA is based on the Singular Value Decomposition(SVD) of the Covariance Matrix of the original dataset. The Eigenvectors of such decomposition are used as a rotation matrix.  The Eigenvectors are arranged in the rotation matrix in decreasing order according to its explained variance. This last term is related to the EigenValues of the SVD.\n",
    "\n",
    "PCA is a potent technique with applications ranging from simple space transformation, dimensionality reduction, and mixture separation from spectral information.\n",
    "\n",
    "Follow this lab to view another explanation for PCA. In this case, we are going to use the concept of rotation matrices applied to correlated random data, just as illustrated in the next picture.\n",
    "\n",
    "<img src=GaussianScatterPCA.svg>\n",
    "\n",
    "Source: https://en.wikipedia.org/wiki/Principal_component_analysis\n",
    "\n",
    "As usual, we must import the libraries that will use in this lab."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np                         # Linear algebra library\n",
    "import matplotlib.pyplot as plt            # library for visualization\n",
    "from sklearn.decomposition import PCA      # PCA library\n",
    "import pandas as pd                        # Data frame library\n",
    "import math                                # Library for math functions\n",
    "import random                              # Library for pseudo random numbers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To start, let us consider a pair of random variables x, y. Consider the base case when y = n * x. The x and y variables will be perfectly correlated to each other since y is just a scaling of x."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAFrxJREFUeJzt3X9sXWd9x/HPdw5OthCvDY1NSOrZpN66SjUtM7W7sBEoltKASJgQdCRQJtaoGiC6TgNHyQidalH2RyiTulVJYRS1UztB5UZ1CDKFMI0mUROcJWtTSNpCGzdLQulolYlGCd/9cU+6W+fa9zz3nnPvOfe8X9KV74/HPl/dRN/veZ7znOcxdxcAoHh+q9kBAACagwIAAAVFAQCAgqIAAEBBUQAAoKAoAABQUBQAACgoCgAAFBQFAAAKak6zA5jNJZdc4j09Pc0OAwByY//+/b9w90Vx2ma6APT09Gjfvn3NDgMAcsPMfh63LUNAAFBQFAAAKCgKAAAUFAUAAAqKAgAABUUBAICCogAAQEFRAACgoDJ9IxgAhNo0dkj37Xnude/1dc7XxK0rmhNQhtEDANAyBkcnLkj+knTk5GkNb9nV+IAyjh4AgJZw+cYd+vU5n/HzIydPNzCafKAHACD3qiV/VEYBAJBrg6MTJP8aUQAA5Nbg6IROvHImVtu+zvkpR5M/FAAAuRSS/OeYmAVUAReBAeRO/+adevnVc7HazjHp6Jfel3JE+UQPAECuXL5xR+zk3zG3jeQ/C3oAAHLjsg3jOhvzeu+6oW7dvubKdAPKOQoAgMwbm5zSLQ8eiNXWJD17B2f9cVAAAGTa8JZdQTdxfeUjV6UYTWvhGgCAzApN/nd+5CqtuXpJihG1FnoAADJp7bbdsZP/vDbTU6OrUo6o9VAAAGROyJn/HBPJv0YUAACZEnKDl8Qc/3pQAABkRsgNXgz71I8CACATQlb07JjbpoO3rUw5otZHAQDQdL0j44q7nifJPzmJTAM1s5Vm9hMzO2pmI7O0e4eZnTOzDyVxXAD5F5L8+zrnk/wTVHcPwMzaJN0laVjSMUmPm9l2d3+yQrsvS/puvccEkH+V9u6dDfv6Ji+JHsA1ko66+zPufkbSA5JWV2j3GUnflnQygWMCyLG123YHJf/lyxaS/FOQxDWAJZKeL3t9TNJgeQMzWyLpg5LeI+kds/0xM1svab0kdXd3JxAegCzh7t7sSKIHYBXemz6kd6ekz7t71fld7r7V3QfcfWDRokUJhAcgK0j+2ZJED+CYpEvLXi+V9MK0NgOSHjAzSbpE0iozO+vuYwkcH0AOhMzxl0j+jZBEAXhcUp+Z9UqaknSDpI+WN3D33vPPzewbkh4h+QPF0TMyHrst0zwbp+4C4O5nzezTKs3uaZP0dXd/wsxujj6/u95jAMivkOS/fNlC3X/TtSlGg3KJ3Ajm7jsk7Zj2XsXE7+6fSOKYALIvJPn3dc4n+TcY+wEASEVI8u9a0M40zyagAABIXOiwz96NwylGg5lQAAAkZmxyKij5rxvqZtiniVgMDkAi1m7brR89/cvY7dcNdev2NVemGBGqoQAAqFto8meOfzZQAADUJWQdf0n62R3s4JUVXAMAULP+zTtJ/jlGAQBQk7Xbdgct7UDyzx6GgAAEC5npM8fYuD2r6AEACBJ6dy/JP7soAABiCZ3jzyYu2ccQEICqQqd5sq5PPtADADCrWpI/Z/75QAEAMKOxyamg5M+wT74wBASgok1jh4I2bufu3vyhAAC4QOjevczxzyeGgAC8Dsm/OOgBAHjN4OiETrxyJlZbbvDKP3oAACSFJf+uBe0k/xZADwAouLHJKd3y4IHY7ee1GTt4tQh6AECBhSb/jrltemp0VYoRoZEoAECBhST/vs75OnjbyhSjQaMxBAQUUOgc/+XLFrK0QwuiAAAFww1eOI8CABRM3OTPNM/WRwEACiL0Bi+Sf+ujAAAFcNmGcZ2Nv3Uvd/cWBLOAgBbXv3ln7ORvIvkXCT0AoIVdvnGHfn0uXvZnzL94EukBmNlKM/uJmR01s5EKn681s4PR4zEze1sSxwUws56R8djJXyL5F1HdPQAza5N0l6RhScckPW5m2939ybJmz0p6l7u/ZGbXS9oqabDeYwO4UOg0T4lhn6JKogdwjaSj7v6Mu5+R9ICk1eUN3P0xd38perlH0tIEjgtgmtDk37WgneRfYEkUgCWSni97fSx6byaflPSdBI4LoMzY5FRQ8u/rnM+ibgWXxEVgq/BexYFHM3u3SgXgnTP+MbP1ktZLUnd3dwLhAa0vdI4/d/dCSqYAHJN0adnrpZJemN7IzPol3SPpend/caY/5u5bVbpGoIGBgYCZy0AxMccftUpiCOhxSX1m1mtm7ZJukLS9vIGZdUt6SNLH3P2nCRwTgErTPEn+qFXdPQB3P2tmn5b0XUltkr7u7k+Y2c3R53dL+oKkN0n6JzOTpLPuPlDvsYEiGxydCJrmSfLHdOae3VGWgYEB37dvX7PDADKnd2S88oW2GZD8i8PM9sc9wWYpCCBnLttA8kcyKABAToxNTqlnJP4F3465bSR/zIq1gIAcCL3Ba91Qt25fc2WKEaEV0AMAMi40+fd1zif5IxZ6AECGhazmKbF3L8JQAICMCpnpw1LOqAVDQEAG9W/eGTTTh+SPWlAAgIzp37xTL796LnZ7ZvqgVhQAIEMGRydiJ3+Wcka9uAYAZETPyHjstl0L2lnKGXWjBwBkQEjyZx1/JIUCADTZZRvCzvwnbl2RXjAoFIaAgCapZftGzvyRJAoA0ARrt+3Wj57+Zez2LO2ANFAAgAYbHJ3QiVfOxG6/fNlCkj9SQQEAGih0aQfO/JEmCgDQICHJn6Ud0AjMAgIaoH/zzqAzf5I/GoEeAJCy0GEf7u5Fo9ADAFLUMzIeO/mztAMajQIApKSXu3uRcRQAIAUhyzkvX7aQu3vRFFwDABI0Njmlv37wQOzk39c5nx280DQUACAhtezdy5k/mokCACQgNPlzsRdZQAEA6hSyro9Jepbkj4ygAAB1GN6yS0dOno7VtmNumw7etjLliID4mAUE1Gjttt0kf+QaPQCgBpdtGNfZmFN95rUZyR+ZRA8ACDA2OaWekfjJv2Num54aXZVuUECNEukBmNlKSV+V1CbpHne/Y9rnFn2+StL/SvqEu/84iWPP6OC/SQ/dlOohUCwuabVLq+fG/AUrXfTVF1MLCa2q913SjdtTP0zdPQAza5N0l6TrJV0h6c/N7Ippza6X1Bc91kv653qPOyuSP1JgkswCHs0OGPn17A+lez+Q+mGSGAK6RtJRd3/G3c9IekDS6mltVkv6ppfskXSRmS1O4NiVPfr3qf1pAGiIZ3+Y+iGSKABLJD1f9vpY9F5oG0mSma03s31mtu/UqVO1RfSrY7X9HgAUSBIFoFJPd/olsjhtSm+6b3X3AXcfWLRoUW0R/e7S2n4PAAokiQJwTNKlZa+XSnqhhjbJue4Lqf1pAGiI3nelfogkCsDjkvrMrNfM2iXdIGn65evtkj5uJUOSfuXuxxM4dmX9H5b+bFtqfx6tzSW5x3w0O1i0pgbNAqp7Gqi7nzWzT0v6rkrTQL/u7k+Y2c3R53dL2qHSFNCjKk0D/Yt6j1tV/4dLDyBA8NIOX+QGL+RXIvcBuPsOlZJ8+Xt3lz13SZ9K4lhAWkL27p1j4u5e5B53AqPwzt/dGzf5d8xt09EvsaIn8o8CgEIbm5zSLQ8eiN2edX3QSlgMDoUVso6/xA5eaD0UABRSyGqeknTnR67Smqsr3rsI5BZDQCic/s07Sf6AKAAomOEtu/Tyq+dityf5o5UxBITC6BkZD2rPxu1odfQAUAghyb9rQTvJH4VAAUDLC0n+y5ct1N6NwylGA2QHBQAtbXB0InbbdUPduv+ma1OMBsgWCgBa1qaxQzrxyplYbdcNdev2NVemHBGQLVwERsvZNHZI9+15Lnb75csWkvxRSBQAtJTQu3uXL1vIsA8KiwKAljE4OhF7yEdi2AegAKAlhCzlbJK+wg1eAAUA+Tc4OhE7+XPWD/w/CgByrX/zzthLO8xrM5I/UIYCgNwKWdHTJD01uirVeIC84T4A5FLvSPzk37WgXc+ytANwAQoAcqdnZFxxV3NeN9TN0g7ADBgCQq6ErOvDUs7A7CgAyIXQG7zWDXWT/IEqKADIvNAbvFjaAYiHawDItOEtu4Lv7mVpByAeegDILDZuB9JFAUAmXb5xR1DyZwcvIBxDQMictdt2B63rQ/IHakMBQKaEzvbhBi+gdgwBITNCx/w58wfqU1cPwMwWmtmEmR2Jfl5coc2lZvYDMztsZk+Y2WfrOSZaU0/A0g4M+wDJqHcIaETSo+7eJ+nR6PV0ZyX9jbv/oaQhSZ8ysyvqPC5axNjkVNDdvR1z2xj2ARJS7xDQakkrouf3Stol6fPlDdz9uKTj0fNXzOywpCWSnqzz2Mi54S27dOTk6djtmeYJJKveHkBXlODPJ/rO2RqbWY+kqyXtrfO4yLnQ5M/SDkDyqvYAzOx7kt5c4aONIQcyszdK+rakW9z95VnarZe0XpK6u7tDDoGcCNnERWJpByAtVQuAu793ps/M7ISZLXb342a2WNLJGdq9QaXkf7+7P1TleFslbZWkgYGBgDkhyIOQvXslhn2ANNV7DWC7pBsl3RH9fHh6AzMzSV+TdNjdt9R5POTY8JZdQcmfmT5Auuq9BnCHpGEzOyJpOHotM3uLme2I2iyX9DFJ7zGzA9GDvfkKJnTMn+QPpK+uHoC7vyjpugrvvyBpVfT8P1Sauo2CCh3zJ/kDjcFSEEhVSPLvWtBO8gcaiKUgkIqxySnd8uCB2O27FrSzdy/QYBQAJG7T2CHdt+e52O1J/kBzUACQqNDVPPs652vi1hXpBQRgRhQAJGbT2KHYyd8kfYU5/kBTUQCQmLjDPh1z23TwtpUpRwOgGgoA6hYy5k/yB7KDAoC6DI5O6MQrZ2K1nWMi+QMZQgFAzULW9eHMH8geCgBq0jsyrrir+ixftlD333RtqvEACEcBQLCQHbzWDXWzlDOQUSwFgSCXbYif/Ps655P8gQyjACC2yzbE37h9jokbvICMYwgIVYXe3TuvzfTUKCt+A1lHAcCsQtfx54IvkB8UAMwoZI6/xAVfIG8oAKiIvXuB1kcBwAVCkv8ck45+iU1cgDxiFhBep3/zTpI/UBD0APCakGmeLO0A5B8FAJLC7u5lUTegNVAACi50+0bO/IHWwTWAAqtl716SP9A6KAAFFZr855jYuB1oMRSAAqrlzJ/ZPkDr4RpAwYxNTgUlf27wAloXBaBAxiandMuDB2K3/9kdnPUDrYwhoIIISf4dc9tI/kAB0AMogJBF3ZjmCRQHBaDFhazr07WgnZk+QIHUNQRkZgvNbMLMjkQ/L56lbZuZTZrZI/UcE/ENjk7ETv7Lly0k+QMFU+81gBFJj7p7n6RHo9cz+aykw3UeDzFsGjuknpHx2MM+bOICFFO9BWC1pHuj5/dKWlOpkZktlfQ+SffUeTxUsXbb7uClHUj+QDHVWwC63P24JEU/O2dod6ekz0n6TbU/aGbrzWyfme07depUneEVS+jevX2d87ngCxRY1YvAZvY9SW+u8NHGOAcws/dLOunu+81sRbX27r5V0lZJGhgYiL8lVcH1b96pl189F6stm7YDkGIUAHd/70yfmdkJM1vs7sfNbLGkkxWaLZf0ATNbJWmepA4zu8/d19UcNV5neMuu2MmfmT4Azqt3CGi7pBuj5zdKenh6A3ff4O5L3b1H0g2Svk/yT87lG3foyMnTsdp2zG0j+QN4Tb0F4A5Jw2Z2RNJw9Fpm9hYz21FvcJhdz8h47GmeJjZxAfB6dd0I5u4vSrquwvsvSLpgkNndd0naVc8xURK6gxereQKYjrWAcigk+XfMbSP5A6iIApAzIcmfHbwAzIYCkBNjk1NByZ+lHQBUw2JwOTC8ZVfsmT6StG6oW7evuTLFiAC0AgpAxoXc4CWxgxeA+CgAGRYy5COxgxeAMFwDyKhekj+AlFEAMujyjTsUdxEkE8kfQG0oABnTv3ln7Lt7JelZkj+AGlEAMmRwdCL2BV/O/AHUi4vAGdE7Mh572IeN2wEkgR5ABgyOTsRO/mziAiApFIAMiLt3b1/nfE3cuiLdYAAUBgUgJ0j+AJJGAciBdUPdJH8AieMicAZ0LWivOAxkYpongPTQA8iAvRuH1bWg/XXvdS1oJ/kDSBU9gIxg6WYAjUYPAAAKigIAAAVFAQCAgqIAAEBBUQAAoKAoAABQUBQAACgoCgAAFJS5x999qtHM7JSknzc7jhlcIukXzQ4ihrzEKeUn1rzEKRFrGrIe5++5+6I4DTNdALLMzPa5+0Cz46gmL3FK+Yk1L3FKxJqGvMQZB0NAAFBQFAAAKCgKQO22NjuAmPISp5SfWPMSp0SsachLnFVxDQAACooeAAAUFAUgJjNbaGYTZnYk+nnxDO0uMrNvmdlTZnbYzK7NYpxR2zYzmzSzRxoZY9nxq8ZqZpea2Q+i7/IJM/tsA+NbaWY/MbOjZjZS4XMzs3+MPj9oZm9vVGwVYqkW69ooxoNm9piZvS2LcZa1e4eZnTOzDzUyvmkxVI3VzFaY2YHo/+YPGx1j3dydR4yHpH+QNBI9H5H05Rna3SvpL6Pn7ZIuymKc0ee3SvpXSY9k9TuVtFjS26PnCyT9VNIVDYitTdLTkt4a/Tv+5/TjSlol6Tsq7d45JGlvk77HOLH+saSLo+fXNyPWOHGWtfu+pB2SPpTh7/QiSU9K6o5edzYj1noe9ADiW61Sclf0c830BmbWIelPJX1Nktz9jLv/T8MiLKkapySZ2VJJ75N0T4PiqqRqrO5+3N1/HD1/RdJhSUsaENs1ko66+zPufkbSA1G85VZL+qaX7JF0kZktbkBs01WN1d0fc/eXopd7JC1tcIxSvO9Ukj4j6duSTjYyuGnixPpRSQ+5+3OS5O7NjLcmFID4utz9uFRKSpI6K7R5q6RTkv4lGlq5x8zmNzJIxYtTku6U9DlJv2lUYBXEjVWSZGY9kq6WtDf1yEpF5vmy18d0YeGJ06YRQuP4pEo9l0arGqeZLZH0QUl3NzCuSuJ8p78v6WIz22Vm+83s4w2LLiHsCVzGzL4n6c0VPtoY80/MkfR2SZ9x971m9lWVhjb+LqEQJdUfp5m9X9JJd99vZiuSjK3Cser9Ts//nTeqdFZ4i7u/nERs1Q5Z4b3pU+bitGmE2HGY2btVKgDvTDWiyuLEeaekz7v7ObNKzRsmTqxzJP2RpOsk/bak3Wa2x91/mnZwSaEAlHH39870mZmdMLPF7n486uZX6u4dk3TM3c+foX5LpQKQtTiXS/qAma2SNE9Sh5nd5+7rMhirzOwNKiX/+939oaRjnMExSZeWvV4q6YUa2jRCrDjMrF+lIb/r3f3FBsVWLk6cA5IeiJL/JZJWmdlZdx9rTIivifvv/wt3Py3ptJn9u6S3qXSdKhcYAopvu6Qbo+c3Snp4egN3/29Jz5vZH0RvXafSRaJGihPnBndf6u49km6Q9P00kn8MVWO1Uib4mqTD7r6lgbE9LqnPzHrNrF2l72n7tDbbJX08mg00JOlX54e0GqxqrGbWLekhSR9r4hlq1Tjdvdfde6L/m9+S9FdNSP5SvH//hyX9iZnNMbPfkTSo0jWq/Gj2Vei8PCS9SdKjko5EPxdG779F0o6ydldJ2ifpoKQxRTMvshZnWfsVat4soKqxqjRU4dH3eSB6rGpQfKtUOpt7WtLG6L2bJd0cPTdJd0WfH5I00MT/n9VivUfSS2Xf4b4sxjmt7TfUpFlAcWOV9LcqneT9l0rDk02JtdYHdwIDQEExBAQABUUBAICCogAAQEFRAACgoCgAAFBQFAAAKCgKAAAUFAUAAArq/wA2fD7Xi+A8vQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "n = 1  # The amount of the correlation\n",
    "x = np.random.uniform(1,2,1000) # Generate 1000 samples from a uniform random variable\n",
    "y = x.copy() * n # Make y = n * x\n",
    "\n",
    "# PCA works better if the data is centered\n",
    "x = x - np.mean(x) # Center x. Remove its mean\n",
    "y = y - np.mean(y) # Center y. Remove its mean\n",
    "\n",
    "data = pd.DataFrame({'x': x, 'y': y}) # Create a data frame with x and y\n",
    "plt.scatter(data.x, data.y) # Plot the original correlated data in blue\n",
    "\n",
    "pca = PCA(n_components=2) # Instantiate a PCA. Choose to get 2 output variables\n",
    "\n",
    "# Create the transformation model for this data. Internally, it gets the rotation \n",
    "# matrix and the explained variance\n",
    "pcaTr = pca.fit(data)\n",
    "\n",
    "rotatedData = pcaTr.transform(data) # Transform the data base on the rotation matrix of pcaTr\n",
    "# # Create a data frame with the new variables. We call these new variables PC1 and PC2\n",
    "dataPCA = pd.DataFrame(data = rotatedData, columns = ['PC1', 'PC2']) \n",
    "\n",
    "# Plot the transformed data in orange\n",
    "plt.scatter(dataPCA.PC1, dataPCA.PC2)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, what is the direction in which the variables point?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Understanding the transformation model pcaTr\n",
    "\n",
    "As mentioned before, a PCA model is composed of a rotation matrix and its corresponding explained variance. In the next module, we will explain the details of the rotation matrices. \n",
    "\n",
    "* `pcaTr.components_` has the rotation matrix \n",
    "* `pcaTr.explained_variance_` has the explained variance of each principal component"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Eigenvectors or principal component: First row must be in the direction of [1, n]\n",
      "[[ 0.70710678  0.70710678]\n",
      " [ 0.70710678 -0.70710678]]\n",
      "\n",
      "Eigenvalues or explained variance\n",
      "[1.66779614e-01 1.57631500e-34]\n"
     ]
    }
   ],
   "source": [
    "print('Eigenvectors or principal component: First row must be in the direction of [1, n]')\n",
    "print(pcaTr.components_)\n",
    "\n",
    "print()\n",
    "print('Eigenvalues or explained variance')\n",
    "print(pcaTr.explained_variance_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$cos(45^o) = 0.7071$\n",
    "\n",
    "The rotation matrix is equal to:\n",
    "\n",
    "$$R = \\begin{bmatrix} cos(45^o) & sin(45^o) \\\\ -sin(45^o) & cos(45^o) \\end{bmatrix}$$ \n",
    "\n",
    "And $45^o$ is the same angle that form the variables y = 1 * x.\n",
    "\n",
    "Then, PCA has identified the angle in which point the original variables.\n",
    "\n",
    "And the explained Variance is around [0.166 0]. Remember that the Variance of a uniform random variable x ~ U(1, 2), as our x and y, is equal to:\n",
    "\n",
    "$$Var(x) = \\frac {(2 - 1)^2}{12} = 0.083333$$\n",
    "    \n",
    "Then the explained variance given by the PCA can be interpret as\n",
    "\n",
    "$$[Var(x) + Var(y)  \\ 0] = [0.0833 + 0.0833 \\  0] = [0.166 \\ 0]$$\n",
    "\n",
    "Which means that all the explained variance of our new system is explained by our first principal component. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Correlated Normal Random Variables.\n",
    "\n",
    "Now, we will use a controlled dataset composed of 2 random variables with different variances and with a specific Covariance among them. The only way I know to get such a dataset is, first, create two independent Normal random variables with the desired variances and then combine them using a rotation matrix. In this way, the new resulting variables will be a linear combination of the original random variables and thus be dependent and correlated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "angle:  45.0\n",
      "rotationMatrix\n",
      "[[ 0.70710678  0.70710678]\n",
      " [-0.70710678  0.70710678]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3X+MXNd1H/DvmeGjOEu7GgrawuZYazKuQcb6QW60kJnuPyWrVmpkK2v9MCNIadAUIAI0QCUIRFaVWlGtAi2wsOUCCdASsJEUEhTKlbyWTBu0jGVghAklk96laUZkalm2xFFQMRVXsbQjcnbn9I/dN5x9837cN+++eTNvvh9AkHZ29s19S/HMnXPPPVdUFURElB+FrAdARER2MbATEeUMAzsRUc4wsBMR5QwDOxFRzjCwExHlDAM7EVHOMLATEeUMAzsRUc6sy+JFr732Wt2yZUsWL01E1LdOnjz5D6o6HPW8TAL7li1bcOLEiSxemoiob4nIL02ex1QMEVHOMLATEeUMAzsRUc4wsBMR5QwDOxFRzjCwExHlTCbljkREvWBmrorpI+fwzkINm8sl7L9tGyZGK1kPKzEGdiIaSDNzVTzy4mnU6ssAgOpCDY+8eBoA+j64MxVDRANp+si5ZlB31erLmD5yLqMR2cPATkQD6Z2FWqzH+wkDOxENpM3lUqzH+wkDOxENpP23bUPJKa55rOQUsf+2bRmNyB4unhLRQHIXSFkVQ0SUIxOjlVwEci+mYoiIcoaBnYgoZxjYiYhyhoGdiChnGNiJiHImcWAXkQ0i8pqInBKRMyLyhI2BERFRZ2yUO14CsEdVPxARB8Bficj3VPW4hWsTEVFMiQO7qiqAD1a/dFb/0aTXJSKizljJsYtIUUTmAbwL4BVVfdXGdYmIKD4rgV1Vl1V1J4BPAbhFRG7wPkdE9onICRE5ceHCBRsvS0REPqxWxajqAoC/BHC7z/cOquqYqo4NDw/bfFkiImphoypmWETKq/9dAnArgLNJr0tERJ2xURXzSQB/LiJFrLxRPK+q37FwXSIi6oCNqpifABi1MBYiIrKAO0+JiHKGgZ2IKGcY2ImIcoaBnYgoZxjYiYhyhoGdiChnGNiJiHKGgZ2IKGcY2ImIcoaBnYgoZxjYiYhyhoGdiChnGNiJiHKGgZ2IKGds9GMnIqIQM3NVTB85h3cWathcLmH/bdswMVpJ7fUY2ImIUjQzV8UjL55Grb4MAKgu1PDIi6cBILXgzlQMEVGKpo+cawZ1V62+jOkj51J7Tc7YiajrqYJB8s5CLdbjNnDGTjTg3FRBdaEGxZVUwcxcNeuh5cLmcinW4zYwsBMNuCxSBYNk/23bUHKKax4rOUXsv21baq/JVAzRgMsiVTBI3JQWq2KIqGs2l0uo+gTxNFMFg2ZitNLVNQumYogGXBapAkpX4hm7iFwH4H8B+ASABoCDqvrfk16XiK5Is2oli1QBpctGKmYJwMOq+mMR+TiAkyLyiqr+rYVrE/U104Ac9rxubHDpdqqA0pU4FaOqf6+qP179718BeB0A/w+hgWdaRhj1PFatUFxWc+wisgXAKIBXbV6XqB+ZBuSo57FqheKyVhUjIh8D8AKAB1X1H32+vw/APgAYGRmx9bJEPcs0IEc9r1+qVrh7tXdYmbGLiIOVoP6sqr7o9xxVPaiqY6o6Njw8bONliXqa6Y7DqOf1Q9UKd6/2lsSBXUQEwNcBvK6qX00+JKJ8MA3IQc/bvX0Y41OzeOjQPK5aV8CmIQcAUBRppmp6JXByHaC32EjFjAP4XQCnRWR+9bH/pKrftXBtor5lWkbo97zd24fxwslqM1gu1OpwigKnIKg3FEB32r+a4jpAb0kc2FX1rwCIhbEQ5Y5pGaH3eeNTs20z4Pqytv2cOyvOOrD3yzrAoODOU6IeFGema3tWPDNXxfjULLZOHsb41KxRuqcf1gEGCXvFEPWgoBlw0HM74VfFAqCjzVBJd692WlHDShx/otr+8S5tY2NjeuLEia6/LlG/8O42BQCnKICimWMHAKcg+NiGdVhYrMcOiN7rl5wirlpXwEKt3vb8SrmEY5N7Et5VvLE8ddeNAILfLMJ+Lq/BXUROqupY1PM4YyfqQUEz4NbHri45+PDyEi4urgTiOIupQVUs3sdcbrrHnSFXF2ooimBZFZWEM+WgsRx46QwuLTUCPz2EVeLkNbCbYmAn6lFBC6/uY+NTs22za9PAFjcvv7lcapshL6ud6pygsfh9cmi9P1biBOPiKVGfShLYgvLym4acwEVQvxmyK0nNetw1gupCDeNTswhKIrMSh4GdqG8lOUvTr4pFAFxcrGODU0C55ECwklt3c9ZRi7mdzpSDKmrcDVleAgSOhZU4K5iKIepTu7cP49njb62ZuZoGNjdl8sTLZ5o5evc6FxfrKDlFPL1355qFSml5jp9OZ8ph6wnexdGwMSTN9ecJAztRH5qZq+KFk9U1QU4A3H1z9Iao1gXQoJ2Ftfoynnj5zJqgGxbUk86UwzZy+b35eAmQWtVOP2IqhqgP+eW7FcDRsxdCf661WZf7M0EuLtabm5PC0jCt6Zo0fFRvRD6HefW1OGMnsqSTzTIzc1UceOlMswJkyCngKqcYWpc+M1cNDLRRee6wBdCg50+MVpqljV5FkVRnyibjZV69HQM7UQLetIYb+lpLAAH41n7v3j6MQ6+9vWbD0WK9gcXVGapfGaE74w5SEMHWycOBbwpxFzjd5/sF9bDHbQkbrwDcbRqAgZ3IkHdG7u3A6A1xbp76o3rDt/bbu/Dpx1uXHjWDjaotj9OqwH0+sJJu8fu5SsopkKDxprkTNg+YYycy4HeQxLPH34pME1xcrAc+x3Su69Ztz8xVY824/WrL/UoLg8jq84N+rhspEDYX6wwDO5GBoMXKbnFn4OWA2u4g3jeCidEK7r65EtlnWwDcv2ukOdufGK3gqbtuRKVcaqtvT1NWr9vvmIohMtAL29Rr9WVcta6AklM0bt7lVy1y9OwF3zeloggaqqEHgmQRULN63X7GwE7kw5tPLw85zVrqVlGbdsKUnCLuvrmCF06eR82gpA9Y6Z/ytb0711TSbHAKuOOmT67J97vX90tZBL1JLaviF1N3dHAnybD1rn1MxRB5+OXTP/hoaaVtbouSU8T9u0aaaYI43JTC2KevQZwDyIqy8txLS1feCC4u1vHCySruvrlilLIIqvkWoOtnqPIQ7HSwHzuRx/jUrG8lRrnkYONV6wJnllsmDxtdXwC8uTozHv2v3/f9JBAmrELFrRQJmwXPzFXx4KH5tp/3XiPqOjYE/a5Z9eKP/diJOhSUqni/Vsf84/+6+bV7hJzbG92UO2N+bOZ07KC+aciJ7Oroba/rzoJP/PI9HD17IbTcsfXaftd56NA8Hjw0b60vC1vvpoOBnXItasbp9/2wg5mDNiT5LVz6cUsIZ+aqeOb4W7HvRzX64OigAyhM6uZb0zRhlUBJe7C3vh4PwbaPOXbKraj8bdD3d28f9q2d3r192LjPShDFSiB84uUzHd3TQq2OxctLcArt+X53oTRotmsyXvdNJyhF0ipJD/bW12Odun0M7JRbYUenhX3/meNv+fYkP3r2Qqw+K37cnZpxUzCtLi7WUW9oc8m1KNK8r5m5asez3fJqOqn1zStK0pQJ69TTYSUVIyLfAPAFAO+q6g02rkmUdOEuKn8bFpQuLtbbShlt5H13bx9OfA2XO7bWNgIPHpqHU1g5+Lq+fGX0JmWZIsCBl87EevOykTJhnbp9tmbsfwbgdkvXIrJSBhd1wlBUUPLmk+Pu+vTzwskqZuaqKDnpfViuN4DlhmLT0JVPHPfvGolsJXBxsR66VuAtymTKpHdZ+b9LVX8I4D0b1yICotMoJvzyt05BsHh5CVsnD+PDS+216UFq9WWowrjPSth1po+cw4aE14nS0JVAXRDB7u3DeHLixraUR9DRc34q5RKe3ruTKZM+waoY6kk2yuC8R65dXXLw4eWlZn7bnZ1uXF/Eh5ej0w/v1+p4eu/OZlWMl2AlBXJ5OTzpEae7YlLLqs3qmycn1gbirYZ19+7M3E2ZuCmyhw7NY/rIOe4U7UFdWzwVkX0ickJETly4EH7KC1Hcg5rdSo6tk4ebnRCBleB+bHIP3py6AxuvWrcm7+xyg3rU3H1zuYSJ0Upgh0QFIoN6Vp579e22x8JSUUUR35k5d4r2h64FdlU9qKpjqjo2PGxvAYnyKU4ZnGmwiZrtK64E97B8ctxTiHqB34EYYfnxhirenLoDxyb3NGfp41OzePDQfOIUGaWP5Y7Uk+KUwZnm400qOBTR+eRuplJscXvMtJoYrTRLHL1af1fec1L9cKdob7FV7vgcgH8B4FoROQ/gcVX9uo1rUz50UrroLYNr3cLfeg3TfPz+27bhoUPzkWV/1YVaYAneYzPBx9L1sl2/tsn38QN3Xr+mbQDQ/snI5BMKd4r2FiuBXVXvs3Edyqeg3iWA+Xb0sGvE2ZY+ZLBQ6nY59DtEupM2AN0kAP7px9fj//7q8prHX3vzou89eReY/d50o2bjLHvsPezuSKE66bXiDR42OviFXWP39uG2gOsUBdP37GiO5bGZ00a9UlwiK31ZXOWSg0tLy5F909cXBcMf39CswvnVpSUsN3pjQbVcctY0MTMV1l7AVjMwMsPujpRY1EzbdCYeNONzz/I0Sc+EXePQa+0VH63BdGauGiuoA2uDOmDe5Gto/bo1b1aPzZzumVm+6T147b9tW1u6JskBI5Q+BnYKFLYoOTFaify+KyhVIriyENn6puC+tknHxaII6j4z4oZizTi7FYTer9XXfIrJQ/BrTdd4u1ra6vJIdrEqhgJ12mvFb9EyqO67Va2+jAMvnTHuuCjwL+NzVRdq+Nx//l5Xq1jWFYAHD803x99L4uw09XL3A1TKJd8/N5Y79hYGdlqjdaNPwadEDojuteJ9fGK0grtvrhgdALdQq/t+Cjh69kKz/BEwTwUsGp4lakuXX86XUxAUPW19naLg8S9en/jaPBijPzCwU5N3o4/fbLi1AiLOJqKjZy8kmsFWF2qYPnIOu7cPoyjSc7PhXrFpyMH0vTvwlXt3rKnDb11ITiLujmDKBnPs1BRUr1wUQUO1bYHTpFSu9cShpKoLtZ5ZiOxVQ+vXtf352OS3kMpyx97DwE5NQR+n3e3lfsJ6aXurZih9aadE/N7Md28fbjYFS+PAa4qPgZ2abJ0/aXOWTvF0IyXS+mZuY/MZ2cfATk1JPmYHHfJM3ZNFSsS05JW6i4Gdmkxy5l4zc1U88fKZNWd4Mqh3R7nkQARYWKxnlgJhlUxvYmCnNaJy5t7c6gsnq8yhZ0CAjtoD2GYrfUd2MbBTG7/+LwDacqmsUMlOrwROVsn0JgZ2AhCcI3cXw65aV+h4Zs6cu11OQXomcHaSvqP0MbAPGJPZuN+W8bhB3Q3mFaZs7AvYwttJz3sbwtJ3lA0G9gESVJqWZDYeRLGyC3L39mEcPXsBtfoyiiKhvV2ond/vrL6sbVUnLDukVmwpMECCStM6beca5eJiHc8cf6u5uMagbq7kFPG1vTvRCPideatOTI8HpMHAwD5AOi1BKznFRJ0ByZ/fOaSuu29eSW+Y9mZh2SG1YiomBVnlOqPGEXQs3Mb1RTQUvgcpVALy8JTMA7tGMPbpawLPYD169gIA86oTlh1SK87YLfN2SHRznTNz1czHEXTWp1MsNFviut0An967E7+YugPHJvfgxC/fw8PPn1oTXMJmmxSsKIIHdo3gyYkbMTFaCawWap1pb3Cu/DUtlxw8ddeNbROFOJ02Kf84Y7fM9hbrTmf/JifLu96v1dv6f7hNnTY4Bd9zPnf92ib8+K33OYOPwf30M33kHLZOHsbmcgmbhpw1u3Zdm8sl3yZql5b8G76z7JBaMbBbZjPXmaTSIc7rtX5c975m0OHNx39+Efd9/rrYZ4kOsupCbU3qpbpQg1MQOEVBffnKb9GdacedJLDskFxMxVhm8yCCJJUOQa/nTaB4P66bzvSXVfEMg3ps3t9XvaHYuH7dmjSYm2rhgih1ijN2y2xusU7yF9tvHCtjKeAqpxjYOIpBo/ver9V9+75wQZQ6ZWXGLiK3i8g5EfmZiEzauGa/mhittC1E+i12mUgy+3fHUS6tLVNcrDewsFgPnGkzaHRf0O+cC6LUqcQzdhEpAvhTAP8KwHkAPxKRl1T1b5Neu1/ZynWazP7DFlcnRiuYPnKubQOStw+M+9yg16T0hAVqLohSp2ykYm4B8DNV/TkAiMhfAPhtAAMb2G2J+ottsrgalVrxLsa5/374+VPcKZqyikGg5oIodcJGYK8AeLvl6/MAPu99kojsA7APAEZGRiy87GAI+4ttUjURlKdt5Q3+E6MVPHRoPsGoKYgAuH+1jt2rVza2Uf+zEdj9dqq0TfVU9SCAgwAwNjbGqaAFJourJqkVvxyvyRsCxdO6i3d8ajay3z2beFGnbCyengdwXcvXnwLwjoXrUoSwxdWZuSrGp2abm4zcRdSockfX/tu2BXWHpQ4URXBscg8A+O5MfvRb7W++bOJFnbIR2H8E4LMislVE1gP4HQAvWbguRQiqmti9fXhN8Li4WMelpQa+tncnnt6707hip+S0/++Rx40PD+xKPzXorlc88fIZ3wAe1O6B5afUicSpGFVdEpE/BHAEQBHAN1T1TOKRUaSgxdWw3PuxyT2RH+2vLMqu3XVacgpYaigay/nKpB360dvYGNAgzRYB8NjMad/2AWFYfkqdsLJBSVW/C+C7Nq5F8fgtrj4YsPBpOvs78FL7rBJY6VPSyFdMB7BycIXAv3VCK6cABHRYiKQAnnv17cjnebFmnTrBnac589jM6cDvlYectkU775vCzFw18OCNPAZ112WDTyGdBnVX3PJRd10k6s+MyEs0g1rlsbExPXHiRNdft5fZKHWbmasG9vf24xQF6wqyJuUiArB8PR1BRwOurGVI20a0u2+utJ0VW3KKxjuZWT6ZPyJyUlXHop6Xx7WwvmOrh/v0kXOxmnLVl7Utj86gng6nKLjv89f5LnY/dddNvm0o3LNiW5lWyvTKuQCUDaZieoBpe9aoGRgrKHrXxvXr8OTEjRj79DWhLSBaBW0SM/lztn0uAPUXBvYeEPQXtbpQawbz6kKteVyd+73WDSwzc1UUAj7qU7rcFEtQqgVY6eAIxGsRkKS7I1v+DjamYrrI3TS0dfIwxqdmmx+Lw/6i7v/mqeZfbm/IcGdg7sduBvVsNFTxi6k78MZTv4WKxX78Sbo72jwXgPoPA3uXhOU8w3Z51iNKUd5ZqMU6Bo/alUsONg050U8M0BosbbbaTdICmi1/BxtTMV0StWkoqPY8yuZyqaOP1+OfuQbH3nivo9fMmw8vL+GWLZs6+n14g6XtVruddndky9/BxsDeJVE5z0oHTbfc9gHPvfp2rDRMueTgrxnUm+rLiuM/v+j7vbC8OQA8dddKl0ZvrbnbFyZLbPk7uJiK6ZKonKffR2enKHAK/kmaSrnUrHOOm1tfqAWfoDSogn6HDdXAvLn7OMsKqdcwsHdJVM7TL586fc8O7L3lusCOjH51zmRXecgJ/bNLcuA4UVqYiukSk5yn30dnv01HbuBg6Vr6VMP/7JLUmhOlhYG9izrJeYbVuEflfwdda91/p6Lqz5PUmhOlhYG9x3h3l5aHHN9Wr4L4TaUGTdBvpyDmDc2iArTJgeNE3cbA7iOr5kl+h1M7BYFTFNRbug/amIkOsoai7XfqxyRAs6yQehG7O3p4gysQr6NeEuNTs74f60tOAZeXNHLbOq0V1KmyUi7hw0tLvu2JiyJoqCbqsMkgT2kx7e7IGbtHls2TgurYWzswLqvmesZeAFA0mE2b8Avq7iw8aNGzoYo3p+7o6PX8PnHxQGrKAssdPbJsnmR6eHQeg7pb4vnVvTsxfc8OFMX8KG33mUE/UxRp25KfRi8Vlj5Sr+CM3SPNKoewj+kzc9VcBmxTCmDx8hKAK7Nbk0NDBMD9u0bw5MSN2Dp52Pc5frPwNBY92VGRegVn7B5pNU+KOviAszrg4mId+//3KczMVTExWjF6o1MAR89eABCvo2GSBltB2FGRegVn7B5pVTlEfUwP6xPjreAoOUX8xsjVuWziVV/W5nqGaf8cd0YcdxZuu5cKSx+pVzCw+0ijeVJQgGpdYPOzacjB41+83veN5rGZ03jm+FtWx9kLwgK1H3dGnHXpYdavT+RiuWOXfOaR78YuUzQps5yZq2L/N09F9m3vFW7jrLCZeGU1ILonR7klnpuGHHzw0dKae+1WKSpRL+jKYdYicq+InBGRhohEvtgg66T23DhgmReQZEqwMgsPW0x0ioLd24eb6xHAyu+u5BRXPrncu8NqXpwoj5KmYn4K4C4A/9PCWHItKF8ctOGoUi6tqZgJ+ng/feSclZrvblCspCvcmbhXQYDpe3YErkc8eGge5ZKDA3dez2BOFCLRjF1VX1dVlnMYCKq2ue/z17U9DgBD6wsYn5rFlsnDeOjQfGA1TT+V0pVLK8fPBf0uvvrlnZgYrYTe00Ktjv3fPMV+50QhWO7YJUHldU9O3IjfGLm67fn/590PIw+xBlb6hccVY+9PR4acgu8BIR9eXmqWMoaVGkaVB9YbyvJQohCRqRgR+QGAT/h861FV/bbpC4nIPgD7AGBkZMR4gHkSVG0TdCxbGHdW28na97qCYHlZ0Yh+akdq9YZvV8rWUsawyiOTaph++qRC1G2RgV1Vb7XxQqp6EMBBYKUqxsY186Kjpl4CbAnYaRkl7Zx82AHbJgHZDfgPP38q8HfDTT9EwZiKydDMXBXjU7Md/WyvNnh0N+Qk3YU5MVrBV768A07RP2+0uJrWIaJ2ScsdvyQi5wH8JoDDInLEzrDyr7XFQD8plxw8sGukmR/fNOQ0F0WLIs38/+7tw74LpLu3D2N8ahZbJw9jfGo2NDhPjFYwfc8ObPJZR7i4WOeh0UQBuEEpZUGlikG911v1Wu/1B1abbXn59bAXAP/8M9fgF/+v1rz33duH8cLJake97oN+X5VyCccm93R+U0R9hP3Ye0BYf26TXHND1bhfiqmgwydM+AV1wL8PjgL46zfew9N7dzaD9vjUrG99+sPPnwIQ3rOcnROJzDHHnqKwxl8muWZ3hl/0KR30M/6ZayI3oa4zvJZXxWe87hpB0BuPYm3XyqAgvKwamVZh50Qic30Z2N2AYpKnzXJMYbNMv006rVq7Ai4b9IFxCsCxN96DIrxOvdOKGG+HQtM1gtbfQVgQjjqQIq12ykR51HepmF44fsybN/fmjqsLNTx0aB7rCkDdp1j86pLT1gnw6pIDEWBhsd6WizfR+jpppOW9v1u/TyN+ykMOxqdmm/cYdoh0WFqFnROJzPVdYM/yTFLA/43l2eNvte0OVfgHdeDKjNqkPXAv5JD90jAm43KKgg8+WmpuVFqo1eEUBAUB/D6ERKVV0minTJRHfZeKyXoRLWihMI4Fz47MIDNzVRTS3v/folxyjNMdQUG49XzRjevXtbUTrjcU/2SD+esQUXx9F9izXkSz8QZiMla3z3o3yx0P3Hm98XFxQTnvr3x5B96cugPHJvfg/Zr/G9j7tbr1Y+mI6Iq+S8VkffxY0GHXpkzHeuClM109PKPckvc3CbAmOe+wg8GZViFKT98F9qwX0fzeWARm6ZhKjLEuBMx201Byijhw5/Wxfy4qOGf9Jkw0qPousAPZLqL5vbGYzOCjdki2Vtp00oq3U3HebOLK+k2YaFCxpYAFUe0BorbN+23JD7JxfRENRehzS04Rd99cwdGzF/DOQg2FkFOauB2fqH+wpUAXhaVnTGbEpjXhAPDHX7qx+TOtdfRuEPebFfu9cTAlQpRfDOwWJE05mFbaxF3gtDU+IuovDOwWhB02bcIkT9/pAqeLVShEg6Pv6th7TWvPFL/Dpk341YQ7RUG55LDOm4hi44w9IRstDpgqISKbGNgTimpxYJqmYaqEiGxhKiahsBYHNtI0RERx5WbGnnQBs1NhuyvjpGmixp/V/RFR/8nFjD3LmfHEaCWwoZVpJ8qo8XPmT0Rx5GLGnnWP9qD8eFgTrFZR48/6/oiov+Rixp51j/Ygpse5RY2/V++PiHpTLgJ71j3ag4SlaVpFjb9X74+IelOiVIyITAP4IoDLAN4A8O9UdcHGwOLo5fawJmWMUePv5P642Eo0uJLO2F8BcIOq3gTg7wA8knxI8ZnOjHvFzFwV41Oz2Dp5uHlYddj4494fF1uJBpu1tr0i8iUA96jq/VHPzVvb3jiCOi3afCMKaiPMNr1E/c20ba/NHPvvA/iexev1LO+MO85MOKzCxRYuthINtsgcu4j8AMAnfL71qKp+e/U5jwJYAvBsyHX2AdgHACMjIx0Nthd4Z9xumgMwa6XbjaBrWmZJRPkUOWNX1VtV9Qaff9yg/nsAvgDgfg3J66jqQVUdU9Wx4eFhe3fQZUln3N2ocDEtsySifEqUihGR2wH8EYA7VXXRzpDSlSSNAiSfcXcj6PbbYjIR2ZV05+mfALgKwCsiAgDHVfUPEo8qJUnTKECyNIdbglirL6O4eg5pWodJs1sk0eBKNGNX1X+mqtep6s7Vf3o2qAN2Fi47nXG3liACwLJq8+cYgInIplzsPDVlY+Gy0zRHN6phiIiAnDQBM2WrWqSTNAdLEImoWwZqxp5ltQj7vRBRtwxUYM+yWoQliETULQOVigGyqxbhgdVE1C0DF9izxBJEIuqGgUrFEBENAgZ2IqKcYWAnIsoZBnYiopxhYCciyhkGdiKinGFgJyLKmb6qY3fb3nKDDxFRsL4J7DZ6qRMRDYK+ScWw7S0RkZm+Cexse0tEZKZvAjvb3hIRmembwM62t0REZvpm8ZRtb4mIzPRNYAfY9paIyETfpGKIiMgMAzsRUc4kCuwi8t9E5CciMi8i3xeRzbYGRkREnUk6Y59W1ZtUdSeA7wD4LxbGRERECSQK7Kr6jy1fbgSgyYZDRERJJa6KEZE/BvBvAbwPYHfiERERUSKiGj7JFpEfAPiEz7ceVdVvtzzvEQAbVPXxgOvsA7DOthWUAAADNElEQVRv9cttALJq8nItgH/I6LW7ZRDuERiM+xyEewQG4z5t3OOnVXU46kmRgd2UiHwawGFVvcHKBVMiIidUdSzrcaRpEO4RGIz7HIR7BAbjPrt5j0mrYj7b8uWdAM4mGw4RESWVNMc+JSLbADQA/BLAHyQfEhERJZEosKvq3bYG0kUHsx5AFwzCPQKDcZ+DcI/AYNxn1+7RWo6diIh6A1sKEBHlzMAFdhGZFpGzq60QviUi5azHlAYRuVdEzohIQ0RyVW0gIreLyDkR+ZmITGY9njSIyDdE5F0R+WnWY0mLiFwnIkdF5PXV/1f/Y9ZjSoOIbBCR10Tk1Op9PpH2aw5cYAfwCoAbVPUmAH8H4JGMx5OWnwK4C8APsx6ITSJSBPCnAP4NgM8BuE9EPpftqFLxZwBuz3oQKVsC8LCq/jqAXQD+Q07/LC8B2KOqOwDsBHC7iOxK8wUHLrCr6vdVdWn1y+MAPpXleNKiqq+rah5P+r4FwM9U9eeqehnAXwD47YzHZJ2q/hDAe1mPI02q+veq+uPV//4VgNcB5O7ABV3xweqXzuo/qS5uDlxg9/h9AN/LehAUSwXA2y1fn0cOg8GgEZEtAEYBvJrtSNIhIkURmQfwLoBXVDXV++yrE5RMmbRBEJFHsfJR8Nlujs0m03YPOSM+j7G0q4+JyMcAvADgQU9jwdxQ1WUAO1fX9L4lIjeoamrrJ7kM7Kp6a9j3ReT3AHwBwL/UPq73jLrPnDoP4LqWrz8F4J2MxkIJiYiDlaD+rKq+mPV40qaqCyLyl1hZP0ktsA9cKkZEbgfwRwDuVNXFrMdDsf0IwGdFZKuIrAfwOwBeynhM1AEREQBfB/C6qn416/GkRUSG3eo7ESkBuBUpt18ZuMAO4E8AfBzAK6snP/2PrAeUBhH5koicB/CbAA6LyJGsx2TD6sL3HwI4gpXFtudV9Uy2o7JPRJ4D8DcAtonIeRH591mPKQXjAH4XwJ7Vv4vzIvJbWQ8qBZ8EcFREfoKVickrqvqdNF+QO0+JiHJmEGfsRES5xsBORJQzDOxERDnDwE5ElDMM7EREOcPATkSUMwzsREQ5w8BORJQz/x8XT1ImQx2oMAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.lines as mlines\n",
    "import matplotlib.transforms as mtransforms\n",
    "\n",
    "random.seed(100)\n",
    "\n",
    "std1 = 1     # The desired standard deviation of our first random variable\n",
    "std2 = 0.333 # The desired standard deviation of our second random variable\n",
    "\n",
    "x = np.random.normal(0, std1, 1000) # Get 1000 samples from x ~ N(0, std1)\n",
    "y = np.random.normal(0, std2, 1000)  # Get 1000 samples from y ~ N(0, std2)\n",
    "#y = y + np.random.normal(0,1,1000)*noiseLevel * np.sin(0.78)\n",
    "\n",
    "# PCA works better if the data is centered\n",
    "x = x - np.mean(x) # Center x \n",
    "y = y - np.mean(y) # Center y\n",
    "\n",
    "#Define a pair of dependent variables with a desired amount of covariance\n",
    "n = 1 # Magnitude of covariance. \n",
    "angle = np.arctan(1 / n) # Convert the covariance to and angle\n",
    "print('angle: ',  angle * 180 / math.pi)\n",
    "\n",
    "# Create a rotation matrix using the given angle\n",
    "rotationMatrix = np.array([[np.cos(angle), np.sin(angle)],\n",
    "                 [-np.sin(angle), np.cos(angle)]])\n",
    "\n",
    "\n",
    "print('rotationMatrix')\n",
    "print(rotationMatrix)\n",
    "\n",
    "xy = np.concatenate(([x] , [y]), axis=0).T # Create a matrix with columns x and y\n",
    "\n",
    "# Transform the data using the rotation matrix. It correlates the two variables\n",
    "data = np.dot(xy, rotationMatrix) # Return a nD array\n",
    "\n",
    "# Print the rotated data\n",
    "plt.scatter(data[:,0], data[:,1])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us print the original and the resulting transformed system using the result of the PCA in the same plot alongside with the 2 Principal Component vectors in red and blue"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Eigenvectors or principal component: First row must be in the direction of [1, n]\n",
      "[[ 0.70145269  0.71271602]\n",
      " [-0.71271602  0.70145269]]\n",
      "\n",
      "Eigenvalues or explained variance\n",
      "[0.93956841 0.10075101]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XuclHXZ+PHPNbOzyywoC0IZC4iZUakEiT72QFZqnsUVDfPYScnH8pCGQpqiWSKUlZ08Vo+HTFRE1AyP5Q+eNEEQReUxNYPFHkVcDHbYw8z1++OeWWZn7nuO9xz3er9e+9Kdw31/l5297u99fa/v9yuqijHGmPoRqHQDjDHG+MsCuzHG1BkL7MYYU2cssBtjTJ2xwG6MMXXGArsxxtQZC+zGGFNnLLAbY0ydscBujDF1pqESJx0xYoSOGzeuEqc2xpiatXLlyk2qOjLb6yoS2MeNG8eKFSsqcWpjjKlZIvJmLq+zVIwxxtQZC+zGGFNnLLAbY0ydscBujDF1xgK7McbUGQvsxhhTZypS7miMqX+LV7WzYOk6NnZEGNUSZtZh42mb1FrpZg0IFtiNMb5bvKqdOYteINITBaC9I8KcRS8AWHAvA0vFGGN8t2Dpur6gnhDpibJg6boKtWhgscBujPHdxo5IXo8bf1lgN8b4blRLOK/Hjb8ssBtjfDfrsPGEQ8F+j4VDQWYdNr5CLRpYbPDUGOO7xACpVcVUhgV2Y0xJtE1qtUBeIZaKMcaYOmOB3Rhj6owFdmOMqTMW2I0xps5YYDfGmDpTdGAXkUEi8jcReV5E1orIFX40zBhjTGH8KHfsAg5S1a0iEgKWicjDqvq0D8c2xhiTp6IDu6oqsDX+bSj+pcUe1xhjTGF8ybGLSFBEVgNvA4+q6jN+HNcYY0z+fAnsqhpV1YnAaGB/Edk79TUiMlNEVojIinfeeceP0xpjjHHha1WMqnYAfwYOd3nuRlWdrKqTR44c6edpjTHGJPGjKmakiLTE/z8MHAK8UuxxjTHGFMaPqpgPAf8tIkGcC8VCVX3Qh+MaY4wpgB9VMWuAST60xRhjjA9s5qkxxtQZC+zGGFNnLLAbY0ydscBujDF1xgK7McbUGQvsxhhTZyywG2NMnbHAbowxdcYCuzHG1BkL7MYYU2cssBtjTJ2xwG6MMXXGArsxxtQZC+zGGFNn/FiP3RhjSmbxqnYWLF3Hxo4Io1rCzDpsPG2TWivdrKpmgd0YU7UWr2pnzqIXiPREAWjviDBn0QsAFtwzsFSMMaZqLVi6ri+oJ0R6oixYuq5CLaoN1mM3pgYNlPTExo5IXo8bh/XYjakxifREe0cEZUd6YvGq9ko3zXejWsJ5PW4cFtiNqTEDKT0x67DxhEPBfo+FQ0FmHTa+Qi2qDZaKMabGDKT0RCK9NBDSTn6ywG5MjRnVEqbdJYiXJD3R2QnNzf4fNw9tk1otkOfJUjHG1JiypCeiUbjwQvjMZ5zgbmpK0T12ERkD3ArsCsSAG1X1Z8Ue15h6VWxFS8nTE1u2wJe+BH/6E5x7LjQ2+nNcUzZ+pGJ6gQtV9TkR2QlYKSKPqupLPhzbmKqXKVCnPvf5j43k3pXtRU+4KVl64rXX4Jhj4NVX4YYbYOZM/89hSq7owK6qbwFvxf//3yLyMtAKWGA3dS/TzEgg7bk7nv4nmnKMREVLxfPIf/4zHH+88/+PPgqf+1wlW2OK4OvgqYiMAyYBz/h5XGOqVbbSw9TnUoN6QsUrWm68Eb75TdhzT3jgAdhjj8q2xxTFt8AuIkOAe4HzVfV9l+dnAjMBxo4d69dpjakov0oPSznhJmNOv7cXLrgAfv5zOOIIuPNOGDq0ZG0x5eFLVYyIhHCC+h2qusjtNap6o6pOVtXJI0eO9OO0xlRcppmRXs9JyvelnHCTcZbqe+/BkUc6Qf3CC52eugX1ulB0YBcRAW4BXlbVa4tvkjG1I1PpodtzgpOOCSRF96aG0lUde6WK/nD7Y3DAAU5e/ZZb4Ec/gmDQ/SCm5viRipkCnAa8ICKr4499V1X/6MOxjalquZQeLli6jvaOSF9QB4glJds7Ij0lW4rWLSU05R+r+dXiq2GnMDz+uFOrbuqKqHoN55TO5MmTdcWKFWU/rzGVMmXeE66zRZO1toRZPvugkp731OceYu5jN/DmB3Zjj78+Drvv7uv5TGmJyEpVnZztdTbz1JgyyGUwNd8B18Wr2pky7wl2n/0QU+Y94bq6YyId1BDt5fuP/IqrHv01T31kP16+52EL6nXM1ooxpgy81ndJfU02iQqX1NSO10SntkmthLa8xwfPOJ3Jr63its9+iZ1+dA1tk9Mr03KZETtQ1oGvdZaKMaYMUicypUoE6dYMwTLbMcAlnfPKK85M0n/+E266CU4/Pef2hQLCkEENdHT2uM6aBWeg+Orp+1hwL5NcUzEW2I0pk+TebktzCFVn4DS55w3ewTKXPD04wb29I8Ln3niO6+6/ht5QiBd//jsO/Mqxnu/J5dip7Uw+n99jA8ZdroHdUjHGlInb+i5uAdVriYFccvACtL/XyVdXLuHSJ25h3cjdOHP699j8WhNXr2r37FnncuyqnTVr0tjgqTEVlM/M1Ww5eAEaoj1c/aefc/njN/Honv/BCafMp33oB7LusFTMzFfbpq76WGA3poLy2dPTbcJTQks4REvnFm6763uctOYRfv7pE/mvtjl0Nu44TqaedaZjZxIKim1TV4UssBtTQZ//2Miclxhom9TK8fu2IqlvAMa89Qb333Yhkzau49xjvsOPDzwNlf5/3pl61m2TWrl6+j60toQRnAtFKOhyohSDGxts4LQKWY7dmApZvKqde1e298tdC3D8vum5+MWr2pm7ZC0dkZ6043z+tWe5bsl8IqFBnHjyPFaPSr8o5LIeTeoYwOJV7VzxwFre60w/Z8IWl/aYyrPAbkyFuK3josCTr7zT7zHPMkdVznj2Pr775G9Z+8EPc+b07/GvnUe4nqvQksTtPbGMz1t+vTpZYDemALlM1Ll08Qvc+cx6okklxcl16rkOnLpdABp7e7jqkV8y44XHeGj8FL5z5LeJNA5yPV5rS7igoO523mSlXJXSFMcCuzE5yjTrc9Y9zzNn0Roi8R5uY1DojqYXCCbPEB0aDrmmVhSnDNLrArDLtg5+vfiH7L/hJX465SR+NuWktHx6skKDb6bB1kwTqUzlWWA3JgOvYJ4asnuiSk9SIHcL6gmJ0kO3QdCE5AtA8nIE49/5B7fccyUjOjv45rSLeejjmVdmHNYcKjj4ei2DYBOSqp9VxRjjIXmTCvCeoFOI9o5IxkFJ2HEBSJQiHvLqM9x7+ywaYr188eRrsgb1cCjI5cfsVXAbM601b6qbBXZjPGTLMRcrezGhkw5pmziKn765lBsXXcXfdxnNtNN/wgsf2rPf60IB4dQDxvaVK7a2hItewyW1BNKPY5rysFSMMR5KPVVe8V5/JWHc4CB8+cscdtttLPn4gcw64jy6Qk39XlPKfLfbMgim+llgNyZFIq9ejuXxFAiK9KucSRjTtYV7//hjeOE5fvyZU/n5p0/ELTHvV77bluStH5aKMYVbsxB+sjfMbXH+u2ZhpVtUtNS8eqpc0ieJ14UCub06Oagn3vG5znaW/mEWw197Be65h0VHftU1qEu8zcXKuOm1qTkW2E1h1iyEB86FLesBdf77wLk1H9wz5dVbW8L85MSJOR1HgZ5Y/n1+BY7/xzP8+vrz6NjWzVe+fi2LP3yA54ClAnOXrM26k1I2XpteZ1o4zFQvS8WYwjx+JfSk9Gp7Is7jE2ZUpk0+8MqrC06VSEkDnSrf+utdfOf/3c5zo8bzjeMu5Z3mYSy/+3kaG7z7YB2Rnr56+PaOCOfftZq5S9Yyd9peOadS8lll0lQ/C+ymMFs25Pd4FfDKISc/HvDIdyNw/l2rS9a2pp4u5j98Hce+/BcW7fV55hx+Dl0NjYDT8+/pzq86pyPS47pVnhevmnVbMqA2WSrGFGbo6PwerzCvHPKli1/o97hrUAdKudHYB/79LnfdOZtjXn6Kaz77ZS446oK+oF6MfFIpVrNeXyywm8IcfBmEUnpzobDzeBXyyiHf+cz6jLXqwUzTQ32w97/+zv23XsCem9Zz1vRL+PUBX3QdJC1UrqkUq1mvL76kYkTkN8DRwNuqurcfxzRVLpFHf/xKJ/0ydLQT1MucX8+1RM8rwHn10HN9vhhHvrKMHz/0E95t3pnjT13AKx/Y3fdz5JNKsZr1+uFXjv13wC+AW306nqkFE2ZUdKA0dTnb5PVV+gLUmoXw+JW8NmgDG2O7ML93BktiU/uO4VVDXkqiMc5bfifnL7+TZ1s/wVnHfZd3B7e4vrY5FHBy7BnWnvFiqZSBy5dUjKo+BWz241jG5CpriV5SSWYAZXRgE/NCNzMtsAxwKl2iqjnXpvthUM92fn7/fM5ffid3730Ip3zpB55BHaCzJ8b+44bRmucgpqVSBjbLsZvSKMPkpeT0yrTAMpY1nsvrTSdzV+eZfT311JLMZunmooaFGVdqLJVd39/E3XdczJHrlnPV57/GrCPPo7shlPV9y1/b7LqFnptwKMipB4wF4Nt3rS64rt3UtrKVO4rITGAmwNixY8t1WlMJiZ5yIqgmJi+Br6mbRInetMAy5oVuplm6ARgtm+CBc9GeiGswHBV4N+9gnm1Nl2w+uXEdN973A5q7I3z9hMt4co/98nr/nc+s9yxJTGhtCfP5j43k3pXtmdNTpu6Vrceuqjeq6mRVnTxy5MhyndZUQqbJSz5KlOhd1LCwL6gnny+q7h9vUWVZ47l9KZlsig3q0176C3fdOYeuYIjpp/4o76AOTsooU748MYHKrconY9ljHS4LYSwVY0qhTJOXEiV6owLvuj4fJEanpteDi5CWb097r0hf2V+hQV00xoVP3cZ1Dyzg+Q99lGNPv5ZXR+5W0LGCIrRNaqUl7J66GRoOMWfRC54Dwa5VQXW6LITxKbCLyJ3AX4HxIrJBRL7ux3FNjcpn8pJbjzH+mM5t4V9zP8J5353jmStum9RKwON87TqC2T1nsCE2wnWCUSLfnpyfT/Tko6q8Me+ogqtKmrsj/Hrx1Zzz17u4c8KhnHri93mveWhBxwI46T/GADB32l6uE4lEyFiP71r2WKY7K1N+vuTYVfUkP45j8pQYIKxgHbmrgy/rn2MH98lLbrn4xWc7XepoNwLsyjtcHbqZ2e/DnEVOuiUtV3zwZfTefw4N0e19D3VqY19p45LuqbzedLJrvr1VNqXl5+eFbkZ6YPGqiVzxwNq090wLLOOihoWMkk1s1BFpJZSj3n+bm+/9PuPfeZMrDj6T3+47rehJR5N3G97vZ0+t3f92huUOPMsea3BZCJMb0TLX8AJMnjxZV6xYUfbz1pXUoAhO8DzmuuoI7msWwsMXQyReBRsezrMfn835L+3ZF5AelbNpjryV0+F6NUCAGO/LTk46IvJev4vZ3Ksu54zu2xkl77JR0+vVlzWey+jAJtfjNkgs7fENsRFM7b4u7fHUgVpwLiKze85gSWwqn2p/mRvu+wFNPd2cc+zF/OXD++b082XTEg6x+vJDPZ+fMu8J14HVoAg/nvFJ94HTn+wdT8OkkCBorLo6CwYAEVmpqpOzvc5y7LWqFm6je5PaF9nM3isvZd/3H+1bq2VQZ25BHaBBYgQEWvh3/GIRzwkvmglzh3JG9+3M753Bh7vuYGr3df2COsD83hlp+fZObSRAelAHGCXueXu3gdpESue4F5/gzjvnsLUxzHGn/di3oA70rd7oxW2tF4CdBmW4KXdbFgJAo1jOvbZZYK9Vft5GZ6qMKLRqwuXCE44HwISNOiL/tqZx7jizDYYuiU3ty7fHVNisQ4hoo2dt+EbdxfXxUZLe6yemtD7ezk8eupYVoz9B22nX8tqIMYX8MAVLDCQPa+4/uJpY5dG1ln3CDOcOb+gYQJyeeqpq6yyYnNiyvX4qZ8576Gj32+h8V1fMVHMOhdejJ11gtqMsJ8rBNNAqm1jWeC6jZBMdDKFLgzSJPxtGN0s3Pwv9imu5niAxojjpm406gsdjEzk4sJpRson3dAg7yXYaA72ux+nWBub3uv98G3WEUyef0KWwKIL8by+3TzyCuYd8g96g/39WqQHbTdukVhYsXcd7nf1794lyR9d0TPKyEHM9ZsBuWR9P21TZWI7xZD12v5S7dMyv1RW9UjoPXwz3nVV4uifpAjOfbg6RTn5IF7H41P6AwHDZiiC8GxviuSxuvkNAIk7aJvHfQLy08fTgY33n3SWwlUZxD+oAAbwvNP1SOh0x+M029NVe7j70C1x62DfTgrpbxU2+QkHh8mP2yum1RW2Y4dkpECuJrDEW2P1S7px36m300DGFDZx6pW4im+O51jzekyzpwjOLRk7SBi6RLk6XCJGkyvBEgD2v5+y0HHiEJm6NHuJai56vfIpSGkS5NnS9axBOpHTe/scQ9KZtxN6HX514IrMmnZf22sRAa+KCki1dlNAcCjCsOYQAXxnyN57f+QLa7t8rp1SY12qOOa3y6Jpzd5meZemZqmdVMX6Z24L7/ESBuR3lbk3uvCojMpEgHHe990WkLyW1nl4CBDRGu+7CrbKRSwNd7K8BFtPMh+L9CgXO7z6bYc2NXBS6i+bIW8QkALEYihBI+Xct8RLpfRLVLkC/8sa/PPcxvvjwI6xv+SC/P+EovvrBx1xLHzNV4iRSRMmv7yujDLzr1ObveSg8//uUDkM80A4d45oSSV3xEpxyx6wLgiX9zpyqmKhzDs/PRpV/rquJjynaXKtiLLD7xStADh0D336x/O3JlVvZZC68SitdjpcIkBc1LOTZwFucSoThCPfTzKeID9gl/p3WLEyrSU+mWr7ADrBZhzCIbqcSJqbwaBc83c0r48bx27Y2Lt/5ds/Sx9ebTiaQpa3JF4/UMsrsixm4B/nEGvXtHZG+ZYlbM6xVn7F0NhHsU1X757pa+FyWbOWO5VZjOwr1cUvphIdnf1/q7fiDF8AVw2HRmZ4rKs7vnUGbhljOYASYyjbuJT7QF0/vdD58mWdQByeol7MvMoytTrDtUvhDBJ7uhv0bGXKKcM5OSzxLHyG3qp/E613Xu8m6mEH8+ZS8d9uk1r7yx8QSA4nFwFyrY7zSiIvOhO5tEExJhdXC57paVKgs2QK7X/zKeVfChBlO72tuh/PfI65xr29Olci1P3gBrLjFOyePUxe+JDaV9xjCRII8y2AmEuQEiXAlXWxv2JmNc/cgnEdtuxtVp2Ot6uMF4L0Y3LINXuuFowbBEYNoDb7rXvqI87Ne0fAbPiTv5tSGUeJ9rJylBIusa9UnyzRmEtns/EOGh1Pw53ogLzRWodm9Vu7opwrvKNRPoXk9j3XM3anzh/p+9vW+35YRCHBd6Awu1ev5YHQ7T9DMN3Q7l0sXL/b+i98xCCkyzyLx7MSt0UM4LfhYUZto9GoA+Uc3wbsjTnA7tRl23/En08kghpB+d9FJE6cHH+uXMlJ1+tduqZkOBtOpg/qXUe74ich5bckt653f34QZ+VXHeJXOJsR6oHEwXPxGbu1IVqYlnKuWX2XJebIeez3KpfTSa/GtvvflaMt6Z/p5JqEwu07/IW/MO4q5l17Bqk9eyb8YSaMGuCo2mitjg7mHHj7LNto9ZoImy5aOEYHTgo9lzW9nE3xuO8HbO2GwwJlD+gV1EWhmO93av2/UqY2E2Z42DiCC50WmkR6eiE0k7RWhMEz+WvwuMEeLz4Y1C/OrjvGagZqs0B5mLcyQLqUKpWitx15v1ix06s9T0yKJP6YJM5zXLD7b6YnBjsW3mnbKfxA1B1saR3HkH0ew8fcP0dIcYuv2MfTEftb3/BtNJ/NJwpxChP3Yxv00sx8usyCTZOvYFxXUYwpLu5C/dcNHGuD4MAxKP2BAoCsWJECsb0LU3dEDOT34WF6nG0wXX2r4M+k98wCMPcD5WjTT5Xm3tvfAopksAzY27cI1PTuqbjwXA+u3MbnHRb3QHuZAX2isQpu+W2CvJ4ket2f9+Xrv6p1Yz44Fu3y289bXmBn9JZfzNT6z/cm0lREBphHifwgwjU4OZBu/I8yJZJ9t6bvtCvd0wmtROKARvtCU8SoxRLr6LjINxDgp+Oe8TykCjbhMmOrZ5gxg5s3Zx7VVNvGjxhuQblix8xe8q2LSW0S/i0gxPcwKpSKqSgVStFbuWE8KqUkvE1WIIgTRfr3tTm0kqsJOgS4A3iHGdCIskyjf00bm0kSgXNtNvxuFOyPOYOlRg+BTxU+MShaJBRkk0bKWawIgATjuhszzDtJKXjPXy+es2lchrTG5ljtaj72eVPHtrQg0uKQSmqWbKNJXnz6SAI/RzH/pdr4v3bykMf6bcLxAsoRe70Xv7kQCAqc3w27+/2k0SYzNOoRdZKvvx85IY5kHLF0Hy9WfWvUKpSIGOgvsmVTrRhZeslU3VKmg9A/4TQi3MIi9NcAsung9nncfU6qx/me74eHtyIgAnNQMw0pznsQ68akTkbo0SCMl7smnjrEkf669PjN+dRSqqVpsgLCqGC/lXNTLrzrfXKobSsjPrJ4gXEATDxDm78TYj20845aHLkZU4aEI/HE77NkAXx9csqDepUHm987gseCB3B09kGhSrX0PITqlqSTn7WfLBvfPtdfd0EDKg9cZy7F7KdcSAbnkNyG3O4c1C+GB851BtzIr5VT/l4hyDJ20o9xCmFP8GFSNKNzdCW9E4T8b4eDMg6TFiCncFj2EowNPMzywFVz+rRJ/hSVNOCXWgHF/krQBU8uDVx3LsRerXGVaXvlNiJchngUE+pcmJq+Xngj44WGwfUvaH25qwI3Fv5fwcOj6947jFqmUaYRPEOQZBnMCEU6VCGs1ylXFDKpuig+SbonBsYNgor+DpKkE+k9Ycml2WcZTM8wM7utIZOo81FpqcgCzwO6lXGVa2S4UsSikrg+eWC+9N7LjouBRqpg8mWezDuGK3tNZufMXWH7xQen7klaxEQR4hGa+pdu5Oj6oejthhuQbEl/rdXrqDfFB0rGl/xMoexVMIbLdibrNIF000ynHLLZyxvjOcuxeyjVjrNALRWRzzpOJRJyvcHzArm9a+YQZzlTxGtGIcAOD+Jk28QC9TGEbb+YwUxVwrm7PdMEdndASgDMGlyWo14RcPtfZ7ixt842qYoHdS7kW9drTe+d5vyVWEvzykL/tGKytsSoaQTiXJh6mmTfjg6rLsw2qRhUe3A5/6oLxDfC1wU5wHyhCmS7eAp88Oa+tDl1lWiZgIC8CViHWZcmkHGVarz6S/3tCYWgIF5RCaZVNXKrXwxbvpXFrwaE08DSDOYYIB9HJjQziy7jkyjtjcHcE/hGFqY1wUFNV5UZUcfaRLuVJYj0QCHmMp+iOz2ByDj08zHkssjm+yXUORRZuwX+gLwJWIb50W0TkcBFZJyJ/F5HZfhxzwMh3MFaCzp3DEdcUdj4h43rnteRj8UHVqQT5imznIrYTTQ5A70Th5m2wPgrHheHgQVUT1FXh3dgQZum3Sn+yaLezDpAXtzLIyOYdHYeMg65J3NKKA30RsAopOrCLSBD4JXAE8AngJBH5RLHHrTtet6P55NhD4R1b0k2YkduGGCmqI6z5ZzjCn2jmbA2xQLppI8L7KLza4wT1buArzTChAuvOZLBVmwgGhAWBX5TndxJ5z3uVyKGj81iqOSG11eKeVhzoi4BViB899v2Bv6vq66raDfwBONaH49aPTJOdsk0qkiCeOf4jrknf3WYACiH8kjC/1EE8TC//GdnK6w9HYHgAzhwMo6sv47hToIsW/l2+C22iPDH1sxYIObsk5TXWIs5ywv1ar7DiN86mK6nn9WqPKRk/AnsrkPyp2BB/rB8RmSkiK0RkxTvvvOPDaWtIptvRxCCtW+870UNP7GzklpOswASzanU2jSyNDGKjKNcfKvDVwTB0AA2SehIncD9+pTNQmigICA93UlP5jtVIwNkxKy3vHg/uyYOjtbplZI3z41Pv1ulIizaqeqOqTlbVySNHjvThtCVQqtH7bLejE2Y4u9NMvym/KpzHr/RtglG9ODjcyHPbm/nB+MHQWG+Jp3wkz4ZKKkt8/vdOUJ3b4ZS6RlP3Wc1BtolOyfnzWt4ysob5cY+6AUhO3o0GNvpw3PIq5eh9rpOdkqtwEhUKi2buuI3+59Ow8nfOH1bG6eED27iW6ku9lJ+6f0YSk9ugdKWuqR0ZWwSs7PzosT8L7Ckiu4tII/AlYIkPxy2vUo7eu+bRPQabwD0nf99Z/TeMtqBusvH6jEQ2F7iBR44sf15xRQd2Ve0FvgUsBV4GFqrq2mKPW3alHL2fMMPJbaYONj3/e/d0j9tFxgK5yZdk3l7QVSgMk79eUMVV3/stf15xvowsqeofVfWjqrqHqv7Aj2OWXalH7199hLShB687AisFM37Y9yv5v+eY6+Doa50xn7lbnK9Mm2kHQvGLQA75c5uBWja1VzJQqg9HPqP3hbQhnzsCu5U1fjj62vx63kPHuAdlr5Lc8HBo+1X8IpChcgvKu7+BqbHAXsoPR66j94W2IZ87ggpvmGHqyF7HkfO0tO5t7p9jt7+N6Tc5AT3XQVGbgVpWtbXRRrk2vyhFG/Ld1HfNQmfA1HLrphASdOZApG3ikkWpNtiY24LnejPTb7KqmRzlutFGbfXYq2F6cqFtSO31hIc7C3ktmumezpkww/nDdEsPTb/J+aq7BQJyV2h/ZMDM5xo31ekY5LVMAKXrRWdKL1pKxne1FdirYXpyMW2YMMPp1U+/0dkkI7KZjOmcTOmhCTNcpnUPHIWu5VUla4CVjgRh98/Chr8VfrdXivr2TOlFS8n4rrYCezVMT861DZkGWPPJNyYuBm6DU0df61wkMlUtmJqX212GOBUsl2+Gza/n31Pvd6gCyiSzSXRSvGxZb9UyPqqtwF4N05NzaUO2AVbPdE4BH+6+uwB/UzOqzldM/U1fqEIkFqy7lEgpf55tNBHRLIu9Jd8xZkwL5vAZKdW4zoQZWTohVi3jl9oaPK0V2QZYvZ5PVsgg1tyh+bXTgyrcGj2Ey3u/BsDKxpnsEtia83uX27grAAATHElEQVTBO+WxXRuIEaBZClijpIopECVEA/6v3RNDWPmpa9hv1RyPoCvOnVvis+L1+ZKgU9v+/O8z9+j9LEZI3QB7+Ifhjb9kf185CyJqSH0OnlYjt5RLtgHWXMoZC8k75pCSUXXWAu/WhrTHVaFXA9waPYSVsY+yrPFcXm86GRHo0uy3553ayHsMyZjHbqK37oI6OFv2NYQzbGZRhMDQ0ew37Rvug+mJJXSTOwBe6cLjrnfSd313nPH3p77Or9Sm253rG0/l9t5KT9Kr8clUFtiL4ZVySWwrlipxu5yazvGS74fba73tpJmBV4TOZ++u3/KdnplsiI0gpsKG2Ahuix5Cu44gQIyjA09zbeh6Rgc2ERAYLlsJoryrQzxTDqowu+cMWsitZ19/tKCtCvsJhNLX108OtK715Dc6wTpZv9fh9NQTHYUHL0jqQY9xLgqFpDZzCXyZNsDOppKT9OpgMpWlYorhdcsbHu5UveRas+5nfX7qre/Bl/U75+JV7cxZ9AKRnh239Cc0/g8/DN5Eo3ZlPPRmHUKnDmJ0YFPacxtiI5jafR3LGs91fR6cHv12GhkupQn+2dJAFRMe7uxg5BnUZMfvCjL+/vLiNnciVSEpv1znZGSqXc+kVLX0uaqG+TIeLBVTDl496sh7+Q3y+lntk6mKBmib1MrV0/ehtSWMAK0tYa4cfG/WoA4wjK3M751BZ8pAXqc2Mr/XOc/83hlpaR5wBmHvjh7I3J7TiZWoL1HxgB4ent7jDjY6O115lsmO6f+7yvL7y0su290VkvLLtarLs9ed8ovKZ72ZcqiG+TJFssBejEw17fn8gZa52qdtUivLZx/EG/OOYvnsg2iO/Cvn917UsJBBdNOrAVSdnvrsnjNYEpsKwJLYVP6tg9LeFxA4OLC673Vu/Lh5rFhwD4Wd6fupP0Tie6+lm7esL10ON9dAlG/AyjXweXVYUtM/ua43Uy7VMF+mSLYjQTEOvsz9lrTQnnalPtBeG4GkUOhLswSI9fXUU4P1MI9Uyyhx3rtRRzBa0tM1bkFZtQp64nFKvD04lSoRGmmmm+3Nu9J8xJXuO1rFepzHE7fwj18Z/7dO2dnIr01dkuX4e807YOWzcQz4l1oqFz//rivEeuzF8LunncdI/OJV7UyZ9wS7z36IKfOeYPGq9sLOCTy7xzlZ66RVnV53smbp5qKG9Daqx4Bw4nG3dI7noCykvdYPhdwdCBBrGMQVofP5SNcdHBb+A0va1tJ88SvO7zyXLRC//WL885LjEs6FWrPQWdQrm0ICVj6pQz9TS+VSDfNlimQ99mL51dPOY2u+1AHQ9o4Icxa9ADhplnyd/9Ke7NtzBhc1LKRVNuXVQx4l76Y9FvAYMEs8viQ2FXqctM4oeZeNugthtrOLS09/o47g8dhETgs+lnZhKcY2mhisXXnfDTREtzOXnzL3g2N4do9zOH8pfPuu1YxqCfNoeFeaI2+lvym1J1vqHK7XoGl4uJMuevWR4nrQtdoTz0eNb+dngb1aZBqQSvmALVi6rl9VC0CkJ8qCpesKCuwbOyK0M5Ul3VOdunWX13gFwI26S97nAye4L+nekcKZFljGvNDN/WrcE6meixoW+hrUuzRII9HiUjxb1rP3ykvZt+cM2plKe0eEyxqPZ17oZhqi23e8zq0nm2sqo1Beg6aNg9NLIwtV44Gv3lkqplrk0Yvb2OFe6eD1eDajWnbcVm/UETm/T5W+aphpgWV9E5piHqmYzTrE81hLYlOZ3XNGv9r6xKDsKJd8fOL8qRU2qhBVj1RQfLB3G2EapTeHnzCzcEoq6p7u/+QqOSv7LXyp1zyqg6oOUxzrsVeLPHpxo1rCtLsE8eQAncniVe0sWLqOjR0RRrWEGbfLjuPN752R1nOOueTXwQnUS2JTXXrbmjbo2aVBrug9PWO7UnvxCV6DrRsZwTU9M/qldBIXmgWhG2iS/nc1PTQwv3cGPw39yvX8qvFxgIAQ0FjGtiakpqL+e+v+zL30isxvKnUqo9R3BKbqWWCvFnmMxM86bHzaJKNwKMisw8ZnPY1bfj65p++W/348NpEvBp9KS5MkAvVFDQvTlgkQcZYnCKB9ATdTqWMmbhebTm3kmp4ZnheDudxKU8os2Ebp5aKGhZ4Xiv+TkTx97F9oCy7PeYOK1FRUrhdX31MZyRPTwsOc2vDkCp0aq+owxbHAXi3y6MUl8ujJve5Zh43PKb8+d8natPx86lCnW7BcGftoWs84Eai9UiUBlA933ZG1Tdm4XWyyXSi8ljYYJe9ykX6TecH0XPiux/yQtgmtQPLvwrtcMJI0MQtyv7j6LnWwNLLZmRiVmPFaj4ObJiNbUmAAWbyqnfPvWp3z6wPipCdGtYR5+/0IPR7ZCa9lBBLLDFSCV5v+RVKvPJdUyIMXwIrfkHb5Cw/n2Y/P5vyX9sz74uq7Kp4Cb/yV65IC1mMfQBYsXZfzawW4dsbEvkDltsZMgleqJLk3WypJ03zS2nRN6GbCySmi1F55th7smoXOErf9zhBfTfHoa9kPWD6t2J/ABzZYalIUVRUjIl8UkbUiEhORrFcRU1n5VM0o/Wvi2ya1cvy+ra71LpkqWkotOeSGgkJLOIQAK3f+Ai/ue1Vxk0y8Vid89ZHiG+6nOpgCb/xVbI/9RWA6cIMPbTEl5lVN46YlHGLKvCf6pRmefOUdz7X6vAYxy2VaYBkXBRcyKraJLTt9kGFHXgUTvgF8o/CD1kpPuA6mwBt/FdVjV9WXVTX3+3tTUbMOG0841H/DjFBAXEsZOyI9tHdEUHbMbM31ouC3cCiQ1u5kiXLLxPrxw3r+j+77vlX8wlq10hOugynwxl82QWkAcVuyd//dh+W0jG6kJ+rr7M989MaU4/dt7dfulnCo73m3cstG7Sp+7ZVq2Dw9V7W4JospmaypGBF5DNjV5alLVPX+XE8kIjOBmQBjx47NuYHGX22TWvvlzveY88ec31uqddSz6YkqT77yDstnH9T3WPJgrle5ZdEpk4GwJoqpS1kDu6oe4seJVPVG4EZwyh39OKYpXrQC5a6FSB34TVyczr9rteeEI19SJpkmEmXZrcqYSrFUzABWzFK/peKV7XGb0dk2qZVTDxjrugxwRBt5do9zStDCuDrYF9PUr2LLHY8TkQ3Ap4GHRGSpP80ypZZIZVSDYHxRmaAISnpwF5wBXLd1569q24ed9z+ZOSnllhf3nMHpz+5WuotXrtvDGVMBRZU7qup9wH0+tcWUUOrCX9u6el0nG5XTT090JkClTn5KmQ7U973XuvNXte3DlFe+wNSOlHLLWOFLGWdVK6WQZkCymacDgNvCX9Ugec0br4tM6ghApCfKhQuf7/d+8H8p46xsBUVTxSzHPgBkCpyVkki/LF7VnveFJqrKnEUv9EuzeK2qmPNqi/mqpVJIM+BYYK8AP/crzeUc+QbOULD0Besn/ceYovL8iR2jEtwmX5V0tUWbFGSqmKViyszv/UpTj71g6TraOyKei2OlagmH6Ij0EBQhqkprPP/eEenJ/uYiXNW2D1PmPVHUnURymqWYpYwLZtvDmSplgb3M/N6vNCHTAGQmg5saWH35of0e2332QwW3Ixet8fRIsfnv1DRL6uQrYwYqS8WUWakG+QrNo7udt6U55PJKfySnR4rJf1dsUwtjaoAF9jIr1SCfHxtZg9Pz39LpfxomscbL1dP36etVe+XF8z2OMaY/S8WUWTH7lWaSz5K8mc47d8lactvGOXetLeF+67wkeOXFE+MEuR7HGNOfBfYyK9Ugn9sFIxu3Xq/fg6bZLlpeefFSXPyMGSgssFdAKQb5ki8YufTcW1vC/dqweFU7Vzyw1tc2gfvFI5uKVLgYU0cssNeRxAVjyrwnMgb31N5vpv1MMwkFhcGNDZ69/NSLRz6swsWYwtngaR1yG5RMTDlyG3jMpaImsbFFYsZoa0uYBSd8ktWXH8pPT5xY3slBxpiMrMdeh/JNZWSrqGkJh9Jq3Ys5nzGmtCywm4wVNeFQkLnT9sp6DEudGFM9LBVThxI589TNqL3WpHFL3YDTU7d6cWNqj/XY61C+yxZYKsWY+mKBvQ555czbOyLsPvsh18BtqRRj6oelYupQpuUJcknNGGNqm/XYq0DqtnXFpkFymYWanJrxOr/f7TLGlIcF9gorxfrsqTlzryV8N3ZEPM+/4s3N3LuyvSTrxhtjSktUc1252z+TJ0/WFStWlP281chrlqifC15lOge474Ga2HijlO0yxuRHRFaq6uRsr7Mce4WVYxPmTNvGeZ3HLaj73S5jTGlYYK+wcmzC3Daplaun70NrSzhtPXOv8ySWDihlu4wxpVFUjl1EFgDHAN3Aa8BXVbXDj4YNFKVanz2VVzmj1/mP37e1X47drV02uGpMdSp28PRRYI6q9orINcAc4OLimzVwVGpyUHJQbmkO0dQQYEukp9/5J+823LNdpdyU2xhTHN8GT0XkOOAEVT0l22tt8LSy3JbpDYeCeS0fUI5BX2NMf5UYPP0a8LCPxzN5WryqnSnznmD32Q8xZd4TnhOQMi05kKtyDPoaYwqTNRUjIo8Bu7o8dYmq3h9/zSVAL3BHhuPMBGYCjB07tqDGGm/5pEb8CMpeK0La4KoxlZe1x66qh6jq3i5fiaD+ZeBo4BTNkNdR1RtVdbKqTh45cqR/P4EB8uuF+1GJk6mE0hhTWUWlYkTkcJzB0mmq2ulPk0yyXNMr+fTC/QjKmUoojTGVVWxVzC+AJuBRceqen1bVs4pulQHyS6/kmhpJVMNEeqJ9s0tbC6zEsRUhjalORfXYVfUjqjpGVSfGvyyo+yif9EouvfDkDTjAmV2aeI0FaGPqh808rWL5pFdySY34UQ1jjKl+trpjFcu38iRbasRKFI0ZGKzHXsX8rjwpx7o0xpjKs8BexfyuPLESRWMGBkvFVDk/K09s02pjBgYL7AOMlSgaU/8sFWOMMXXGArsxxtQZC+zGGFNnLLAbY0ydscBujDF1xgK7McbUGQvsxhhTZ6yOvcySN5G2CULGmFKwwF5G+ayvbowxhbJUTBnZsrnGmHKwwF5GtmyuMaYcLLCXkS2ba4wpBwvsZWTL5hpjysEGT8vIls01xpSDBfYys2VzjTGlZqkYY4ypMxbYjTGmzhQV2EXk+yKyRkRWi8gjIjLKr4YZY4wpTLE99gWqOkFVJwIPApf50CZjjDFFKCqwq+r7Sd8OBrS45hhjjClW0VUxIvID4HRgC/D5oltkjDGmKKKauZMtIo8Bu7o8dYmq3p/0ujnAIFW93OM4M4GZ8W/HA9W0QMoIYFOlG+HB2lYYa1v+qrVdYG1L2E1VR2Z7UdbAnisR2Q14SFX39uWAZSQiK1R1cqXb4cbaVhhrW/6qtV1gbctXsVUxeyZ9Ow14pbjmGGOMKVaxOfZ5IjIeiAFvAmcV3yRjjDHFKCqwq+rxfjWkwm6sdAMysLYVxtqWv2ptF1jb8uJbjt0YY0x1sCUFjDGmzlhgj6vm5RFEZIGIvBJv330i0lLpNiWIyBdFZK2IxESk4pUBInK4iKwTkb+LyOxKtyeZiPxGRN4WkRcr3ZZkIjJGRJ4UkZfjv8vzKt2mBBEZJCJ/E5Hn4227otJtSiYiQRFZJSIPVrotySyw71DNyyM8CuytqhOA/wXmVLg9yV4EpgNPVbohIhIEfgkcAXwCOElEPlHZVvXzO+DwSjfCRS9woap+HDgA+GYV/bt1AQep6ieBicDhInJAhduU7Dzg5Uo3IpUF9rhqXh5BVR9R1d74t08DoyvZnmSq+rKqVstks/2Bv6vq66raDfwBOLbCbeqjqk8BmyvdjlSq+paqPhf//3/jBKqq2DRAHVvj34biX1Xxtykio4GjgJsr3ZZUFtiTiMgPRGQ9cArV1WNP9jXg4Uo3okq1AuuTvt9AlQSoWiEi44BJwDOVbckO8XTHauBt4FFVrZa2/RS4CKfcu6oMqMAuIo+JyIsuX8cCqOolqjoGuAP4VjW1Lf6aS3Bum++otrZVCXF5rCp6d7VARIYA9wLnp9zBVpSqRuMp0tHA/iJS8dntInI08Laqrqx0W9wMqK3xVPWQHF/6e+AhwHXdm1LI1jYR+TJwNHCwlrlGNY9/t0rbAIxJ+n40sLFCbakpIhLCCep3qOqiSrfHjap2iMifccYpKj0APQWYJiJHAoOAnUXkdlU9tcLtAgZYjz2Tal4eQUQOBy4GpqlqZ6XbU8WeBfYUkd1FpBH4ErCkwm2qeiIiwC3Ay6p6baXbk0xERiaqwEQkDBxCFfxtquocVR2tquNwPmdPVEtQBwvsyebF0wtrgENxRrurxS+AnYBH4+WY11e6QQkicpyIbAA+DTwkIksr1Zb4APO3gKU4A4ALVXVtpdqTSkTuBP4KjBeRDSLy9Uq3KW4KcBpwUPzztTreE60GHwKejP9dPouTY6+q0sJqZDNPjTGmzliP3Rhj6owFdmOMqTMW2I0xps5YYDfGmDpjgd0YY+qMBXZjjKkzFtiNMabOWGA3xpg68/8Bm1u6cKE0NSMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(data[:,0], data[:,1]) # Print the original data in blue\n",
    "\n",
    "# Apply PCA. In theory, the Eigenvector matrix must be the \n",
    "# inverse of the original rotationMatrix. \n",
    "pca = PCA(n_components=2)  # Instantiate a PCA. Choose to get 2 output variables\n",
    "\n",
    "# Create the transformation model for this data. Internally it gets the rotation \n",
    "# matrix and the explained variance\n",
    "pcaTr = pca.fit(data)\n",
    "\n",
    "# Create an array with the transformed data\n",
    "dataPCA = pcaTr.transform(data)\n",
    "\n",
    "print('Eigenvectors or principal component: First row must be in the direction of [1, n]')\n",
    "print(pcaTr.components_)\n",
    "\n",
    "print()\n",
    "print('Eigenvalues or explained variance')\n",
    "print(pcaTr.explained_variance_)\n",
    "\n",
    "# Print the rotated data\n",
    "plt.scatter(dataPCA[:,0], dataPCA[:,1])\n",
    "\n",
    "# Plot the first component axe. Use the explained variance to scale the vector\n",
    "plt.plot([0, rotationMatrix[0][0] * std1 * 3], [0, rotationMatrix[0][1] * std1 * 3], 'k-', color='red')\n",
    "# Plot the second component axe. Use the explained variance to scale the vector\n",
    "plt.plot([0, rotationMatrix[1][0] * std2 * 3], [0, rotationMatrix[1][1] * std2 * 3], 'k-', color='green')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The explanation of this chart is as follows:\n",
    "* The rotation matrix used to create our correlated variables took the original uncorrelated variables `x` and `y` and transformed them into the blue points.\n",
    "* The PCA transformation finds out the rotation matrix used to create our correlated variables (blue points). Using the PCA model to transform our data, puts back the variables as our original uncorrelated variables.\n",
    "* The explained Variance of the PCA is \n",
    "\n",
    "$$[1.0094, 0.1125] $$\n",
    "\n",
    "which is approximately\n",
    "\n",
    "$$[1, 0.333 * 0.333] = [std1^2, std2^2],$$\n",
    "\n",
    "the parameters of our original random variables x and y\n",
    "\n",
    "You can use the previous code to try with other standard deviations and correlations and convince your self of this fact.   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PCA as a strategy for dimensionality reduction\n",
    "\n",
    "The principal components contained in the rotation matrix, are decreasingly sorted depending on its explained Variance. It usually means that the first components retain most of the power of the data to explain the patterns that **generalize** the data. Nevertheless, for some applications, we are interested in the patterns that explain much less Variance, for example, in novelty detection. \n",
    "\n",
    "In the next figure, we can see the original data and its corresponding projection over the first and second principal components. In other words, data comprised of a single variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3X9wHOWZJ/Dvo9HYHoktj6koZTyg2Plx9i4YLKxKnFPtbpmwMXf8WAEBkoM9Ups611bd1hmWc0VcskHksoWvXAnxXfbqiqpsZavgsmZjUABdypDYOXYhJkiWjOPF3uQgGA92oRSWs0hjPBo998eoxzM9/XO6Z3q65/upcoHare53JPcz77zv8z6vqCqIiCg5uqJuABERhYuBnYgoYRjYiYgShoGdiChhGNiJiBKGgZ2IKGEY2ImIEoaBnYgoYRjYiYgSpjuKm37oQx/StWvXRnFrIqLYmpyc/I2q9rmdF0lgX7t2LSYmJqK4NRFRbInIW17O41AMEVHCMLATESUMAzsRUcIwsBMRJQwDOxFRwjCwExElDAM7EVHCMLATESVMJAuUiIIYf2Mcew7vwZm5M1jduxo7rt2BGz96Y9TNohgZm8pj9/4TeGe2gDXZDHZuW4/hgVzUzQoNAzvFyvgb4xh9eRTnS+cBAKfnTmP05VEAYHAnT8am8njwqaMoFEsAgPxsAQ8+dRQAEhPcORRDsbLn8J5KUDecL53HnsN7ImoRxc3u/ScqQd1QKJawe/+JiFoUPgZ2ipUzc2d8HScye2e24Ot4HDGwU6ys7l3t6ziR2ZpsxtfxOGJgp1jZce0OrEitqDm2IrUCO67dEVGLKG52bluPTDpVcyyTTmHntvURtSh8nDylWDEmSJkVQ40yJkiTnBUjqtrymw4ODirrsRMR+SMik6o66HYeh2KIiBKGgZ2IKGEY2ImIEoaBnYgoYRjYiYgSJnBgF5EVIvJzETkiIsdE5OEwGkZERI0JI4/9AwDXqer7IpIG8I8i8iNVPRTCtYmIyKfAgV3LifDvL32ZXvrT+uR4IiICENIYu4ikRGQawLsAXlDVV8K4LhER+RdKYFfVkqpuAnA5gE+KyFXmc0Rku4hMiMjEzMxMGLclIiILoWbFqOosgJ8CuMHi7x5T1UFVHezr6wvztkREVCWMrJg+Ecku/X8GwPUAjge9LhERNSaMrJjLAPytiKRQfqN4UlWfC+G6RETUgDCyYl4DMBBCW4iIKARceUpElDAM7ERECcPATkSUMAzsREQJw8BORJQwDOxERAnDwE5ElDAM7ERECcPATkSUMAzsREQJw8BORJQwDOxERAnDwE5ElDBhlO0laqnxN8ax5/AenJk7g9W9q7Hj2h248aM3Rt0sojpjU3ns3n8C78wWsCabwc5t6zE8kGv6fRnYKVbG3xjH6MujOF86DwA4PXcaoy+PAgCDO7WVsak8HnzqKArFEgAgP1vAg08dBYCmB3cOxVCs7Dm8pxLUDedL57Hn8J6IWkRkbff+E5WgbigUS9i9/0TT783ATrFyZu6Mr+NEUXlntuDreJg4FEOxsrp3NU7PnbY8To2Jahw46dZkM8hbBPE12UzT780eO8XKH1z+B76OkzNjHDg/W4Di4jjw2FQ+6qbF3s5t65FJp2qOZdIp7Ny2vun3ZmCnWHnx1Iu+jpOzKMeBk254IIdHbtuIXDYDAZDLZvDIbRuZFUNkxjH2cEU5DtwJhgdykQxrscdOsWI3ls4x9sbYjfe2YhyYmidwYBeRK0TkoIi8LiLHRGRHGA0jsrLj2h3oltoPmt3SjR3XJvuf3dhUHkO7DmDdyDiGdh0IbQw8ynFgap4whmIWADygqodF5HcATIrIC6r6TyFcm6jOoi46ft0uvGabuJ3XzIUuxvczKyZZRFXDvaDIDwF8R1VfsDtncHBQJyYmQr0vdYZPPfEpzC/M1x3v6e7BK3e/EkGLrJmDMVDuCZsnz7ycN7TrgGXaXC6bwUsj1zXxVVC7EZFJVR10Oy/UMXYRWQtgAED7PGGUKFZB3el4VLxmm3g5jxOc5FdoWTEicgmAfQDuU9XfWvz9dgDbAaC/vz+s2xK1Ja/B2Mt5US508YqLnNpLKD12EUmjHNSfUNWnrM5R1cdUdVBVB/v6+sK4LVHb8ppt4uW8dp/g5CKn9hNGVowA+C6A11X1W8GbRGQvk7IOhHbHo+I1GNudt3VDXyULZvf+E7h9cw6retKVc5Z3t0+mMhc5tZ8whmKGAPwJgKMiMr107L+o6v8J4dpENQol66ELu+NR8ZptYnXe1g192DeZr8mC2fvq20BVnsNsodiyErBuOAfQfgIHdlX9RwASQluIEsXrqkPzeUO7DtT1gIul+uw1o1ccdWCPwxxAp2mfz3NEBMBfTzfsXnEjC6HafQ6gE7FWDMWKQKCo771Kgj402vWA7c5thFUWC4CGFkIFWeTUaDYNs3Cchb5AyQsuUKJGbfzbjbZ/d/Teoy1sSfNYLVpKpwRQoLh48XlNdwkuWdGN2fmi76BotShqeXcXZgvFuvObtRDKaXEWYP9G4XXxVxJ5XaDEHjtRm7HrAVcfW5lJY+7CAs7OlwOxnzIDdlks5mOGd2YLNT3klZk0ROD7DcVrO0afOYYPFhZtPzk4ZeEkPbB7xcBO1IbsJl6rywyYe9deg5vfcflsT7qmh1x93yB1a+zaYfWpofq1MQvHHSdPiWIoSHCzG5df1ZO2nARVhW1vHmg8Z93v/EB+toChXQcsZlgau16SMbATxVCQOupWWSwAcHa+iBXpLmQz6Zodf85Z9KDNGukt22XTVC/EqiaA7aQys3BqMbATxdDWDX11eUBeg9vwQA63b85Z5hGdnS/ig4VFPHrXJrw0ch2GB3JYmbEOtNUa6S3bbR330M1X1gV8AWx76q3cci4uOMZOFDNjU3nsm8zXBDoBcPtm9wVRxiSoUzploVjCw88eq1zrwoL9MAwQrLfstIjr4WePVSaH7YK6ACxdbIGBnShmrLJCFMDB4zOO32eVJmjn7Hyxsjhpvmi/kUmuiTnk5x3ua+C4ujUGdqKQNLJoZmwqj9FnjlUyQXrSXVieTjmmEtr1tt3Gua3eENzOd9LMjT68tJXj6vYY2IkaZM7tnruwUKnpYqQBTrz1Hg4en0F+toCUCEqqlV4uAOz8+yM1i47mi4uVHrJVKuHYVN52vLlLBOtGxm3fEPxOcLqd38yg6nRvAbja1AUDO5FH1YE825PG++cXKkHZLvf6iUMnK0G4pLVBf3l3V01Qt2LOTd+9/4TteLP5+kBtbrmfUgXG+cb1zFb1pJsaVO3ayu0AvWFWDJEH5s0kzs4XXYMyYD/pVyiWLN8MrOSXVn4C3nvdVrnldmmOVmTpfLuUxIduvtLTdRrFwmLBsMdO5IHf8emwGT1wP71u85vA8EAOE2+9V/MpwooAuHtLf02PvNUFt4IUFiMGdiJL5olQP0MYXq0yDec4MXrgO7etr8tssRtzt8oYOXh8xvJcWbqIVQD1Wle+EU4Tzs28b9IxsBOZmNMC87MFxwUyjageznjwqddQ8JDal58tVHrd33/lbZRUkRLBlo+uwuGT5+qqHVoNW9gN5agC375rU0sDqdXPuV12hYo7jrETmdjliZtXaqZTUll+nxLv9eBTIqaVkt6+V+Ti4iRjorSkisMnz+H2zbm6FZxWwdEp77vVe5Ryr9TmYY+dyMS2V4ty0LQaNlg7Mu75+ouqGB7IYWwqjweePFIJ0m5UgdFnjlkGw4PHZyrZIsbwxv17p+vauXPbety3d7ru2kD96272Zhas0tg8DOzU0ayCl1Oq3c5t6yvnP/zssZrFRV6tyWbw1bGjePzQSd/ttbuXEQzthjeq8+md2mWwus79e6dx397p0Fabcq/U5mFgp0Rz6nXaBcHbN+ewbzJfN2a9dUNfzflGHRO/tm7oayioOzGCod3whpdMmJ3b1jvWkjG+P6yxcKuJYKY0hoNj7JRY5txzIyAZOeF2QfDg8RnLqoMHj88ETnlc1ZN2renipEvKW+JVM8rZDu06YNsjdxvsMf7e+Hm5CWMs3K66IydOg2OPnRLLbQs1u7Hc/NIwi3noO4yx3xuvvgxPBOitL2p5jN7I0qnO1gmSkpkSsRy/dxLGz4Mpjc0RSmAXkb8BcBOAd1X1qjCuSQTUD6XgMu/f6zY555SfXj3MYvT0sz3phodfDPsm86FcxxzUnXg5r6Ta0FwBtaewhmK+B+CGkK5FBMB6KMVjAgkA912G/CyxLxRLUIXn852ucz6kFayegroA//pjlwZud6ObelA0QgnsqvoigPfCuBaRIegyfqvAne4SzM5fwNqR8aW0P7Xdis3sXKFYGRMOwstipLCoAj//9dm6PHevMukUvn3XJjx616bK96VEKkNaxnwFtReOsVPb8juGO7TrgGX2S3Vp3X/5YAFzFy6+WRSKi/hgYREicP00sCabqYwJj03la3b4aWfFktbkuQNwnGhd1ZO2rQfPlaLx0LLALiLbAWwHgP7+/lbdlmLMT40W1YuTh+aAYwSdoV0HLMeRPZRqqRl6GJvK19VRb3fmN8md29bj/r3TlsM5Pcu6MfW1z9Ydd5uMpvbRsnRHVX1MVQdVdbCvr69Vt6UY8zMGbmaVjufnE4AIKuUCqtPwxqbyuP/J6VgFdaB+vmF4IGc7Rm+1AtWph8+Vou2HQzHUMn6XqFuVbv2t95IsdQHHV5VGBaYfqu21jk3lsfMHR3xN4LaDVJdYTnTmPKz89LJPKrNj2k8oPXYR+T6AnwFYLyKnRORLYVyXksNtsZCd4YEcXhq5Dm/uutH3zjnmgLNz23qkU97eGVZm6idUH372WGXrOy98vAeFZnl3/SOtNp8uvGxm4TaBzeyY9hRKj11VvxDGdag9hVEMKozx2bGpPFSXaoe7EJSX7ptdsrzb04TnuUKxUthrVU8aD918pafvM3LGc9kMtm7ow95X3/b1ZhDUBwv1GTeLKBcPM/+cvWxm4TTMElbNGAofh2LIkZea2V4CfxiV/HbvPwGs9nauorwYaPAjl2J4IIevjh11rZdi/n7D2fmibUVEs7u39OMbwxtrjoVdF6YRdouP3FZ+NmuTEWouBnZy5NbT9rpZgl2AyPak69IUjfua3yjemS2g12Ngr24nAF9BPYiDx2ccC2nFjVWhLgPTHdsXi4CRI7eettfNEuwyXM7OF2vG3Xf+/RHs/MERy7H4Ribp8rMFPPDkkZYEdeN+RvvbiddFWGbVhbqscGOM9sTATnWM9LZ1I+PoshnQNoKs1yGW4YEcbt+cc51QLC5q3Zh09X6fjfC6kUVYWjmm7kU6JZVt+BphTGDb/e6Y7th+GNiphjl7xSooVmdCuNVjqWa3kbIX+dkCdu8/EUmmSRwYP5dcNoN7tvTXlA/Y/blrQhkq8fO7pmhxjJ1q2KW3pUSwqGq51ZrTZgnVE6tB+7H52QIuWR1NGmG7MzJx/KaE+sGNMeKDgZ1q2H2sXlTFm7turDvulDLnZXELhafZQyJWtXdEgPv3TleGyjiJ2h4Y2KlGI/tQmlPm3JagU3O0YkikuggaC4K1L46xUw0vqxHtjE3lMfD153Hf3mkG9RZr9ZCI12woigZ77FTDy2pEsziVsE2K7NIwiF153WYLY8EZNQ8DO9VxWo1oXmW6dUMf9k3mOY7eQoL6AmWt1siQHbUOAztZsioTANRvtNAOy+U7TTsET2bItDcGdqqoXgpfvQHyxYkxbWhbt0w6hRXpLg7VhMCuuFmrNTJkR63DwN6BvPTGzTnnjQ61ZDNpjN5yZd31yZsulKszGszFzaqFUYXTD7cCYhQdBvYOY5emtry7qylBVwSYeOs9HDw+g0KxhJRIy5f4x1Uum8H8hYW6TzpW5Y6ZfkjVmO7YYezS1OzKugZ1dr6Ixw+drEy0Mai7y6RT+PZdm/DSyHWYtRm+MmefMP2QqjGwd5hG09G4jL91bt98cYjDa30Wph9SNQ7FNEmrxzu9tqNnWQpzF+qHXHqXpbCotWPp5t2AmNYYnqGPXYqX/997lvVzDh6fqfy/1+wTph9SNfbYm6DR/T1b0Q6roA4A6VRXpe62URXw0bs24ddVe42erwouvctSyFrsC0rOUiK4Z0s/nvgPn7Ytimb0so03ZWNuAij/Xh65bWNdJyHIimFKHvbYmyCM/T2rNdr7d9uIuNq5QrEmy8G45/17p7Ei3VWX5jh3oYShj63E4ZPn2Iv3aFVPGj3LuvHEoZM4eHwGq3rSlimga7KZusnQkmolUFv97pl+SNUY2JsgzPHOINkOfu5X/ZHdfE+73PVDb5zFFz51Rcu2nYu7s/PFSiDPzxaQ7hKkU1KzMYcRvBvpHDD9kAwcimmCMDckCJLtYHc/80So+SO7155+SRWPM6g3rLio6F3WXTP8ZQyzcDKUggglsIvIDSJyQkR+JSIjYVwzzsIc7wzygNvtM5pJd2FVT7oumPi5NoXjXKGIl0auw5tLcxl+s2GIrIgGzCsWkRSAfwbwRwBOAXgVwBdU9Z/svmdwcFAnJiYav+lrTwI/+Tpw7m1AUoCWgJVXAJ/5Wvnvf/J14NwpYOXl5WNX3+lwDYfz3M557i+Aye+V7y9dAKT8/6hauamodJFtUwZTy4FlvUDhvbq/0pqLVZHa69ndT+tOqL+OuW2292wDV6+7orzqyUwVr735dusbFJRY/7vw+nv3JgUMfhH45fPlZ8ZP45b1ABfmnM+BAplLy18Wzjo/d2Y1z1AK2PxF4KZvlZ+9H3354jORuRS48tal13AKyKxq7H4GL89/GEK+j4hMquqg63khBPZPAxhV1W1LXz8IAKr6iN33BArsrz0JPPufgKJFr7IrXX7oSxcuHktngJv/e+0P0+oa5vPcznnuL4CJ7zb2GqhhG9faB/ajv45hYE8qq+fOzO4ZWveHwMmf1T7HYdzP4OX5D0MT7uM1sIcxFJMDUP1EnVo61hw/+bp1UAeAxWL9P4Ziofw9btcwn+d2zuT3fDedqGNYPXdmds/Qm//XX1D3ej+Dl+c/DK26j4UwArvtJ8mak0S2i8iEiEzMzMxYfItH504F/x67a1QfdztHmeJH5MjtWQ37GfIaG7w8/2Fo1X0shBHYTwG4ourrywG8Yz5JVR9T1UFVHezrC1B2dOXlwb/H7hrVx93OkfpJSSKq4vashv0MeY0NXp7/MLTqPhbCCOyvAviEiKwTkWUAPg/gmRCua+0zXyuPU1npSgOpZbXH0pmLk6pO1zCf53bO5i/6bjpRx7B67szsnqF1f1j/HIdxP4OX5z8MrbqPhcCBXVUXAPw5gP0AXgfwpKoeC3pdW1ffWZ58WLn0IcF41195BTD8P4E//uulv5Pyf60mKmquYXOe2zk3fQsY/NLF+0tX4z2Q1PKLWQUNUuOPmv7f/MfmXNdrtskfx59Bo9dtxWsM8LMMLlX+t7ryCvdTa0g5W8vtHKD87zdzKRyfO7O6Z2ipnfc+U36Oq5+JzKVVr0Eau5/By/Mfhlbdx0LgrJhGBE53JFdDuw5YFoXKZTOV2i9OzKtPgfIjbFdErFUu2TBilxSD94/vavi6n/hwL375rlNaXzDplKB3Wbfv8shef1/UGVqZFUNtyCqoA94XH40+c6xu9akCkQb1ZvIS1IM8LMWSNlTzfv7CAtaNjGNo14GWF5Gj+GKtmAT66thRx79fOzIOAOgS4N99qh/fGN5YU2hsZSbdtI034sz/bq/epbsExcXaT8+pLqmpLeNnR6R2KRtN0WBgbzNhPJDff8V+oU516FhU4PFDJ/H4oZM15zCoN8eqnjTOFxfrhrfu3tKPwY9cWvN797olnhVuk0cM7G3EywPpJfBz+7n2k04JHrq5vKm33e+v+ve4bulTlZmXobSwy0ZT/DCwtxG3B9JrT4wbRref3mXdlgHcTpAdkVgZkjh52kbsHrz8bAFDuw7gvr3TjiV8x6byGNp1gEG9xVIilUqZds75HN4KUiGUlSGJgT0CRgA2Zzs4PXh2WS5A+Q2hehs88scpIHuxqFopu2t3Lb9BdXggV7dVodWWeFa4TR5xKKbFnIZTdm5bj/v3TvuumLsmm/G1DR7Vmr+wYLtNnRfVQdvr5tNeNLojErfJIwb2FnMaR39p5Drct3fa1/WMoHG/j+8TAI/etQnDA7lK6mMnOztfRJdNkfOedBeWdadsM4Uy6RS2bujD0K4DlSB6++YcDh6fiTSocpu8zsbA3mJuE1s5m0kzK7mqoPHws8c89zg//uFe7N5/wvebSJIt2nxMWtW73HHS8fbNOeybzNd8Ats3mfc8bELUDBxjbzG3iS277eyqCYB7tvRXtlIbm8rj/fMLntvwy3fnOBbvUX6p120ll83g4PGZhvekJWoWBvYWc5vYspo0G/rYpXVb1+2bzFcmXXfvP1G3apHCkRJx/J0xtZDaEYdiWszLxJZ5fHRo14G6CdVCsYSHnz2G3ftPsPfdRCVVx9+Z3c+fqYUUJQb2CPid2LLr/Z2dLzacydHJusR+TN3MSF+0+52FmQVDFBYGdgdRFVIy3zcbIBWP6i1qeYl/seQc3b0EaKYWUjtiYLcRVSElq/sC/nqZ5CyXzWDugwXLFMaUCBZVfQXo6t688aZ8/95pBnmKDAO7jagKKdktNFpUQKS8oURKBMu6BYViMwvJRiPdJbhkRXfTPqG45f0bq0gbwaqK1C6YFWMjqmwHp+sbJWBKqlhIYPc9l81g9x3X4KGbr3RN+Wz0+kZ+eTPqqTh1BohaiYHdRjMLKdnVigGAlZm0p2u4jQ/HUX62UAmCj9y20XILPCdONV8EqOT9A82pp8LUR2oXDOw2mlVIqbpYl+Lix3UjuPsNZklTPXzhp2iOEbi9FuEKUmTLDqsqUrvgGLuNZmU7WO0lWv1x3c/Ycm/EG0s3i/HzsKtJbsV4D/CTfhh2PRWmPlK7YGB3EPaDPzaVty0mVdNT9SCTTuGvbt2Iibfeq9vart0J3Dvj78wW8Ohdm+oCpdM1gWjTD5n6SO1CNIJNGQYHB3ViYqLl943a0K4Dtj1Qp12PMumUY8VAczZGOzMKl7kVIMtm0uhd3o38bKHys8llM/jtZTtsv+fovd7fGIniSEQmVXXQ7bxAPXYRuQPAKIDfBfBJVe28aO2D0ySa065HbmO/wwM5TLz1Hp44dNJ3LfdWEqDyhuQU2NNdgrkLF/PMS6qVIY2/fK1FjSWKsaCTp78AcBuAF0NoS+LZTaKt6knbTvrlsplKBUe7TBoAOHh8pq2DOlAefjHeoOxeb0rKeezmrJ9CsVR+M2j3F0nUBgIFdlV9XVWZpOuRXabNQzdfiZ3b1iOdqk+JeW/uA6wdGcf9e6dtM2mAeKTUZatSOe1+Ft+88xrMOkwgM64TuWtZuqOIbBeRCRGZmJmZadVt24pbil3JIjfdWF1qVd2xeuFLtsdb/ruh2VmVVv+w5i4sVN6MnH4WTA8kCsZ18lREfgxgtcVffUVVf7h0zk8B/GevY+ydOnnqxGli1Ukumyn31uXiylQv0ilBqaRoVlECuz1Ec9kMXhq5zvF7nSaDL9kwYpvrz8lTSrrQJk9V9fpwmkR2xqbyDddUr3yfzzGKZq5crbzZWPAyZGR8gnngySOOk8pEZI0rTyNm9E7jJCWCe7b0o3dZfT0XI3sl6CrM4YEcvnnnNZbzDkTkLGi6460A/geAPgDjIjKtqttCaVmHsKvm2K4EwDfvvAbDAzl8Y3hjpUytkW9eKJYqPW3zQiQj6Hutc28c87NRNxEFDOyq+jSAp0NqS2I5BbI4ZLNUu3tLf902fgBqxsSN4RPFxVWmxsIk87lupW2N1b+NzkEQdSKWFGgytxrdXuqhZDNpzF1YiLyiYzaTxjeGN9YcG5vKO46FG0HdmDAd2nXAslbOA08eAWBftzxub4BEUYr1GLvbop12YFeje/SZYwCs87mrZdIp3HTNZW2RwD16y5U1XxtvWm4TnNVB2S5Al1TrcvOrMQWSyLvY9tjbcbcaqyEXu0A2WyhibCpfVzhqZSYNEWB2vli5xu79J1CMeGON3mWpup+r1/mBLhGsGxl33b/VaYcqo3IiEbmLbWCPauu6atWBfKVpuCQ/W8B9e6cdFwIZbXWrImm3jVsrzVuUB/Y6PGL06POzBaS7xHEjabtrGj+fvzzi6ZZEHS22QzFR71Zj3jBjtlC0DFZO/WyvbW2HYQirNri1K2Wxkqi4qOhd1m35d27XHB7INX/JLFECxDawR71bTRhpil7bunVDX8vimQCed46yq/fy7bs24de7bsSizdj7uUIR37zzmqbsUEVEMQ7szdq6zqugnwy8tnVsKo+9P3+7rudvtTgoDAp43jLOrfaN05tvM7amI6Ky2I6xR71bjZ9t28xyPto6+swxy4nTZm2JZ5QJ9vpzdDrXbau4sHeoIqKy2AZ2INrAYBW0vDA2XbZjzqyx20qvGcL+xBP1my9Rp4p1YI+SOWh1OWxtV81pXN0qhdNJJp1yfGPxsrdoSgSLqk0LuuyVE7UeA3sA1UHLy76jbj1iPxOyq3rSeOjmKyt1WqzqslTvk5rtSeP98ws1wzqZdIrj2kQJxMAeEqthh60b+mw3oLbiZ0L2oZuvrHtjcRvy8Fp8i4jijYE9REGHHbxOyGYz6br7eLl3EoZFuqQLi1q/PUiXxDbBiyh0fBpCEkbdGre6MUB5+MRcs6WT3PGv7vB1nKgTMbCHwLwK1WqzaS+scrvv2dLPXO8qAx8eQEpq3/xSksLAhwciahFR+3Hd87QZkrbnqV2tcC/7e5I/n/3BZ3F67nTd8ct6L8Pzn3s+ghYRtU5oe56SO7e6NZy0DM+ZuTO+jhN1osQF9iiCqN2k55psJvTywp3+JrFy+UrMfjBreZyIyhI1xh7WWLdfTnVrnMoLm7lNwEb1+trJ3IU5X8eJOlGiArufIBomp4JWXssLewnaUb2+dlJU6xILdseJOlGihmKirNFulyPuNExTzcvGIVHXoCeieEhUjz3qGu1WvJYX9hK02/H1EVH7CRTYRWS3iBwXkddE5GkRyYbVsEZEXaPdite6416Ctt/XF4fNvv3KpKx/TnbHiTpR0KGYFwD+3JLGAAAGhElEQVQ8qKoLIvLfADwI4MvBm9WYdi0TazVMY85u2bqhD/sm87a1y43rAN5eXztu9h2GWz5+C/ae2Gt5nIjKQlugJCK3Avicqt7tdm7SFij5ZVUJ0lyNMeibUlIXTf3+3/2+ZbpjdnkW//D5f4igRUStE8UCpT8FUN+VSrBGc8rtJkoPHp8JLegmdaLVKqg7HSfqRK6BXUR+DGC1xV99RVV/uHTOVwAsAHjC4TrbAWwHgP7+/oYa206CDHW0Iuh6zcYhouRxnTxV1etV9SqLP0ZQvxfATQDuVodxHVV9TFUHVXWwr68vvFcQkSA55a3IbmnHiWQiao2gWTE3oDxZeouqzofTpNYImjESpNfdiqDrNRuHiJIn6Bj7dwAsB/CCiADAIVX9s8CtarIwMkYaHeowxuULxRJSS/uk5rjfqGcrl63EuQvnLI8TUVmgHruqflxVr1DVTUt/2j6oA+EszW+k111dNgAASqqV70laAG6WC6ULvo4TdaJErTz1KozJy0aGOljrJbhCyfp3ZHecqBMlqlaMV2FljPgd6khqCiIRtZeO7LFHlTHCWi/BZZdbV62wO07UiToysEeVMcIUxOBGPjmCdFe65li6K42RT45E1CKi9tORQzFANBkj7VrLJk5u/OiNAIA9h/fgzNwZrO5djR3X7qgcJyJuZk1EFBtea8V05FAMEVGSMbATESUMAzsRUcIwsBMRJQwDOxFRwjCwExElDAM7EVHCxHKBUqNb0lEyjL8xzgVKRA5iF9jDqKVO8TX+xjhGXx7F+dJ5AMDpudMYfXkUABjciZbEbiiGpW87257DeypB3XC+dB57Du+JqEVE7Sd2gZ2lbzvbmbkzvo4TdaLYBXaWvu1sq3tX+zpO1IliF9hZ+raz7bh2B1akVtQcW5FagR3X7oioRUTtJ3aTpyx929lYtpfIHcv2EhHFBMv2EhF1KAZ2IqKECRTYReS/ishrIjItIs+LyJqwGkZERI0J2mPfrapXq+omAM8B+FoIbSIiogACBXZV/W3Vl70AWj8TS0RENQKnO4rIXwH49wDOAdjqcN52ANsBoL+/P+htiYjIhmu6o4j8GIDVsr6vqOoPq857EMAKVX3I7aZMdyQi8s9ruqNrj11Vr/d4z/8NYByAa2AnIqLmCTQUIyKfUNVfLn15C4DjXr5vcnLyNyLyVpB7h+BDAH4TcRuaKemvD+BrTIKkvz4g3Nf4ES8nBVp5KiL7AKwHsAjgLQB/pqr5hi/YQiIy4eUjTVwl/fUBfI1JkPTXB0TzGgP12FX19rAaQkRE4eDKUyKihOnkwP5Y1A1osqS/PoCvMQmS/vqACF5jJNUdiYioeTq5x05ElEgdHdhFZLeIHF8qZPa0iGSjblOYROQOETkmIosikpjMAxG5QUROiMivRGQk6vY0g4j8jYi8KyK/iLotzSAiV4jIQRF5fenfaKK2wBKRFSLycxE5svT6Hm7l/Ts6sAN4AcBVqno1gH8G8GDE7QnbLwDcBuDFqBsSFhFJAfhrAP8GwO8B+IKI/F60rWqK7wG4IepGNNECgAdU9XcBbAHwHxP2e/wAwHWqeg2ATQBuEJEtrbp5Rwd2VX1eVReWvjwE4PIo2xM2VX1dVU9E3Y6QfRLAr1T1DVW9AODvAPxxxG0Knaq+COC9qNvRLKp6WlUPL/3/vwB4HUBi9rfUsveXvkwv/WnZhGZHB3aTPwXwo6gbQa5yAN6u+voUEhQQOpGIrAUwAOCVaFsSLhFJicg0gHcBvKCqLXt9sdvM2i8vRcxE5CsofzR8opVtC4PXIm0JIhbHmNoVUyJyCYB9AO4zlQGPPVUtAdi0NHf3tIhcpaotmTNJfGB3K2ImIvcCuAnAZzSGuZ8+irQlxSkAV1R9fTmAdyJqCwUgImmUg/oTqvpU1O1pFlWdFZGfojxn0pLA3tFDMSJyA4AvA7hFVeejbg958iqAT4jIOhFZBuDzAJ6JuE3kk4gIgO8CeF1VvxV1e8ImIn1Glp2IZABcD49FEsPQ0YEdwHcA/A6AF5b2bf1fUTcoTCJyq4icAvBpAOMisj/qNgW1NNn95wD2ozzh9qSqHou2VeETke8D+BmA9SJySkS+FHWbQjYE4E8AXLf07E2LyL+NulEhugzAQRF5DeXOyAuq+lyrbs6Vp0RECdPpPXYiosRhYCciShgGdiKihGFgJyJKGAZ2IqKEYWAnIkoYBnYiooRhYCciSpj/D1SUYgM6Wa+RAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "nPoints = len(data)\n",
    "\n",
    "# Plot the original data in blue\n",
    "plt.scatter(data[:,0], data[:,1])\n",
    "\n",
    "#Plot the projection along the first component in orange\n",
    "plt.scatter(data[:,0], np.zeros(nPoints))\n",
    "\n",
    "#Plot the projection along the second component in green\n",
    "plt.scatter(np.zeros(nPoints), data[:,1])\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PCA as a strategy to plot complex data\n",
    "\n",
    "The next chart shows a sample diagram displaying a dataset of pictures of cats and dogs. Raw pictures are composed of hundreds or even thousands of features. However, PCA allows us to reduce that many features to only two. In that reduced space of uncorrelated variables, we can easily separate cats and dogs. \n",
    "\n",
    "<img src = 'catdog.png'>\n",
    "\n",
    "You will learn how to generate a chart like this with word vectors in this week's programming assignment."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}