chapter4-training.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training Models\n",
    "\n",
    "* Closed form or Gradient Descent (Stochastic, Batch, Mini-Batch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Linear Regression\n",
    "### Representation \n",
    "$\\hat{y} = \\theta^{T}\\cdot x$\n",
    "* Represent one instance of a prediction (an $x_i$)\n",
    "* $x$ is feature vector - $x_0$ always is (1) - for constants\n",
    "* $\\hat{y}$ is prediction (1 value)\n",
    "* $\\theta^{T}$ is parameter vector. Transposed to dot because ML usually represents as column vector\n",
    "\n",
    "### Error\n",
    "$\\text{MSE}(\\theta) = 1/m\\sum_{i=1}^{m}(\\theta^Tx^i - y^i)^2$\n",
    "* Diff predicted value to real value for all points (squared for non-negs)\n",
    "\n",
    "### Closed Form Equation\n",
    "Find the parameter vector that minimize MSE --- $\\hat{\\theta}$\n",
    "\n",
    "$\\hat{\\theta} = (X^TX)^\\text{-1}X^Ty$\n",
    "\n",
    "* $X$ is matrix with row = data point, col = features\n",
    "* $y$ is vector of $y_1$ to $y_m$ (1D array with m values)\n",
    "* Runtime\n",
    "    * $O(n^3)$ - n being number of features -- because Inverse of $X \\cdot X^T$ is $O(n^3)$\n",
    "    * Scales linearly with number of samples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Trying it out\n",
    "* Using GDP vs Hapiness data downloaded and processed in (datasets/gdp-happiness)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using Closed Form"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Country', 'GDP Per Capita', 'Life Satisfaction'], dtype='object')"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gdp_happiness = pd.read_csv(\"datasets/gdp-happiness/gdp-vs-happiness-2017.csv\")\n",
    "gdp_happiness.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x11df16588>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3X2cnGV97/HPb2YfkuyGJC4hGkJ4aEBPQiFAWsAgSmitQgD7Eq0KhdaXpfaAUB8gejxII6fnFEGrCC3NoU9IbIWggEi1loBKjsQmNIkQoK4gJIFCiCQmIezDzO/8cd+zzO7eM3vP7NzzdH/fr1deu3vPPfdcO5n9zTW/67p+l7k7IiLS/jKNboCIiNSHAr6ISEoo4IuIpIQCvohISijgi4ikhAK+iEhKKOCLiKSEAr6ISEokGvDN7ONm9riZPWZm/2RmU5J8PBERKc2SWmlrZocCDwML3f2Amd0B3O/u/1DqPgcffLAfccQRibRHRKQdbdy48WV3nx3n3I6E29IBTDWzIWAa8Hy5k4844gg2bNiQcJNERNqHmT0b99zEUjruvgO4AXgOeAHY4+7/mtTjiYhIeYkFfDObBZwHHAnMBXrM7MKI8y4xsw1mtmHnzp1JNUdEJPWSHLT9LeAZd9/p7kPAN4G3jj3J3Ve5+xJ3XzJ7dqw0lIiIVCHJgP8ccIqZTTMzA84Enkjw8UREpIwkc/jrgTXAo8BPw8daldTjiYhIeYnO0nH3a4BrknwMERGJRyttpW3s2jfA5m272bVvoNFNEWlKSc/DF6mLezbtYMVdW+jMZBjK5/nCe4/j3MWHNrpZIk1FPXxpebv2DbDiri28NpRn78Awrw3luequLerpi4yhgC8tb/srB+jMjH4pd2YybH/lQINaJNKcFPCl5c2bNZWhfH7UsaF8nnmzpjaoRSLNSQFfWl5fbzdfeO9xTOnMML27gymdGb7w3uPo6+1udNNEmooGbaUtnLv4UJYuOJjtrxxg3qypCvYiERTwpW309XbXLNDv2jegNw9pOwr4ImOkbYqn3tzSQwFfpEjxFM/XCAaCr7prC0sXHNyWwTBtb25pp0FbkSJpmuKp9Qvpo4AvUiRNUzzT9OYmAQV8kSKFKZ7dHca0zizdHTZqimcj6vUk9ZhpenOTgHL4ImM4AAYGuI0cb0S+O8nHLLy5XTXm+u04ViEBc/dGt2HEkiVLXJuYSz2Umpmya98AS69by2tDr/d8p3RmuO+y01h+08Pjjq9bsSyxAFmqLbV+TM3SaW1mttHdl8Q5Vz18SZ1yveZCXrswQweCvPambbsjj29/5UBiQbJUW2r9mLVcvyDNTTl8SZWJZqaUymsvPmxm3fPdyrFLrSngS13EHXhMelB0opkpperyLJgzve71elQjSGpNKR1JXNyBx3oMisbpNZeqy9OIej2qESS1pB6+JCru4p56LQIq1WsGRn2y6OvtZt6sqWx/5cCoNvT1dnP8YTPrGngb8ZjSntTDl0TFHXis1wAljO4193Rluf+x/+LKNWvpyr7+ycJBJQek7SjgN4F2nhYXd+Cx3gOUfb3dPNz/Mlet2czAcDA1eWA4ePwr12wGjIHhdNTTkfRQSqfB7tm0g6XXreXCW9ez9Lq13LtpR6ObVFNxBx7rPUBZSCEVgn2xrGXIZmzUMZUckHagHn4DpaUyY9yBx3oOUEalkApynh+1whY0HVLagwJ+A9Uzb91ocRf31GsRUFQKCaC7I8P15weDuCo5IO1GAb+BtLCmccbWkRnM5bjsjKP50MnzRwK7pkNKu1HAbyAVr5q8yQx4T5RCUskBaTcK+A2mhTXVq8VCrUJQL6zw1f+BtDMF/CagnmTlajngrW3+ytu1b4DHn98DGIvmHqTXao3Vc1p2YgHfzN4MfKPo0FHA59z9y0k9pqRHrQa80zJTqlr3bNrBp+7czFAumL7akYEvvX+x3hBrpN6djcTm4bv7U+6+2N0XAycBrwLfSurxJF1qNeDdyG3+GrF7ViV27RvgqjVbRoI9wHA+WJjWrG1uJY3YU7heC6/OBH7u7s/W6fGkzdVqoVajZkq1woK77a8cGLcADYKFaVqENnmN6GzUK4f/AeCf6vRYkhK1GPBuxEypVkkjzZs1lVx+/ErknGvqcC00orOReMA3sy7gXOAzJW6/BLgEYP78+Uk3R9pMnAHviQbF6j1TqlUW3PX1dnP9+cfxyTE5/OvPP76p2tmqGtHZqEcP/93Ao+7+YtSN7r4KWAXBnrZ1aI+kSNxBsXrOlGqlBXeFN0PN0klGvTsb9cjhfxClc6QBGjEoFker7WTV19vN6cccwunHzG7aNrayeu53kGgP38ymAb8N/HGSjyMSpZlTJ1pwJ42QaMB391eBviQfQ6SUZk+daMGd1Jvq4UviGjXfvNVSJyJJU2kFSVSjyxYodSLyOgV8SUyzzDdX6kQkoJSOJKaRZQtEZDwFfElMtYOmzV5jRqRVKaUjialmJWGjc/4i7UwBXxJVyaBpnJx/PWuHi7QbBXxJXNxB04kWSqn3LzI5yuFL0yiX82/WMgkirUQBv4212uBnuYVSmvEjMnlK6bSpVk1/lMr5N3uZBJFWoB5+G2r19EdU9UCVSRCZvNT38Ntx1kczV4mcDJVJEJmcVAf8Vk17TKSd0x8qkyBSvdSmdKLSHleu2cwP//Ollkl9lKL0h4hESW0PPyrtMTDsfPT2R8m7N0VvfzLpJqU/RGSs1Ab8qLQHwKuDOaAxVR2L1SLdVK/0RzuOg0h6tfPrObUpneK0x7TO7LjbsxnjwScbk96pdJZNI+fb37NpB0uvW8uFt65n6XVruXfTjrq3QaRW2v31nNqAD0HaY92KZdzy+yfR3TH6qdg/kONz9zzGqf/nAVY/8mzNHjNOcK5kkVGcF2hSbwitPv1TpFgaXs+pTekU9PV2c/oxs7n+/KCqYzZj7B8I0jqvDgUpn8/e/RgYXHDy4ZN6rLhpmrizbOIUG0tyJlK7Tv+Mo50/9qdVGl7Pqe7hFyv09lees4hpneOflpXf3jqpd/pKeg9xZ9lM9Ekg6R5LT1eWgVx7Tv8sp90/9qdVO09nLkh9D79YX283Z7zlEIa/5eNu68zapN7pK+09xJllM9ELNMkeS+GTg3nwXE0J3yTjTv9s1R5ys2zbKLVXzf4NrUYBf4y+3m6uOWdRkMYpksv7pN7pq+k9TDTLZqIXaFI9luKgV5DPO/df/jYWzJk+4f0bteCtFm8yafjYn2btPp1ZAT/CBaccDhakcTqzRi7vk36nT6r3UO4FOtnHjAqQu/YN8OCTL5E1G3Vud0eW/eGU1omuObaHfOWaLcyc1sWiuQcl9gdWqzeZNHzsT7t2Xs0dK+CbWRaYU3y+uz+XVKOawQUnH867Fr2xZu/0u/YNcHhfD/dddhr7B3N16z1U22OJCpAOrLhrCx0ZGxfci4NeuZ509IK3PB/92kbyJLPgrZZpmDR87Jf2NWHAN7OPAdcAL8LIX6kDxyXYrqZQ7p2+kvRAVPA8/rCZNWljnJ5rpT2W6F74ZsAYGB7du+3pypLz1z8BTdSekgvehpJb8FbrNEy7f+yX9hWnh38F8GZ335V0Y1pFJemBJAf5krp2VIDMWgZGZ3Ho6c6y8pxFnPGWQ+jr7Y7VnuIecsZsZGVzQRL58CTSMO38sV/aV5xpmduAPUk3pFVUOtUxyZ2aKrl2JYuvogJkzvPk8qNnL+XyPhLsK2nPyIK3C0+ku2P0u0gS+XAVk4uv1XZJk8rE6eE/DTxkZt8BRl4F7v6lxFrVxCpNDyQ5yBf32pUOWPb1dnP18oXjBq2BsrnrqPYM5vLsOTDErn0D4waUTz/mEK4///i65MOVhplYu5YLl9fFCfjPhf+6wn+pVmkAr/Ug39ixg4muXU3a555NO7j2vq10Zoyh4TzXnLNo5A+/XNAc257XhnPk8nkuXf1oyQBSz0CsNExpWl+QDhMGfHdfCWBm04MffV/ci5vZTOBW4FiCgd4Pu/uPq2xrzUxmPnY1AbxWQa1UD6zctSv9RBI1x/7a72zlXce+cSRgxvldH39+D3902wYGcrB3YBgoHUAUiBtP6wvSIc4snWOBrwFvCH9+GbjI3R+Pcf2vAN919/PNrAuYNpnG1kItPrZWE8AnG9Qm6oGVuvZEn0jGvvnV4g+/r7ebGVO76MpmGRgervo6Uj9aX5AOcQZtVwGfcPfD3f1w4JPA/53oTmZ2EHA68LcA7j7o7rsn09jJqmVtmaiNtpNU7eBvuQHLqJowtfrDVwBpLRrYToc4Ofwed3+w8IO7P2RmPTHudxSwE/h7Mzse2Ahc4e77i08ys0uASwDmz58fu+HVaIWPraXSTZMJoFGfSEp9Yli3YlmslNVEaTEtUGo9Gthuf7Fm6ZjZ1QRpHYALgWdiXvtE4GPuvt7MvgJ8Gri6+CR3X0XwKYIlS5aMr1pWQ83e6yyXbppsAB2b9in35jfRH37ctFhSAaRVC6+1Ao2ntLc4Af/DwErgmwRLb34I/GGM+20Htrv7+vDnNQQBv2GqDZr1CDBxZknUKoDu2jfAngNDDJYpbVzqD7/S2RyVBJA4z7OmDopUL84snVeAyyu9sLv/l5ltM7M3u/tTwJnA1iraWFOVBs04AaaeVRgn2wMr/n1y+TydWWNKRzb2m19SabG4z7OmDopUr2TAN7Mvu/ufmtm3CaZUjuLu58a4/seA1eEMnaeJ98mgpqKCcSFoFlYVlgrU9dxRqh7ppqjfp7sDbr7gBBbNnREraCbRzriBvBXGYESaWbkefiFnf0O1F3f3TcCSau8/WeWCcZxAPVGAabUqjFG/T1c2y4ypXRWNBdS6nXEDebOPwYg0u5IB3903ht8udvevFN9mZlcAP0iyYZNVLhgDsQJ1vXeUSnqWRK0CZq3bGbddmvkjMjlx5uFfHHHsD2rcjporN2897pz2ieYmJ1WFMan5/bWca13LdlbSrkLhtds/cjLrVizTgK1IBcrl8D8IfAg40szuLbppOtD0pZInCsZxA3Vxb7anK9jVqVAIrFyPs1mnDlbTO6/H71JJuzR1UKQ65XL4/w94ATgY+GLR8b3AliQbVQsTffyvJDXQ19vNw/0vR+b8owJVs08drCRg1vN3USAXSZa5l1/rZGZHAc+7+2vhz1OBOe7+i1o3ZsmSJb5hw4aaXrNc7zRuz3XXvgGWXrd2VEGxKZ0Z1q1YFnnNuOc2u3b6XUTalZltdPdYk2Pi5PDvAIrzHzngzmoa1gjlcs2lbhu7CUQldWyS2PCkUZtSJLl5i4jUX5yVth3uPlj4wd0Hw3n1LSVubz4qhbF0wcGxc/5xB3JLtWfs8aRTKuWeF02DFGkvcQL+TjM7193vBTCz84CXk21WbcUNmlFTOT9152buv/xtsXP+caYOlmrP2ONXL1/ItfdtTWxl6epHnmXlfVvpyhrD4a5Wxc+LpkGKtJc4OfxfA1YDcwlq6WwjqIffX+vGJJXDj5uH3rxtNxfeun5kw46Cro4MN5x/XEWzW8r14KPac99lp7H8podHHe/MGl1ZY//g68emd3dw+0dO5vjDZlb2RIyx+pFn+ezdj406Vm5cohlnHIlIZTn8OLV0fg6cYma9BG8QeyfbwHqqZHFUVAoDYHA4P1I6OG6gLTXjpFR7Nm3bPe74UM4Zyo1+Q65FSmXXvgFWfnv8/jXZjEU+L5o9I9Ie4qR0MLOzgUXAFDMDwN0/n2C7aqaSPHQhhfGpOzczOCbQ1qpmS6n2LD5sZuSbTUFPd3ZkM/HJtmH7KwfozGYYzOVGtyPnys+LtLEJZ+mY2S3A7xEUQjPgfcDhCberZsau4uzuyHDpOxaUPP/cxYdy/+Vvo6tj9FNTq8HKUqtKF8yZzhfee9y4xwXo6cqy8pxFNVtZOm/WVHIRqbxrzlmonrxIG4uTw9/i7scVfe0Fvunu76x1Y5LI4Rfs2jfA6vXPcfODP6Mrm51wxsu9m3aMG6ysx+yY/hf3ctaNPxr1CSOJue+F3y9rxlAuzzXnLOKCU1rmfVxEQjXN4QOvhV9fNbO5BGUVjqy2cY30Vw/1MzDsIxtrl5vxknQhs1J58QVzpnPD+45PfGaMtrMTSZ84Af/bZjYTuB54lKA2/oSbmDebaipbNmqwsl7BWIOxIulSrnja+9z9TuB2d98N3GVm9wFT3H1P3VpYI622iEjBWERqrdyg7WfCr3cVDrj7QCsGe6htaWCRJDWqlIa0v3IpnV1m9iDjyyMDsbc4bCpRqZL+F/eyadtuFh82kwVzptfkcZppoVIztUUm1uyVVqW1lQv4ZwMnEmx1+MUy57WU4lTJ5+7+Kbc98tzIbRedOp/Pn/frIz9XEyyb6Q+2mdoiE9Mm7ZK0clscDgKPmNlb3X0ngJllgF53/1W9GpiU/hf3jgr2ALf9+DkuOuUIFsyZXlWwbKY/2GZqi8SjTdolaXHKI3/FzA4ysx5gK/CUmV2ZcLsSt2nb7pLHi4Pl3oFhXhsKSitMlFONU064XvlZlTZuPa02sUBaT5yAvzDs0b8HuB+YD/x+oq2qg8UlauIsPmxm1cGy1B9sT1eWzdt2s/qRZ1l63VouvHU9S69by72bdoy7Rq3eEBQ8Wo8mFkjS4szD7zSzToKAf5O7D5lZ+eW5LWDBnOlcdOp8bvvx6Bz+gjnT2bVvoKpgGVVO+P1L5rH8pofpyBj7BoLaNaVSLLXMuau0cWvSgjhJUpzSCpcDK4DNBAO58wnm5r+t1o1JsrRCKcWzdGb1dI38oa3rf7nq0gqFwd6eruy4ksfFiksdJ7WdoGbpiLS3WpdHvhG4sejQs2Z2RrWNazYL5kwvOUi7bsWyqoJlYSbQ5oiSx8WKPzU8/vweMmEl0oJaDNiVW8ClNwORdCm30vZCd7/dzD5R4pQvJdSmuis1o6WS+vdRStXX7zDIZm0kxXLPph1ctWYzA8O1r31fiqZsiqRPuUHbnvDr9Ih/vQm3q65qNaOlMODa/+JeNoezgK5evnDceUFcD3rz/S/u5co7xwf77o7xA3a1GtCtdhaSiLS2cvPw/yb89t/cfV3xbWa2NNFWJSDpzboLPWaA14bydGcNyxiXvmMBPV1Z9g+O3mxkYDjPJ+/cHDzWmM1WpnVmueX3T+L0Y2aPu34teuSa7y2STnGmZX415rGmdc+mHbz1Lx7gg6se4a1/8cC46ZCTnQ43KiUUDroO5JzXhvLc9ODPGM5HD4xHbWEIkMdZNPegyOvXokeuKZsi6VQuh38q8FZg9pg8/kFANs7FzewXwF4gBwzHHUmupV37BvjkHZsYzhM2Az5xx6ZxK04nMx0uqsdc0JXNcsnpR3HTgz8bl7aJ0lWU2y93/cn0yDVlUySdys3S6SLI1XcQ5O0LfgWcX8FjnOHuL1fRtpp4/PlfhcH+dcP54HhxygSqL0lcanAWgp7zh06ez4dOns/X1z/HTQ/205XNMJjLk8vnR7WtqyPD/R87bVwRtyR65JrvLZI+5XL4PwB+YGb/4O7P1rFNNVaqV127tWPFPWYYncMv7jl/7Myj+dDJ88vO9Y+q2JlUj1w190XSJc5K21fN7HpgETClcNDdl8W4rwP/Gq7M/Rt3X1VdM6u3aO4MOrM2KlfemTUWzZ1R08cp7jEXBmmjes7FQbaSXrZ65CIyWXEC/mrgG8By4KPAxcDOmNdf6u7Pm9khwPfN7El3/2HxCWZ2CXAJwPz582M3PK6+3m6++L7juXLNFrIZI5d3rj8/mXx1NT3mSu6jHrmITEac0gob3f0kM9vi7seFx37g7m+v6IHM/gzY5+43lDonydIKUdMytdJURFpdTUsrAEPh1xfM7GzgeWBejEb0ABl33xt+/07g83EaNVlRu1iN7R1rpamIpE2cgP+/zGwG8EmC+fcHAR+Pcb85wLcsqA/TAXzd3b9bbUPjmmgXK9DmICKSTnGKp90XfrsHiF00zd2fBo6vsl1VmWgXq4Jq5rUr/SMira7kSlsz+yMzOzr83szs781sj5ltMbMT6tfE+MrtYlWs0nnt92zaMeHGJbVSrx2xRCR9ypVWuAL4Rfj9B4HjgKOATzC6XHLTKLeLVbFKSinUs9BYPd9YRCR9yqV0ht29MGC7HLjN3XcB/2ZmX0i+aZWL2sXqrF+fw6yernHnxp3XXq9CYxpXEJGklevh583sTWY2BTgT+Lei25q2ytZJh7+B7o4MHeFeImuf3Fmyt9zX283xh80sG1DrVWiskhLNSvuISDXK9fA/B2wgKJR2r7s/DmBmbweerkPbKlboJQ8UFagpVK+8cs0WZk7rYtHcgyrevaoehcbivrFoOqmIVKvswisz6wCmu/srRcd6wvvtq3VjJrvwavO23Vxw6yMjm4WPNa0zSx6vKkjWY5bOvZt2lN1HN6l9b0WkddVs4ZW7DwOvjDm2fxJtS8yufQN897EXSgZ7gFeHgtuuXLOZmdM6WTR3xrgdpUoF9UrKGlT75jDRuII2LhGRyYiz8KrpBXvCjk7llDMw7Hz09kfJ++u9/VqlSiZ7nXJvLNq4REQmI86OV00tKm8fx6uDuZEplv0v7q3J1Mukp3BOdmcuEUm3CXv4FtRGuAA4yt0/b2bzgTe6+08Sb10M5XabAujpzvLaUJ5ciW0GOzMZNm3bXZNUST1SLiqTLCLVitPD/yvgVILFVxBsWXhzYi2qULndpt6/5FD++oIT6SjzWw7l8yw+bGZNUiVRbRkYztHTFWtHyNjiTCcVERkrTsA/2d0vBV4DCGfsjF/J1CCFNEd3RFS/d/MLgNGVHR9wp3VlR1IiC+ZMr0mqpDjl0p0NFgJkMsbymx7WqlkRabhY5ZHNLEu4J6CZzYYS+ZMGOXfxocyc1sVHv7ZxZCYOBOmUXx0YZGB49Myd7g7jlgtPHDVLp1apknMXH8rCNx3EWV99GPCRKZQTrZpVcTYRSVqcgH8j8C3gEDP7c4INzP9noq2qwqK5B5Efs0/tgaFhPnnnFjIZg5yP2mf29GMOGXeNSqZeRtXcL9g/mKM7m2FwOF4uX4upRKQeSgZ8MzvS3Z9x99VmtpGgvIIB73H3J+rWwhgKveOrz17Itd/ZSmcmw2AuR94ZNXvHzfjOZadFbhReiYlq7lcyfVI1dESkXsr18NcAJ5nZA+5+JvBkndpUkbG946uXL+TYuTPYc2CQS1f/B0O54ZFzu7MZ9g+WXpgVR5ya+5WUY9BiKhGpl3IBP2Nm1wDHmNknxt7o7l9KrlnxRPWOV977OPdf/jbmzZqRyCKlcjX3iz85xB0T0GIqEamXcrN0PkAwM6cDmB7xr+GiKkwO5pyzvvow6/pfTmSRUpya+4VqlsCE0ye1mEpE6qVs8TQAM3u3u/9LPRpTafG0qGJiBYWiYhC8MfR0Zdk/mKt4FkzU7JnP3fPTUTX3i3P41Q7AapaOiFSjJsXTzOxCd78dWGhm/23s7c2Q0unr7eb9S+aNCr4FhTz48YfN5OH+l6sKwqXGB6448xguOuWIcbN0JjMAW8kMIRGRapTL4feEX3sjbiv/saBOdu0b4I4N2yNvK+TBqw3CUff77Lceo6crSy4sunb+ksNG3UcDsCLSzEoGfHf/m/DryrG3mdmfJtmouLa/coCsWeRtVy9fSF9vN5urrJNTqkZPYZZP1JuGBmBFpJlVWy1z3KydRpg3aypDufH5+2ldGY6dO+P1c6oIwuVq9ED09oMagBWRZlZtwI/uVtdZX28315yzaNzx4TwjBcuqDcLF9+vpHl+Lp9SbxrmLD2XdimXc/pGTWbdimVbMikjTmHCWTuSdzJ5z9/m1bky1WxyuXv8sK7+9FSNYWVtcQqEQcMvNgolz22M79oys4lX5AxFpFpXM0ikZ8M1sL9GDswZMdfea75Y1mT1t+1/cy1k3/ojB3OtNjrPfayXTKKt90xARSUpNpmW6e1Msropr/2CO7o4sg0WlFCYanK10Bk+pqZMqfiYiraDltzgsmDdrKvsHhkcdOzA0XHZwNmqlbtRgbDlJb2soIlIrbRPwr/vuk+OK9FuJKZsFUTNxBnN59hwYih2wa/GmISJSD4kHfDPLmtl/mNl9ST1G/4t7IxdgdWasbOAdO4OnM2vk8nkuXf0oS69bG2uXKs29F5FWUY8e/hVAovXzS1WwHMxNHHgL0yhvvuAEMhZM6awkNaO59yLSKmo+06aYmc0Dzgb+nAQXa5WqYHn2r78pVuDt6+1mxtQuurJZBobjD/oW1Gp7RBGRJCXdw/8ycBUJ74G7YM503r9k/KyY7219MXYufrKpmb7e7glLIYuINFJiAd/MlgMvufvGCc67xMw2mNmGnTt3Vv14F5x8xMjq2oKxg6eFOvVRbwKTSc2Uu66ISLNIMqWzFDjXzM4CpgAHmdnt7n5h8UnuvgpYBcHCq2ofbN6sqeTGLCIr7qHHmStfTWpGc/BFpFUk1sN398+4+zx3P4Jg96y1Y4N9LZXroVcyV76S1Izm4ItIK0l00LbeSvXQk6pTr/r3ItJK6hLw3f0h4KF6PFaUpObKaw6+iLSStllpC0E+fel1a7nw1vUjC6cKRc2uXr6w5nPlNQdfRFpJVeWRkzKZaplRG5p3ZCCbydCVDfekPXshxx46o+Zz5VUpU0QapZJqmW3Tw4+qaTOcD+rjFwZUr/3O1kSCsubgi0graJuAP9GWhKCiZiKSbm0T8Mfm07s7jM7s6GqZGlAVkTRr62mZ6/pf5qoxi6KUdhGRtGqrgA+Mmnu/dMHBrFuxTAOqIiK0YcBXqQMRkWhtk8MHlToQESmnrQK+thsUESmtrQK+Sh2IiJTWVgG/lUsdqKa+iCStLQZti0sbtOJ2gxpoFpF6aPmAXypYtkKgh9EDzYUyy1fdtYWlCw5umd9BRFpDS6d02mFWjgaaRaReWjrgRwXLrFlLBUsNNItIvbR0wI8KlvsHczz2/J4GtahyrTzQLCKtpeXr4a9+5Fk+e/djo45N6cywbsUygJYZvFVNfRGpRiX18Ft+0PbYQ2fQ251l30Bu5FhnJsPq9c/xVw/1t8zMl77ebgV6EUlUS6d0IEjrDOdHf0oZzOW4+cH+lh7MFRGptZYP+FE58MvOOJqurGa+iIgUa/mUDoy5KW0PAAAJ7klEQVSvgw9w80P9o87RzBcRSbuW7+EXFO8rq5kvIiLjtUUPP0orllgQEUlS2wZ80MwXEZFibZPSERGR8toy4KvUsIjIeG2X0lGpYRGRaG3Vw2+H6pkiIklpq4CvUsMiIqUlFvDNbIqZ/cTMNpvZ42a2MqnHKlCpYRGR0pLs4Q8Ay9z9eGAx8C4zOyXBx9OCKxGRMhIbtPWg7vK+8MfO8F/itZi14EpEJFqis3TMLAtsBBYAN7v7+iQfr0ALrkRExkt00Nbdc+6+GJgH/KaZHTv2HDO7xMw2mNmGnTt3JtkcEZFUq8ssHXffDTwEvCvitlXuvsTdl8yePbsezRERSaUkZ+nMNrOZ4fdTgd8Cnkzq8UREpLwkc/hvAv4xzONngDvc/b4EH09ERMpIcpbOFuCEpK4vIiKVaauVtiIiUpoCvohISijgi4ikhAK+iEhKKOCLiKREWwZ87XglIjKedrwSEUmJturha8crEZHS2irga8crEZHS2irga8crEZHS2irga8crEZHS2m7QVjteiYhEa7uAD9rxSkQkSluldEREpDQFfBGRlFDAFxFJCQV8EZGUUMAXEUkJc/dGt2GEme0Enq3wbgcDLyfQnFai50DPAeg5gHQ+B4e7++w4JzZVwK+GmW1w9yWNbkcj6TnQcwB6DkDPwUSU0hERSQkFfBGRlGiHgL+q0Q1oAnoO9ByAngPQc1BWy+fwRUQknnbo4YuISAwtG/DN7F1m9pSZ9ZvZpxvdnskys8PM7EEze8LMHjezK8LjbzCz75vZz8Kvs8LjZmY3hr//FjM7sehaF4fn/8zMLi46fpKZ/TS8z41mZvX/TSdmZlkz+w8zuy/8+UgzWx/+Pt8ws67weHf4c394+xFF1/hMePwpM/udouNN/7oxs5lmtsbMngxfD6em7XVgZh8P/w4eM7N/MrMpaXsdJMLdW+4fkAV+DhwFdAGbgYWNbtckf6c3ASeG308H/hNYCHwB+HR4/NPAdeH3ZwH/AhhwCrA+PP4G4Onw66zw+1nhbT8BTg3v8y/Auxv9e5d4Lj4BfB24L/z5DuAD4fe3AH8Sfv/fgVvC7z8AfCP8fmH4mugGjgxfK9lWed0A/wh8JPy+C5iZptcBcCjwDDC16P//D9L2OkjiX6v28H8T6Hf3p919EPhn4LwGt2lS3P0Fd380/H4v8ATBC/88ggBA+PU94ffnAbd54BFgppm9Cfgd4Pvu/kt3fwX4PvCu8LaD3P3HHvw13FZ0raZhZvOAs4Fbw58NWAasCU8Z+xwUnps1wJnh+ecB/+zuA+7+DNBP8Jpp+teNmR0EnA78LYC7D7r7blL2OiAo3T7VzDqAacALpOh1kJRWDfiHAtuKft4eHmsL4UfSE4D1wBx3fwGCNwXgkPC0Us9BuePbI443my8DVwGFvSr7gN3uPhz+XNzukd81vH1PeH6lz00zOQrYCfx9mNa61cx6SNHrwN13ADcAzxEE+j3ARtL1OkhEqwb8qJxjW0w3MrNe4C7gT939V+VOjTjmVRxvGma2HHjJ3TcWH4441Se4rWWfA4Ke7YnAX7v7CcB+ghROKW33HITjE+cRpGHmAj3AuyNObefXQSJaNeBvBw4r+nke8HyD2lIzZtZJEOxXu/s3w8Mvhh/DCb++FB4v9RyUOz4v4ngzWQqca2a/IPiYvYygxz8z/GgPo9s98ruGt88Afknlz00z2Q5sd/f14c9rCN4A0vQ6+C3gGXff6e5DwDeBt5Ku10EiWjXg/ztwdDhq30UwUHNvg9s0KWHO8W+BJ9z9S0U33QsUZlhcDNxTdPyicJbGKcCe8KP+94B3mtmssKf0TuB74W17zeyU8LEuKrpWU3D3z7j7PHc/guD/dK27XwA8CJwfnjb2OSg8N+eH53t4/APh7I0jgaMJBiqb/nXj7v8FbDOzN4eHzgS2kqLXAUEq5xQzmxa2sfAcpOZ1kJhGjxpX+49gdsJ/Eoy2f7bR7anB73MawcfKLcCm8N9ZBLnIB4CfhV/fEJ5vwM3h7/9TYEnRtT5MMEDVD/xh0fElwGPhfW4iXHjXjP+Ad/D6LJ2jCP5Q+4E7ge7w+JTw5/7w9qOK7v/Z8Pd8iqJZKK3wugEWAxvC18LdBLNsUvU6AFYCT4bt/BrBTJtUvQ6S+KeVtiIiKdGqKR0REamQAr6ISEoo4IuIpIQCvohISijgi4ikhAK+NB0zm2NmXzezp81so5n92Mx+N7ztHWa2Jyw78JSZ/TBcoVu475+Z2Q4z2xRWWjw34vp/YGY7w3O2mtkfTbK9bzSzfzazn4fXu9/MjqnyWrea2cLw+/8xmXaJjKWAL00lXGhzN/BDdz/K3U8iWBhTvDr0R+5+gru/GbgcuMnMziy6/S/dfTHwPuDvzCzqdf6N8Jx3AP/bzObEbF/HmJ8N+BbwkLv/mrsvBP4HEOt6Y7n7R9x9a/ijAr7UlAK+NJtlwKC731I44O7PuvtXo052903A54HLIm57AhgGDi71YO7+EsHim8PNrMfM/s7M/j38BHEejHwiuNPMvg3865hLnAEMjWnvJnf/kZn1mtkDZvaoBfXnC9c7woJa9/9oQQ37NWY2LbztITNbYmZ/QVAtcpOZrQ5vuzv8xPO4mV0y4TMpMoYCvjSbRcCjFd7nUeAtYw+a2ckEVTd3lrqjmR1FsIKzn2BV5lp3/w2CQH59WKkSgvrxF7v7sjGXOJagkmOU14DfdfcTw+t9MfxEAPBmYJW7Hwf8iqCm+wh3/zRwwN0Xe1BeAuDD4SeeJcDlZtZX6vcSidIx8SkijWNmNxOUnRgMA3HkaWN+/riZXQjsBX7Po5eT/56ZnQYMAH/s7r80s3cSFG/7VHjOFGB++P333f2XlTafIF10OsEbz6G8nurZ5u7rwu9vJ0hN3TDB9S4vjGUQFP86GthVYZskxRTwpdk8Dry38IO7X2pmBxPUlinlBIINYwr+0t0nCp7fcPexaSAD3uvuT406GHxS2F+mveeXuO0CYDZwkrsPWVAFdEp429g3obI1TszsHQRVJE9191fN7KGia4nEopSONJu1wBQz+5OiY9NKnWxmxwFXExQQm6zvAR8rpF3M7IQY91kLdBfP9DGz3zCztxOU6X0pDPZnAIcX3W++mZ0afv9B4OGIaw9ZUDKb8FqvhMH+LQTbGYpURAFfmkqYfnkP8HYze8bMfkKwfd2KotPeVpiWSRDoL3f3B2rw8NcCncAWM3ss/DlOe38X+O1wWubjwJ8R1FdfDSwxsw0Evf0ni+76BHCxmW0h2Hf2ryMuvypsy2rgu0BHeP61wCPV/YqSZqqWKVJnFmxheZ+7H9vgpkjKqIcvIpIS6uGLiKSEevgiIimhgC8ikhIK+CIiKaGALyKSEgr4IiIpoYAvIpIS/x+YaJyzp4f/WwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "gdp_happiness.plot.scatter(x=\"GDP Per Capita\", y=\"Life Satisfaction\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Implements the closed form\n",
    "def theta_best(X,y): \n",
    "    X_t = X.T\n",
    "    return np.linalg.inv(X_t.dot(X)).dot(X_t).dot(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# .values to get numpy array out of pandas series\n",
    "# np.c_ ~= zip\n",
    "\n",
    "# Note: needs to add 1 to a feature for constant to show up in theta\n",
    "\n",
    "X_gdp = np.c_[\n",
    "    np.ones(gdp_happiness[\"GDP Per Capita\"].values.size),\n",
    "    gdp_happiness[\"GDP Per Capita\"].values\n",
    "]\n",
    "\n",
    "theta_gdp_happiness = theta_best(X_gdp, gdp_happiness[\"Life Satisfaction\"].values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([4.66027944e+00, 4.31820655e-05])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "theta_gdp_happiness"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x11de33f98>]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XeYVOX1wPHv2dlCWZorEukqlgAK6hpQjAVSjChoJGpsJEZJfrE30Bhi7EGxYyNojIImFAuiiQ2NSpAEFFZAVKJSLYD0smX2/P64d5bZ2ZnZO7tzp57P8+zD7J07d94ZZs+8933PPa+oKsYYY3JfQbobYIwxJjUs4BtjTJ6wgG+MMXnCAr4xxuQJC/jGGJMnLOAbY0yesIBvjDF5wgK+McbkCQv4xhiTJwrT3YBwe+65p/bs2TPdzTDGmKyxYMGC9ara0cu+GRXwe/bsyfz589PdDGOMyRoissLrvjakY4wxecICvjHG5AkL+MYYkycs4BtjTJ6wgG+MMXnCAr4xxuQJC/jGGJMnLOAbY7JfbS08+yw8/HC6W5LRLOAbY7JXTQ08/TQcfDCcdhpMmuQEfxOVrwFfRK4QkSUislhEnhGRFn4+nzEmT1RVweOPw3e/C2efDSJO4J83DwqsHxuLb++MiHQBLgXKVbUvEADO9Ov5jDF5YNcuePBB2H9/+NWvoG1bZyinogJ+/nMozKhqMRnH73enEGgpItVAK2Ctz89njMlF27bBo4/C+PHw1Vdw1FHwyCNwwglO79544lvAV9U1IjIeWAnsBF5V1Vcj9xORUcAogO7du/vVHGNMNtq8GSZMgHvugQ0bYMgQeOYZOPZYC/RN4OeQTgdgOLAP0BloLSLnRO6nqhNVtVxVyzt29FTh0xiT69avh7FjoUcP+P3vYeBA+Pe/4fXX4bjjLNg3kZ+zGz8APlfVdapaDTwLHOXj8xljst1XX8E110DPnnDLLfCDH8D778OsWXDkkeluXdbzcwx/JTBQRFrhDOkMAazYvTGmoZUr4Y47nLTK6mpnAva666BPn3S3LKf4OYY/T0SmA+8DNcAHwES/ns8Yk4WWL4c//Qn++ldnmGbkSBgzBnr1SnfLcpKvWTqqegNwg5/PYUy4DdsqWb1xJ107tKSstCTdzTGxLF0Kt93mTMAWFcFvfuMM5Vjihq8sadXkjBcWrmHMjAqKCgqorq3ljtMOYVj/Lululgn3wQdw660wYwa0bg1XXun87L13uluWF+ySNJMTNmyrZMyMCnZV17K1soZd1bWMnlHBhm2V6W6aAZg7F4YOhcMOg9deczJvvvgC7rzTgn0KWQ/f5ITVG3dSVFDALnbXUSkqKGD1xp02tJMuqvDWW062zezZUFbm9O4vugjatUt36/KSBXyTE7p2aEl1RNGs6tpaunZomaYW5TFV+Oc/nUD/73/Dd74Dd90Fo0ZBaWm6W5fXbEjH5ISy0hLuOO0QWhQV0KakkBZFBdxx2iHWu0+l2lp47jkoL4cTT4TVq526N59/7ozTW7BPO+vhm5wxrH8XBvXa07J0Uq2mBqZOdbJulixxUiofewzOOQeKi9PdOhPGAr7JKWWlJUkN9JbmGUdVFUyeDLff7uTT9+kDU6bA6adb1coMZf8rxsSQT2meCX2x7drl1KIfN865Qvaww5wSxcOHWy36DGcB35gowtM8Q5k/o2dUMKjXnjnX0/f8xbZ9u1Oi+M47nZo3Rx5pJYqzjH0dGxNFKM0zXCjNM5d4un5h82YnnbJHD7jqKujd20mznDMHfvITC/ZZxHr4xkTRtUNLqoLBetvSmebp11xC3OsXdm2F++6DBx5wgv6JJ8L11zuLj5isZAHfmCjeXb6eWt39e2EBdWmeqZ7I9XMuIdr1C+22rOeA8TfBYxOdYZzTToPf/c4ZqzdZzQK+MRFCwxzVwd0RP1BQwKBee6Z8ItfvuYTQ9QujZ1TQbet6Rs6Zxs8rXiVQYyWKc5EFfJO3YvXUow1zFAcKWLJ2S8onclNRMmJYm138aNU0ip+ejKgiI0fCtddaieIcZAHf5KV4PfVYZRpAU16vx9eSEWElilsUFTmlD0aPthLFOcyydEzeaSwzJVaZhj6d26W8Xo8vJSM++ABGjIC+feH5552yB59/7iwWbsE+p1kP36SUlwlPvydFvQyTxCrTEBrvDj8z8HviNmklI+bOddIrX3oJ2rZ1Mm4uuwz23DO5DTYZywK+SRkvE56pmBSNNUzSujjAolWb6oJqKLCGcu/LSkvSVq+nySUjVOFf/3IqV77xhlOi+JZbnBLF7dsnv6Emo4mqNr5XipSXl+v8+bbOeS7asK2SQeNms6t6d6BtUVTAnDGD6wKZl32SZebCNXU99apgLUMO2os3ln1DcWD3F41C9pZWUIVXXnGC+5w5Toniq6+GX//aqlbmGBFZoKrlXva1Hn4GyeVCXV6GUVK5iEmopz5l3komzP6Elxd/BUBljfPc10xfBAiVNVlWWqG2FmbOdAL9ggXQrZszNn/++dDS1gbIdxbwM0SuF+rykm2SjkVMHnprOVXBhtsDUgARFQMyegWtYNApUXzrrU6J4v32sxLFpgHL0skA+bAeq5dsk1QvYhKtXk5IUGsJ1tYf7szIFbSqq+Evf4HvfhfOOssZypkyBZYtc3r1FuxNGOvhZ4B8WY/Vy4RnKidFo51RAJQUFnDniEMAUp6R49muXU6gHzcOVqyAQw+FGTPglFOsRLGJyQJ+Bsin9Vi9ZJskexGTeM8TnmZZFQxy8fH7c9aA7nXPn3EraG3fDhMnOiWKv/zSKVH80ENWtdJ4YgE/A0QGnozrTWa45kx2N3ZGkaovn0Zt3uysD3vPPbB+PQwe7AzdHHecBXrjmQX8DGHrsTZNMia7Q0F9w7bKenn4GWHDBqdE8f33p7VEcS5nkKVbKt9bC/gZJGN6k1kimZUkMy5L6quv4O67neGa7dvhpz91An0aShS/sHANo6cvIiAFBLWWO0f0y6kMsnRK9efOt9kdETlQRBaG/WwRkcv9ej6Tf5K1KlVGZUmtWgWXXgr77AN33eWsE7t4sTMhm4Zgv2FbJVdNXUhljbKjOkhljXLl1IU5lUGWLun43PnWw1fVj4H+ACISANYAz/n1fCb/JGuyO51ZUqHT+e4b19Jhwr3wxBNOamWGlChesnYLNRGJTDW1zvZjDuiYnkbliHR87lKVvzUE+J+qrkjR85k8kKy8/XRlSb2wcA3nXvMEK4eeRtv+BxP865NOieLly2HSpLQHe0es0iuZU5IlW6Xjc5eqMfwzgWdS9FwmjyQy2R1rciwdWVKb/v0fSi64ilkfzWFXUTGPlQ/nqaNO4/lbRmTUPE6fzu0oCki91b+KAkKfzu3S2KrckI7Pne/F00SkGFgL9FHVr6PcPwoYBdC9e/fDV6ywkwCTfF4mx1KSLfHee075g1mz2FrSiicOO5nHy4exsVU72pQUMvmCAfTrlllVLGcuXMM10ysIFAjBWuXOEblV9iPdmvu5S6R4WioC/nDgIlX9UWP7WrVM44dUVuGMShXeftspaPb661BWxo6LLuHYHb1ZF2iVnjYlyNIyM1ciAT8VY/g/x4ZzTBolK5snYarwz3/C97/vXCD14Ycwfjx88QWtbryBsWcflbK6Qc1VVlpCv27tM7Z9xhtfx/BFpBXwQ+DXfj6PMfGkfHLMY4liu9jOpJqvPXxV3aGqZaq62c/nMZkvdBVrOvK3U1aFMxiEv/0N+vWDU0+FTZucbJvly50VpqLUo7ees0klu9LW+C4TrmL1tTddXe3UtbntNvj0U+jdGyZPhjPOgEL7EzOZwz6NxlfJLH/QXEkvXWElik2WsU+l8VXaJkz9tH27U7Vy333ht7+FvfeGl15yxut/+lML9iZjWQ/f+Ko5E6YZlwq4ZYtTovjuu50Sxccf7wzdHH+8lSg2WcECvvFVU68mzIRx/zobNjjlie+/35mI/clPnMqVgwalpz3GNJEFfOO7RCdMGxv3T1nPP1Si+OGHYds2Z7jmd7+Dww/37zmN8ZEFfJMSiUyYxqsi+O7y9f73/FetcpYQ/POfoaoKzjwTrrsO+vZN7vMYk2I2u5QH0pkD3xSxxv1bFwf8rR/+2WdOtcr99nN69WedBcuWOSmXFuxNDrAefo7LqLFwj2KN+2+vCvpTP/yjj+D22+Hpp528+QsvhNGjoUePJLwaYzKHBfwclkk58ImKNu6/YVtlckskLFzoXCw1fbpzFezll8NVVzlplsbkIBvSyWHZngMfWXYgaSUS5s2Dk092LpR65RVnInbFCqewmQV7k8Osh+/KuJzvJEjXSk5+anKJhMgSxXvsATffDBdfDO0zq/68MX6xgE92jnN7kY4VdVIhoRIJqvDqq06gf/dd6NTJycD5zW+gtNTfhhqTYfI+4Ecb575m+iLatyqiT+d2GREcm3P2kbcleGtr4cUXnUA/f37MEsXG5BNPAV9EAkCn8P1VdaVfjUqlaDnflTXKbya/T61q2nv7yTj7SHrRsEwWDDqTsLfe6iw4su++Tonic8+F4uJ0t85kgVwc3g1pNOCLyCXADcDXUBcVFTjEx3alTLRxboAdVUEArpleQftWxfTp3Dbl//nZlGWT9j+S6monrfK22+CTT+C734WnnnIumrISxcajXB3eDfGSpXMZcKCq9lHVg92fnAj2UD/zo1VRoMH9lTW1jHpyPkfe/gZT3kveAuteLoZKNMsm3jH9vPjqhYVrGDRuNudMmsegcbOZuXBN0p8jpspKeOQROOAA+MUvoFUrp4e/eDGcc44Fe+NZeAfLlwv7MoCXv4ZVQE6vWBUa516ydgsXPjmfypr6Pf5d7u/XP78YBM4e0LwLcrz2IhLJsol3TD97LWk7C9mxAyZOdCZg166FAQOcMfoTT0xZ5cq0n9WYpIpX0iNX/n+99PA/A94SketE5MrQj98NS7Wy0hKOOaAjd45we/vFDXv7ADe+uLRZ3/iJ9CK85p3HO6bfvZYlazdTEBFgfc3137IF/vQn6NkTrrjC6dm//jrMnQtDh6Ys2Kf1rMb4IhfTmCN56eGvdH+K3Z+ctru3v5lfPfFfqiOG94sC0qxv/ER7EV6ybOIdM3Tbj17LCwvXMHr6IiprtN52X+rdf/utU574vvvSXqI4m+ZWjHe5msYcrtGAr6o3AohIG+dX3eZ7q9LM6e3vxR+H9XWGccIEa7VZ3/hN6UU0lmXT2DGb22uJFphDQS8y2JcUerv61fMw09dfOyWKH3rIKVF86qlOoG9GieLmDsXkw6l/vsr1NGYvWTp9gaeAPdzf1wPnqeoSn9uWdmcP7AHiDOMUBYRgrTb7G9+PXkRZaQljT+ods53Neb5YgXnJ2i0UUH/4pFVRgEfOPZxjDugY95jRr32IyIZavdoZn5840SlRfMYZTgmEZlatTMZ8Rj6c+uezXE5jFlWNv4PIv4HrVfVN9/fjgNtU9ahkN6a8vFznz5+f7MM2W7weYaK9xdD+rYsDbK8KJqUXEQpiARGqg7XccHIf58uqGe0MPWbQuNnsChvXalFUwNihvblp1tIGk9stigqYM2Zwo4uULFq1iXMmzWNrZU297a2KAnTeuJZJX82m58ypzlWy554L117rjNU3U6zXE2pzImYuXNPgSzSX0vdM9hCRBapa7mVfL2P4rUPBHkBV3xKR1k1uXRaK9Y2faG8x2v79ujWvjkt4bznk5peWckLf79Rrc1N6LdGGLgIi3DhrKVURwb6kUOrOHBp7X6L1kPfbsIrfvjeN4UveoraggF2/PJ8W11/nTM4mSTKHYnL91N/kJk9ZOiIyVkR6uj+/Bz73u2GZLtHsF7+yZfysiBl16CJYS3EgYiinOMCfzytnWP8unl5nePbRod+uYMIL43ht0m/5ycdz+Ev5ME649K98/IdxSQ32MV9PM4ZiIqt55oJsWyzHJMZLwD8f6Ag8Czzn3v6ln43KBokGWr8Cc6JBLNE/6IuO60VJ4e600BtO7kNNbf1hwFpV+nRuB3h/ncOq1lCx4EGe+/NFHPfZfB468mcc/ZvHuXXwBaxt3cGX8fCklVfOUZZqmvu8ZOlsBC5NQVuySqKBNtm9y/Axcq+TsokMQYXvC8qoY/blrAHdKSstoU2LwpjPF+11VgVr2byzmg3bKil7f55T0Oy11yjeYw+46SbeOfanTHhtJUUFBbTwORXOhmKis1TT/BBz0lZE7lXVy0XkRZzaOfWo6rBGDy7SHpgE9HWPcb6qzo21f6onbZubnpfoxF2yJvqiBe7GglgiE5Ze9o333oW/zl01QbS2lsGrKrjw7ac5YtUS2GsvuPpqp0RxmzaNHs/4L9pEepuSQiZfMKDZ80zGX8matH3K/Xd8M9pyH/BPVR0hIsVAq2YcK6mSkZ6XaG8xGb3LWD2xOWMGx/3DTGTC0su+8SaB6y5eW72RZ/7wEL9+92/0//JT1rbZk5t//H/89qnbKOtYv625nAqXDSzVND/EDPiqusC92V9V7wu/T0QuA/4V78Ai0hY4BviFe7wqoKo5jU2WZJ6+JhqomhvYmppp0tgfdHgPu9l//MEgZS89zxF/vIljli1lRfvvMOaES3i272BatGrJsF1Q5vH1mtTIh6tMjbe0zJE4PfVwv4iyLdK+wDrgLyLSD1gAXKaq2xNtZLJl+pWS8YY3mhqM4/1BRzvb8frHX6+tJQX1ShQXHXAg1wy/mmcP+D7BAqc2UcB6jRnL5jdyX8yALyI/B84C9hGRmWF3tQE2eDz2YcAlqjpPRO4DrgXGRjzPKGAUQPfu3RNrfRNl8ulrY0NNzemJRfuDjjdENGfM4Lh//KG2tqqt4eQPXmX0ohdovXYV9OsH06ZR+NOf8v2KL3lxRgWtfOg12rh/8tnQWm6LN2nbA9gHuB0nUIdsBSpUtSbqA3c//jvAe6ra0/39+8C1qjo01mNSOWnblAlUvwNMohOrzZ0LWL1xJ5t3VnHRlA8SnqzbsK2SIbe8zKnz/8Gv583gO9u+ZWGXA9n3nttpO+KUelUrm3o1crz9c32hCmO8SsqkraquAFaIyNnAWlXd5R68JdAV+CLegVX1KxFZJSIHqurHwBBgqcfX4LtET19TEWASGWpqTk8s/LVUBYNEpNU3frazZQtVt93FGw/cR9mOzcztfjBXDr2SDw84nMkDB9IvokRxIm318j5bCqExTeNlDH8qEF43JwhMA47w8NhLgCluhs5npPGCrWi9xlAgCl2MFC+l0UuAaW6vOxVDTdFeS2GBU+WyONDIsEtYieK9N23i7f0O5/6BpzO/ax8AWmjzKol6fZ8zfQ7GmEzlJeAXuhk2gJNt4wbwRqnqQsDTqYafmrsalJcAk6zFxv3OlIj2WloWFfLg2YfRrmVR9C+rb75xShQ/+KBToviUU+D669lUuDeLZ1TQJklt9RrIM3kOxphM5iXgrxORYao6E0BEhgPr/W1W8sTrNQKeepReUhqTNcTgd6ZErNcSdZH21ath/HinRHFlJZx+ulOi+OCDnbZCUtvqNZBbCqExTeMl4P8GZ1hmAiA4a9ye52urkigZq0E1FmCSPcTgZ6aEp2D52Wcwbhw88QTU1sYtUZzMtiYSyC2F0JjEeaml8z9goIiU4mT1bPW/WcmTrNWgwgNMqJb9hm2VlJWWxH2OTEwdjBksly2D22+HKVMgEIBf/QpGj66rWpmK15JIILcUQmMS46WHj4gMBfoALcTNwFDVm3xsV9I01mtMZGigrLSEd5evjzpWH+04sfbNBPWCZUUF3HorTJsGLVrApZc6tW46d67bP5VpkBbIjfGHlxWvHsGpgXM8TiG0EcB/VPVXyW6Mn3n4TVm1KnJ7Y3ny4fsDSVtdyTf/+Y8T6GfOdIqYXXwxXHEFdKy/RGEyV4oyxiRXsle8OkpVDxGRClW9UUTuwqmNn1XipWBG61FG69H2KGsdd6w+/DiLVm1Keupg0oZU3n7bCfSvvgpuiWIuvhg6dIi6u6VBGpMbvAT8Xe6/O0SkM05ZhX38a5J/vA5LRMu6uXraIp6+YIDnMX+vGSdezy6aPaSiCq+95tSif+cdp0TxHXfUlSjesK2S1TGuRbA0SGNyg5eA/6Jb1/5O4H2cuvZ/9rVVPkgkdTJaj7YqqJz12H8484iuTJ2/utExfy8ZJ7GCeOT2sSf15uZZS5uW9qkKL77oBPr//he6dnUunrrgAmjpBOwp763gxllLKQ4INbWa1Po9xpjMEa942s9UdRowWVU3ATNEZBbQQlU3p6yFSZLIsES0Hi1AVU0tU+evZtbFR7O9Ktjo0Eq8jJNYX0C9927bYPsfZy5psI5so0MqwSDMmOEM3VRUwD77OPn0550HJbsfM+W9FVz//GL39VHXjsgvE0uDNCb7xVvT9jr33xmhDapamY3BHhIblgj1aCODLDiBdntV0PPi1WWl0Re6jrX260J37L9eO4PK9iqPQyrV1fDkk9CnD5xxBlRVwVNPwSefwIUX1gv2G7ZVcuOLSxocIlAgUdfajfVajDHZIV7A3yAib+KWR478SVUDkyUUxL0uYD2sfxdevvT7FBdGBN8kjV3H+gLq36191LOLkNYlgehtr6x0evAHHggjRzrpldOmweLFcM45UNjwZG71xp0UBRp+BKqDzauJY4zJTPHG8Ifi1LN/CrgrNc3xV7yLp6Lp1akN40f4M3Yda1y8V6c23HHaIVw9vYKqmvqBv3VxgBtP7sPxB+21uw07dsCkSc4E7Jo1MGCAM0Y/dGi9EsXRdO3QkmCUtNwbTu5tvXhjcpCXPPyOqrrOvV0AlKrqFj8ak6p6+IlmvPh5hWmsYy//eisn3v8OVcHd/z/1ct+3boWHHoK77oJ16+DYY+H3v4chQxoN9OFC6wIERKgO1nLDyX04e2CPpL5GY4x/EsnD9xLwn8appxPEWaawHXC3qt7Z3IZGSkXAz6aLiKIu0tKjVV2JYjZuhBNOgOuvh6OPbvLzZGL5B2OMN8m+8Kq3qm5xF0J5GRiDE/iTHvBTIZsuIgofgupWs5U9Jk5wShRv3VpXopjy5leftlIGxuQHLwG/SESKgFOACapaLSLxTwsyWLZdRFS2eT1ld4+HRx+FXbuczJuwEsUmN9lZl/FDvCydkEdxljNsDbztrnXryxh+KsTK1tm4vYrp81ex/OvkFQMNlXHYsK0y8Qd//rlzFey++8IDDziB/qOP4JlnmhTsm9UWk1IvLFzDoHGzOWfSPAaNm83MhWvS3SSTIxodw4/6IJHCxhYxb4pULmIe3oO67/VPePK9lXX3nXdkd24afnDUfZO5NmtUH3/slCiePNkpUXz++TBmTF2J4qawBb+zRzbNMZnMkJQxfBE5R1Uni8iVMXa5u0mtyxChcevlX2+tF+wBnpy7kvMG9qRXpzZNCpZNWgGrogJuuw2mTt1doviqq6BL8wKzLfidXbJpjslkn3hj+K3df9tEuS9rx/AjLVy1Keb2Dq2LmxQsY/3RLlm7mXYti+vmC1Zv3EmPz5bQ/p474YUXnBLF114Ll1/uFDdLAgsg2SXb5phMdokZ8FX1Uffm66o6J/w+ERnka6tSqH+39jG3NzVYRvuj3VUT5MIn51McCLCrJsjhKz/k4rnTaP+/BVS1bUfxjTfCJZfUlShO1qSdBZDsYoXqjJ+8ZOk8gHPFbWPbslKvTm0478juPDm3/hh+r05OyeCmBMvIP9qqYC3B2loqa5QjPn2fS+b+nQGrFrOuVXtuP+4XTD/iJF698qS6P+pkjrlbAMk+VqjO+CXmpK2IHAkcBVwO3BN2V1vgVFXtl+zGpHLSNtLyr7eycNUm+ndrT4fWxXV/bHOWr2948ZPH4BvqpW/eUcXUGx7mgrefpv+Xn/BlaRmPDBzB3w/5IbuKWtCmpJDJFwygX7f2zhW2D7xbr6xCMibtLM3PmNyUrAuvioFSd5/wcfwtOMsc5pRendrEnKSdM2Zwk4JlWctCyv7xGjU338IxH1awsl0nrv3xxTzbdwhVhUV1+4XOGl5YuIZrpi2qV04BkjPmHuviKvsiMCZ/xBvD/xfwLxF5QlVXpLBNaRMro2XOmMH0izHWH/U4m7az/Ymn6PzwvRR+8jGFBx3Ev2+4m3N37EewIFBv3+IAddcBRAv20HAYKVlB2tI1jckvXsbwd4jInUAfoEVoo6oO9q1VPooXLJud0VJZycJb7qVswr103/QVy/bah8/H3sf3rrqA1luqaPHn99heFYx4kDB/xbdOdcwowb64sH4p5GQFaUvXNCb/eAn4U4C/AyfhFFEbCazzs1F+aSxYNjmjxS1RHBx3B/3XrmHh3gfwxyEX8sZ+34MqoeTOt/jDSX2iliKuCmq9CeNwxQHh5UuOplcnZ0QtmUHa0jWNyT9eSiuUqepjQLWq/ktVzwcGejm4iHwhIh+KyEIRSc9srCs8WG6trGFXdS2jZ1TUKzWQ6CIpbN3q1KHfZx+47DJ2du/Jhefcxinn3sUbvQbUlSmurFFufmkpY0/qHXUVrWiKA8L4n/WrC/YQe5WsaKtTNcbSNY3JP156+NXuv1+KyFBgLdA1gec4XlXXJ9yyJFu9cSdaW7+HrbXaoEfrKSVu40anvs299zq3f/xjuP56Kg/9Hu+Mmw3VDVesKioooG/ndrx86fcbZOFEKi4sqNezD0lmkLZ0TWPyj5eAf4uItAOuwsm/bwtc4WurfNC6OEBlxBh5ZVBpXRxosG/McsHffAP33LO7RPHw4U6J4iOOcB6HMwF7zfQKKmuiB+ay0pIGq2idXt6VqfNXN1j5Klq7khmkLd/bmPzSaMBX1Vnuzc3A8QkeX4FX3XLKj6rqxAQfnzTbq4K0KCpoUJSq4SRqFGvWwHhvJYpDQfTpeSuZ8OanFAcCDQJztEB72ZADPAXeZAdpq4VvTP6IVzztQuAtVf1URAR4HPgpsAIYqaofeDj+IFVdKyJ7Aa+JyDJVfTvieUYBowC6d+/e1NfRqFjDHnGHQ774AsaNg8cfh2AQzj3XqXVz4IFxn6ustIRLhuzPWQO6xwzMkYE2kcBrQdoY0xTxJm0vw6mDD/Bz4BBgX+BK4H4vB1fVte6/3wDPAd+Lss9EVS1X1fKOHTu23sUoAAAUg0lEQVR6b3mC4k3INqgV/8kn8MtfQq9eTrA//3z49FP4y18aDfaRz9mvW3sLzsaYjBBvSKdGVUMTticBT6rqBuB1EbmjsQOLSGugQFW3urd/BNzU7BYnILxcQq9ObaIOh4Snau7z1f94+ItX6PLai06J4ksugauvbnaJYmOMyQTxAn6tiOwNbASGALeG3eclLaQT8JwzGkQh8LSq/rOpDU3UH57/MOqiJuHDIaFUzQNWLuPiuVP50afvsa24JTuvuIqWY65JWoliY4zJBPEC/h+A+UAAmKmqSwBE5Fjgs8YOrKqfAUkvsOZFY4uahGx8ZTaTnhnL0f9bwKYWpdx99NnMOPIUHrrkB/TbK3opBas9Y4zJVvFq6cxy169to6obw+6aD5zhe8uaId6iJr32KoU33oBbbqHXv/5F+9ZOieLJ/U9ke0krWhQVxJzITWXtGftiMcYkW9y0THfd2o0R27b72qIkiLqoiSr9Fr4NFz8A8+Y54/L33ce8I07kry9/SlFBAS3i5LWnsvaMFTUzxvjBy4VXWadXpzYc3auMd5dvoKA2yAmfzOXiuX9n/28+Z3uX7rR+9FEYORJKShgKDDy4W6O96VTVnvH6xWJnAMaYROVkwN+wrZIPPl/PqYvf5KK5U+n17Wr+t0dXrhh6Ja/2O563z/ohZSWJ5cCnqvaMly8WOwMwxjRFo8XTxHGOiPzB/b27iDTIp88YlZXsePAR/vnwKO556W6qA4X8dvi1/PBXD/Jc38FUU8Cby76pVzTNi4QLqzVRY18sXorAGWNMNF56+A8BtcBgnDz6rcAM4Agf25W4nTth0iS44w66rV7Nos4H8MfBFzglimV3hcqqoPKHFxZT+wKMHdqbvl3aeR4WSbSsQVOGXRqrl2NljY0xTeUl4A9Q1cNE5AMAVd0oIsU+t8u7XbucypV33QVff031oKOZdcnNXP3tngSJXop4h1tP5/rnF1NaEqCmVusNi8QL1F7LGjRn2CXeF4uVNTbGNJWn8sgiEsAphIaIdARi1/ZNNRG4/37o3593Th/FBZ+1pPJb783bVukUTwtNjL67fH2zx8eTkdET64vFyhobY5rKS8C/H6cOzl4icivOAua/97VViSgpgUWL2FDcmgvHzW5QljikZVGAndWxK2MWFRSwZO3mpKRe+j3sYmWNjTFNEa9a5j6q+rmqThGRBTjlFQQ4RVU/SlkLvdhjD1av2tQgyIb7xaAedGhVzG0vL4t6vzNMIkkJ1KkYdrGKmcaYRMXL0pkOICJvqOoyVX1QVSdkXLB3RQuy4f4y5wsO+k7bqAuehBYK79O5bVICdXhGT+viAMUBYezQ3hagjTFpFS/gF4jIDcABInJl5E+qGuhVeJBtFSWoB0T4bN1WaiICemih8GH9uyQ19XJY/y6MHdqb6lqluLCAm19aysyFa+I+pkGZZmOMSaJ4Y/hnAqe4+zRcby8Dhca2l6zdzIVPzqeyZveShturgox/9RNqFQoLoGVRYdTlBBMdH48swRyyYVslN7+0lKqaWqpqnG3x5gPsYipjjN/iFU/7GBgnIhWq+o8UtqnJQumUfTq3484R/Rg9o4KASN0yhqGMnJLCAh48+zD6dG4bMxPGS68+VglmSGziNpV1eowx+SvepO05qjoZ6C0i3428X1Xv9rVlCYrWQ54zZjBvLvuGP764pC7YAxQHCmjXsqhZwbSxEsyJTNzaxVTGmFSIN4bf2v23FGdIJ/yn1Od2JSRauYGrpy1i4/Yqjj9oL2pqtd7+yciYiVeCGRIrxWAXUxljUiHekM6j7r83Rt4nIpf72ahEReshVwWVEx94l/EjDvHlQqWoJZjDtm/YVkmPstbMuvhotlcF484H2MVUxphUaGq1zCuBe5PZkOaIlZJZVeMUFpszZjBzxgxm9cadtC4OsL0qyIZtlQkF1MhyC706teG8I7vz5Nz6Y/i9OrWJOrzUL8YXRIhdTGWM8VtTA370IjVpUlZawunlXesF35DQWHi/bu2bXDYhMoCPPak3fTu347IhB3DewJ71snSaMwFrF1MZY/zU1ICvje+SOhu2VTJ1/uqo94XGwpsaiKM97vrnFtO6OEBQnaJrI8q71e1vE7DGmEwVc9JWRLaKyJYoP1uBzilsY6NWb9xJQKKfdIw9ybnCNRSIw4UCcWPHjnwcOHn90WrR2wSsMSZTxQz4qtpGVdtG+Wmjqhm1UlbXDi2pDjYcw29VXEDfzu1279OEQNxYyYbIL41ULZRijDGJanTFq2xQVlrCDSf3abC9ppa62jlNDcT16uKUNCzZEO1LY1j/LswZM5jJFwxgzpjBdsWsMSYjiGrmDMeXl5fr/Pnzm/z4KfNWcOOLSxGgsqaWkoAgBeJ5cRMv9y1es5mbX1pqJRCMMRlBRBaoarmnfXMp4INzBeyJ979DVXD362pRVMCcMYPj9uYTqWXT1C8NY4xJtkQCfkaNxSfD9qogGnFlrdZq3CyZRDN4YqVPWgE0Y0wmy4kx/HCPvfsZ1REnLZVBjVoHPyRaJk6gQHhz2TeeSxVHK+8QmcFjjDHp5HvAF5GAiHwgIrP8fq7lX29l5qIvG2wvDlBXMTOaaJk42yuD3DBzCYPGzW60jj1E/9LwkvZpjDGpkooe/mVASlbJilXQrFaJm34ZKxMnVq59NJZ/b4zJdL4GfBHpCgwFJvn5PCGxCpoNPXjvRidQQ6mUN57cp8Hwj5eeuuXfG2Mynd+TtvcCo0nRilm9OrXh9PIuTJ1ffwjmlaVfeyqWVlZawvEH7cXvX1hcb7vXnroVQDPGZDLfevgichLwjaouaGS/USIyX0Tmr1u3rtnPe/aAno320OOtHducnrqlZBpjMpmfPfxBwDARORFoAbQVkcmqek74Tqo6EZgITh5+c5+0a4eWBDX2gideUieb0lO3lExjTKbzrYevqtepaldV7YmzIPrsyGDvh3g99ERSJ8tKS+jXrb3nnr2lZBpjMl3OXXgFTg+9995t69WpB/9KF1tJZGNMNkhJwFfVt4C3UvFcEHt4xa/USUvJNMZkg5y70jbW8Mryr7eyeuNOxp7UO+mpk5aSaYzJBjk3pBNteEVrnQXNSwLuEoVDe9O3S7ukZtNYSqYxJtPlXA8/2vBKZVCpqtnd47/5paW+BOVEJnqNMSbVci7gRw6vFAeEFkVW48YYY3JuSAfqD6+0Lg5w0oR3691vE6rGmHyUcz38kLLSErp2aMn2qiBjhyZ/otYYY7JNTvbwoWFq5tiTetO3c3Inao0xJpvkZA8/WmrmzbP8mag1xphskZMB3xYjMcaYhnIy4NuVr8YY01BOBvxsvvI1XulmY4xpjpyatA2vR5+NV75aiWVjjJ9yJuDHCpbZEOih/kRzqCzE6BkVDOq1Z9a8BmNMZsuJIZ1oWTnXTF+UVcMiNtFsjPFbTgT8aMGyskZ5et7KNLUocTbRbIzxW04E/K4dWlIVDDbYPuHN5VnTy8/miWZjTHbIiTH8stISLj5+f+567ZN624sDu4dEsmHyNhsnmo0x2SMnAj7AWQO6M+HN5VTW7B4Wqa6tZfGazZwxcW7WZL6UlZZYoDfG+CInhnTACZR3jqg/JDJ2aG9ufmmpLS5ujDHkUA8fGg6J2OLixhizW04FfGg4JGKZL8YY48iZIZ1oLPPFGGN2y7kefiTLfDHGGEfOB3ywzBdjjIEcH9KxypPGGLNbzvbwrfKkMcbUl5M9/GjF1Cz/3hiT73Iy4FvlSWOMaci3gC8iLUTkPyKySESWiMiNfj1XJKs8aYwxDfnZw68EBqtqP6A/cIKIDPTx+epY/r0xxjTk26Stqiqwzf21yP1Rv54vkuXfG2NMfb5m6YhIAFgA9AIeVNV5UfYZBYwC6N69e1Kf3/LvjTFmN18nbVU1qKr9ga7A90Skb5R9JqpquaqWd+zY0c/mGGNMXktJlo6qbgLeAk5IxfMZY4xpyM8snY4i0t693RL4AbDMr+czxhgTn59j+HsDf3XH8QuAqao6y8fnM8YYE4efWToVwKF+Hd8YY0xicvJKW2OMMQ1ZwDfGmDxhAd8YY/KEBXxjjMkTOR3wbQEUY4zZzRZAMcaYPJGTPXxbAMUYYxrKyYBvC6AYY0xDORnwbQEUY4xpKCcDvi2AYowxDeXspK0tgGKMMfXlbMAHWwDFGGPC5eSQjjHGmIYs4BtjTJ6wgG+MMXnCAr4xxuQJC/jGGJMnRFXT3YY6IrIOWNHEh+8JrE9ic7JNvr9+sPcg318/5Od70ENVO3rZMaMCfnOIyHxVLU93O9Il318/2HuQ768f7D1ojA3pGGNMnrCAb4wxeSKXAv7EdDcgzfL99YO9B/n++sHeg7hyZgzfGGNMfLnUwzfGGBNH1gd8ETlBRD4WkeUicm2629McItJNRN4UkY9EZImIXOZu30NEXhORT91/O7jbRUTud197hYgcFnaske7+n4rIyLDth4vIh+5j7hcRSf0rbZyIBETkAxGZ5f6+j4jMc1/P30Wk2N1e4v6+3L2/Z9gxrnO3fywiPw7bnvGfGRFpLyLTRWSZ+3k4Mp8+ByJyhfs3sFhEnhGRFvn2GfCFqmbtDxAA/gfsCxQDi4De6W5XM17P3sBh7u02wCdAb+AO4Fp3+7XAOPf2icA/AAEGAvPc7XsAn7n/dnBvd3Dv+w9wpPuYfwA/SffrjvFeXAk8Dcxyf58KnOnefgT4P/f2b4FH3NtnAn93b/d2Pw8lwD7u5ySQLZ8Z4K/ABe7tYqB9vnwOgC7A50DLsP/7X+TbZ8CPn2zv4X8PWK6qn6lqFfA3YHia29Rkqvqlqr7v3t4KfITz4R+OEwBw/z3FvT0ceFId7wHtRWRv4MfAa6r6rapuBF4DTnDva6uqc9X5i3gy7FgZQ0S6AkOBSe7vAgwGpru7RL4HofdmOjDE3X848DdVrVTVz4HlOJ+XjP/MiEhb4BjgMQBVrVLVTeTX56AQaCkihUAr4Evy6DPgl2wP+F2AVWG/r3a3ZT33tPRQYB7QSVW/BOdLAdjL3S3W64+3fXWU7ZnmXmA0EFqnsgzYpKo17u/h7a57re79m939E31vMsm+wDrgL+6w1iQRaU2efA5UdQ0wHliJE+g3AwvIr8+AL7I94Ecbd8z6tCMRKQVmAJer6pZ4u0bZpk3YnjFE5CTgG1VdEL45yq7ayH1Z+x7g9G4PAx5W1UOB7ThDOLHk1Hvgzk0MxxmG6Qy0Bn4SZddc/gz4ItsD/mqgW9jvXYG1aWpLUohIEU6wn6Kqz7qbv3ZPw3H//cbdHuv1x9veNcr2TDIIGCYiX+Ccag/G6fG3d0/voX67616re3874FsSf28yyWpgtarOc3+fjvMFkC+fgx8An6vqOlWtBp4FjiK/PgO+yPaA/19gf3f2vhhnwmZmmtvUZO6442PAR6p6d9hdM4FQhsVI4IWw7ee5WRoDgc3uqf4rwI9EpIPbW/oR8Ip731YRGeg+13lhx8oIqnqdqnZV1Z44/5+zVfVs4E1ghLtb5HsQem9GuPuru/1MN4NjH2B/nInKjP/MqOpXwCoROdDdNARYSv58DlYCA0Wkldu+0OvPm8+Ab9I9a9zcH5wMhU9wZt2vT3d7mvlajsY5tawAFro/J+KMR74BfOr+u4e7vwAPuq/9Q6A87Fjn40xSLQd+Gba9HFjsPmYC7sV3mfgDHMfuLJ19cf5YlwPTgBJ3ewv39+Xu/fuGPf5693V+TFgWSjZ8ZoD+wHz3s/A8TpZN3nwOgBuBZW4bn8LJtMmrz4AfP3alrTHG5IlsH9IxxhjjkQV8Y4zJExbwjTEmT1jAN8aYPGEB3xhj8oQFfJNxRKSTiDwtIp+JyAIRmSsip7r3HScim92SAx+LyNvu1bmhx/5RRNaIyEK30uKwKMf/hYisc/dZKiIXNrO93xGRv4nI/9zjvSwiBzTxWJNEpLd7+3fNaZcxkSzgm4ziXmjzPPC2qu6rqofjXBgTfmXoO6p6qKoeCFwKTBCRIWH336Oq/YGfAY+LSLTP+d/dfY4DbhORTh7bVxjxuwDPAW+p6n6q2hv4HeDpeJFU9QJVXer+agHfJJUFfJNpBgNVqvpIaIOqrlDVB6LtrKoLgZuAi6Pc9xFQA+wZ68lU9Ruci296iEhrEXlcRP7rnkEMh7ozgmki8iLwasQhjgeqI9q7UFXfEZFSEXlDRN4Xp/Z86Hg9xalz/1dx6tdPF5FW7n1viUi5iPwJp1rkQhGZ4t73vHvGs0RERjX6ThoTwQK+yTR9gPcTfMz7wEGRG0VkAE7FzXWxHigi++Jcwbkc56rM2ap6BE4gv9OtUglO7fiRqjo44hB9cSo5RrMLOFVVD3OPd5d7RgBwIDBRVQ8BtuDUdK+jqtcCO1W1vzqlJQDOd894yoFLRaQs1usyJprCxncxJn1E5EGckhNVbiCOulvE71eIyDnAVuAMjX45+RkicjRQCfxaVb8VkR/hFG672t2nBdDdvf2aqn6baPNxhouOwfni6cLuoZ5VqjrHvT0ZZ2hqfCPHuzQ0l4FT/Gt/YEOCbTJ5zAK+yTRLgNNCv6jqRSKyJ05dmVgOxVksJuQeVW0seP5dVSOHgQQ4TVU/rrfROVPYHqe9I2LcdzbQEThcVavFqQDawr0v8ksobo0TETkOp4rkkaq6Q0TeCjuWMZ7YkI7JNLOBFiLyf2HbWsXaWUQOAcbiFA9rrleAS0LDLiJyqIfHzAZKwjN9ROQIETkWp0zvN26wPx7oEfa47iJypHv758C7UY5dLU65bNxjbXSD/UE4SxkakxAL+CajuMMvpwDHisjnIvIfnOXrxoTt9v1QWiZOoL9UVd9IwtPfDBQBFSKy2P3dS3tPBX7opmUuAf6IU199ClAuIvNxevvLwh76ETBSRCpw1px9OMrhJ7ptmQL8Eyh0978ZeK9pL9HkM6uWaUyKibN85SxV7Zvmppg8Yz18Y4zJE9bDN8aYPGE9fGOMyRMW8I0xJk9YwDfGmDxhAd8YY/KEBXxjjMkTFvCNMSZP/D/BTenv2FnKzgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "gdp_happiness.plot.scatter(x=\"GDP Per Capita\", y=\"Life Satisfaction\")\n",
    "x_lin = np.linspace(0, 80000, 1000)\n",
    "y_lin = theta_gdp_happiness[0] + theta_gdp_happiness[1]*x_lin\n",
    "plt.plot(x_lin, y_lin, \"-r\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using Scikit-Learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None,\n",
       "         normalize=False)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# .reshape(-1,1) to make an array 2D -- even when we only have 1 feature\n",
    "# Edited: can easily obtain column vector from pandas df[[\"col\"]] rather than df[\"col\"]\n",
    "\n",
    "from sklearn.linear_model import LinearRegression\n",
    "lin_reg = LinearRegression()\n",
    "lin_reg.fit(gdp_happiness[[\"GDP Per Capita\"]], gdp_happiness[[\"Life Satisfaction\"]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([4.66027944]), array([[4.31820655e-05]]))"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lin_reg.intercept_, lin_reg.coef_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gradient Descent\n",
    "### Steps\n",
    "1. Fill $\\theta$ with random value\n",
    "2. Take a small step (learning rate) in a direction that lower $\\text{MSE}$\n",
    "   * Too small -- takes long time\n",
    "   * Too large -- overshoot to other side of curve\n",
    "   * Many stuck in local minimum\n",
    "\n",
    "Linear regression is convex, so there's only one local minimum, which is the global minimum\n",
    "\n",
    "__It is important to scale the features for gradient descent to work well__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Batch Gradient Descent\n",
    "For each parameter $\\theta_j$, take partial derivative at $j$ to find the slope for each $j$. \n",
    "\n",
    "$\\frac{\\partial}{\\partial \\theta_j}\\text{MSE}(\\theta) = 2/m\\sum_{i=1}^{m}(\\theta^Tx^i - y^i)x^i_j$\n",
    "\n",
    "Combining all $\\theta_j$ together in one vector $\\nabla_\\theta\\text{MSE}(\\theta)$ represents partial derivative of all $\\theta_j$\n",
    "\n",
    "$\\nabla_\\theta\\text{MSE}(\\theta) = \\frac{2}{m}X^T(X\\theta - y)$\n",
    "* $X$ is the entire training data set\n",
    "* $\\theta$ is the current parameter vector\n",
    "* Note that we compute the whole data set every time\n",
    "\n",
    "$\\nabla_\\theta\\text{MSE}(\\theta)$ represents a slope. We would like update the value by walking down (opposite) the slope at step size (or learning rate or eta sign) = $\\eta$\n",
    "\n",
    "$\\theta_\\text{next} = \\theta - \\eta*\\nabla_\\theta\\text{MSE}(\\theta)$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([19232.53960348]), array([18724.37121949]))"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# From the warnings, lets scale X_gdp first\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "Xscaler = StandardScaler()\n",
    "X_gdp_scaled = np.c_[\n",
    "    np.ones(gdp_happiness[\"GDP Per Capita\"].values.size),\n",
    "    Xscaler.fit_transform(gdp_happiness[[\"GDP Per Capita\"]])\n",
    "]\n",
    "\n",
    "(Xscaler.mean_, Xscaler.scale_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([5.49078023]), array([1.1127658]))"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Then scale y\n",
    "yScaler = StandardScaler()\n",
    "y_gdp_scaled = yScaler.fit_transform(gdp_happiness[[\"Life Satisfaction\"]])\n",
    "\n",
    "(yScaler.mean_, yScaler.scale_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-5.66757208e-17],\n",
       "       [ 7.26599927e-01]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Implements Batch Gradient Descent for linear regression\n",
    "\n",
    "def update_theta(theta, eta, gradient):\n",
    "    return theta - eta * gradient\n",
    "\n",
    "def find_gradient(theta, X, y):\n",
    "    m = X.size\n",
    "    return 2/m * X.T.dot( X.dot(theta) - y )\n",
    "\n",
    "def bgd(X,y,initialTheta, eta=0.1, rounds=100):\n",
    "    theta = initialTheta\n",
    "    for i in range(rounds):\n",
    "        gradient = find_gradient(theta, X, y)\n",
    "        theta = update_theta(theta, eta, gradient)\n",
    "        \n",
    "    return theta\n",
    "\n",
    "bgd(X_gdp_scaled, y_gdp_scaled, [[0],[0]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 205,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([-5.78764824e-17]), array([[0.72661923]]))"
      ]
     },
     "execution_count": 205,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Compared it with scaled linear regression\n",
    "\n",
    "from sklearn.linear_model import LinearRegression\n",
    "lin_reg_gdp_scaled = LinearRegression()\n",
    "lin_reg_gdp_scaled.fit(X_gdp_scaled[:,1].reshape(-1,1), y_gdp_scaled)\n",
    "\n",
    "lin_reg_gdp_scaled.intercept_, lin_reg_gdp_scaled.coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 206,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1a2ba37e48>]"
      ]
     },
     "execution_count": 206,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xt4lNW1x/HvIgZIxBovWG1A8IpWraCptqX1gijWikbUqrVqqy221raiYmnto3iOPVLj/VIrSrX1ihWkKLaIheqRHtAgICDQYhEkeEkLQYEAIdnnj83EXOadzOWdmbyT3+d5eCTJ5H33YLJmz9prr23OOUREpHB0y/cAREQkXArsIiIFRoFdRKTAKLCLiBQYBXYRkQKjwC4iUmAU2EVECowCu4hIgVFgFxEpMDvl46Z77rmn69+/fz5uLSISWfPmzfu3c653R4/LS2Dv378/1dXV+bi1iEhkmdmqZB6nVIyISIFRYBcRKTAK7CIiBUaBXUSkwCiwi4gUGAV2EZECo8AuIlJgFNhFRHJh5Uq4+mpYvz7rt1JgFxHJpnnz4Pzz4cAD4b774LXXsn5LBXYRkbA5B3/+MwwZAhUV/u/XXONn7cOHZ/32eWkpICJSkLZtg6efhttug0WLoLwcbr0VRo6EXXfN2TAU2EVEMvXxxzB+PNx1F9TUwOGHw6OPwgUXQPfuOR+OAruISLpqauDuu+HBB31wP/FEeOghOPVUMMvbsBTYRURStXixT7c8+SQ0NsI558Do0T6f3gkosIuIJMM5eOUVqKqCF1+E0lK4/HIYNQr23z/fo2tFgV1EJJHt22HyZB/Qq6uhd2/4r/+CK66APfbI9+jiUmAXEYln0yZ45BG44w5fpnjQQT6XftFFUFKS79ElpMAuItJSba3fSHTffbBuHXz5y3D77XDGGVBUlO/RJUWBXUQEYMUKH8AffRS2bPGB/LrrYPDgfI8sZQrsItK1zZnj8+fPPQfFxXDxxX6X6CGH5HtkaVNgF5Gup6kJpk3zu0Jfew3KymDMGPjJT2DvvfM9uowpsItI17F1Kzz+uE+5LF0K++4Ld94Jl10Gu+yS79GFRoFdRArf+vXw29/CPffABx/AwIHwxBNw7rk+/VJgFNhFpHCtXu1n5A8/DBs3wsknwx/+AEOH5nXLf7YpsItI4Vm40C+IPv20//j88+Haa/1MvQvIOLCbWU/gVaDHjus965y7MdPrioikxDl4+WUf0GfMgF69/GLoVVf5XHoXEsaMfSswxDm30cyKgdfM7M/OuTkhXFtEJLGGBnjmGd+Ua8ECX9Vyyy2+j8tuu+V7dHmRcWB3zjlg444Pi3f8cZleV0QkoY0bfe78zjt9Lv3QQ2HCBLjwQujRI9+jy6tQcuxmVgTMAw4E7nfOzY3zmJHASIB9u9jbIhEJ0Qcf+OqWBx6Aujo47ji4/3447TToptM+IaQzT51zjc65gUAf4BgzOzzOY8Y75yqccxW9e/cO47Yi0pUsWwbf+x706wfjxsFJJ/ldo6+8AqefrqDeQqhVMc65OjP7G3AqsDjMa4tIF+QczJ7tF0SnToWePf1moquvhgMPzPfoOq2MX+LMrLeZle34ewkwFFiW6XVFpAtrbPQ90L/yFfja13xwv+EGn0v/zW8U1DsQxox9H+D3O/Ls3YBnnHMvhHBdEelq6uvh97/3W/5XrPAnE913H3z3u/7EIklKGFUxbwGDQhiLiHRV//mPXwC97z7fD/2LX/QljCNGRKYHemeinacikj8rV/oTiiZM8LP1007zPdCPO66gt/xnmwK7iORedbVfEH32WT8jv/BCv+X/sMPyPbKCoMAuIrnhHPz5zz6g/+1v8JnP+GD+k59AeXm+R1dQFNhFJLu2bYOnnvJb/hcvhj59/N+//30f3CV0Cuwikh0bNsD48XD33VBTA0cc4Vvmnn9+QfZA70wU2EUkXGvW+GD+4IPwySd+h+iECXDKKVoQzREFdhEJx6JFPsXy5JM+n37uuTB6NBx1VL5H1uUosItI+pzzC6FVVX5htLQUrrgCRo2C/v3zPbouS4FdRFK3fTtMmuQD+rx5sNdecPPN8MMfwu6753t0XZ4Cu4gkb9Mm+N3v/Kaid9+Fgw/2ufSLL/YNuqRTUGAXkY599BHce69vwLVunW/OdeedcMYZapfbCSmwi0iwf/7TN+R69FFfj37mmX5B9CtfyffIJAEFdhFpb84cuPVWmDIFuneHSy7xPdAHDMj3yCQJCuwi4jU1wfPP+wXR2bP9QdDXXw9XXgmf/Wy+RycpUGAX6eq2bIHHHvMpl+XL/dFzd98Nl14KvXrle3SSBgV2ka5q3Tp/IPS998KHH8KgQX5z0bnnwk4KDVGm/3siXc2qVb6i5eGHffnisGF+QXTIEG35LxAK7CJdxfz5Pn/+zDM+gF9wgW+b+4Uv5HtkEjIFdpFC5hzMmOED+ssv+5z5T38KV10Fffvme3SSJQrsIoWooQEmTvRNuRYuhH32gXHj4PLLoaws36OTLFNgFykkn3wCDz0Ed90F770Hn/+8bwHwrW9Bjx75Hp3kiAK7ZM2U+TVUTV/O2rp6PldWwuhhA6gcpCPQsuL99+Gee3yVy4YNcPzxfvv/aadpy38XpMAuWTFlfg0/n7yI+oZGAGrq6vn55EUACu5hWrrUp1sef9x3XBwxwle4HHNMvkcmeZTxS7mZ9TWzWWa21MyWmNlPwxiYRFvV9OXNQT2mvqGRqunL8zSiAuIcvPoqDB/uUy1PPQXf+x784x/wxz8qqEsoM/btwDXOuTfNbBdgnpnNcM69HcK1JaLW1tWn9HlJQmOj791SVQVz58Kee8LYsf5gi9698z066UQyDuzOufeB93f8/RMzWwqUAwrsXdjnykqoiRPEP1dWkofRRFx9ve+ueMcdsGIFHHAA3H8/fOc7/sQikTZCXVUxs/7AIGBunK+NNLNqM6uura0N87bSCY0eNoCS4qJWnyspLmL0MHUHTNq//w033QT77utn5bvvDs8+6/u5XHGFgroECm3x1Mx6AZOAq5xzH7f9unNuPDAeoKKiwoV1X+mcYgukmVTFFEJVTVrP4Z13/Oz8kUf8bP300/2C6Ne+pi3/kpRQAruZFeOD+hPOuclhXFOir3JQedqBuBCqalJ+Dm+84fPnkyb5Jlzf/jZcc41fIBVJQRhVMQZMAJY65+7IfEgihVFVk9RzaGqCadPghBN8NctLL8F11/nzRCdMUFCXtIQxYx8MXAQsMrMFOz73C+fciyFcW7qofFXVhJn+Sfgctm71LXJvuw3eftv3bbnjDl+2uMsumTwFkVCqYl4DlPiTUO1aUkxdfUO7z/cs7sbgcTOzkncPO/0TrzJol62b+MGyl2H/kbB2re+s+NhjcN55UFyc+ZMQQTtPJc+CZshBa4T1DU3NwTLsvHui1Ek61x89bEDzC8U+H9fy3eqpfGvhX+i1rR5OOskvjp58shZEJXQK7JI3iWbIdZvbz9bjySTwthV2+qdyUDmf+cfbbP31HQxdOBPD8f4pw+n1Pzf604pEskSBXZrlurww0Qw5aINTPGHl3UPbVOUczJwJVVUMmT4ddt4ZfnwljBpF3379QhmrSCJq+ybAp7Pnmrp6HJ/OnqfMr2n1mMHjZrLfmGkMHjez1dfSkWiGHG+DU1DCIqzdrBlvqtq+3fdtqaiAoUNhwQL41a9g9WrfRldBXXJEgV2Ajkvzkgn8qQoKyN3MGDVxAT126kZp8ac/oqXdiyju1jq8h7mbtXJQObeMOILyshIMKC8r4ZYRR3T8rmXjRt8y98ADfd/zTZt8T/R334Vf/MLvGBXJIaViQhTlnZId5ZfDXliE1ouLLTU6vzG5bVXMpm2NFBcZZSXFbKhvyMq/cUqbqj78EO691/c9X78eBg+Gu+/2XRfVA13ySIE9JFHfKdlRfjkbdeWxf5ebnl/C+iQXSxsaHTv32IkFN56S9n0ztnw53H47/OEPsG0bnHmm3/L/la/kb0wiLWhaEZKo75TsKL8clDbJNL9dOaic0u6pzS/y1vr373+Hyko49FAf1C+5BJYtg+eeU1CXTkWBPSRR7z/eUX45m90aU/03ymnr36amTwP34MH+gIvrr4dVq+DBB+Hgg3M3FpEkKRUTkkLoP54ov5xKt8ZU1xpSKW3MWevfLVv8rPz22/3JRP37+wXSSy/15YsinZgCe0jiLQQWWv/xZBYW01lrCFpELS3uxoij+zBrWW3uFqTXrfMHQt97r18cPeooX8J4zjm+42KIorzYniv6N0qPAntIwug/XgjSqZ6pHFRO9ap1PDFnNS0b9TuMin67c3PlEVkc8Q7vvgt33uk7Km7aBKee6hdETzwxK1v+p8yvYfQfF9LQ5J9xTV09o/+4EIjGYnsuRL0gIZ8U2EOUSf/xQpHuWsOsZbW0PX0lzHYBMW1ngDfvu40Tn/+9PwTazNehX3stHJHdF5OxU5c0B/WYhibH2KlLuvzPUEw2Smy7CgV2CVW6aw25WHxungFu285xK99k5OuT+eqqhTTs3IviUaPgpz+FPn1Cu18i8TpXJvp8VxT1goR8UmCXUHW01hCUM83F4vMdLy7h1PkzGPn6ZA6tfZcPeu3OLSd8h1nHn8VLY88I7T4SjkIoSMgXBXYJVaK1hkQ506wuPn/8MTz0EE//ehyf++TfLN9zX6497Sr+9PnjaSgqxrZkfotU7VZaHHdT1m6l6ske0xUKErJFgV1CF7TWkChnOnvMkObHhLb4vHatL1H87W9hwwY+2P9IfjHsR/xt/4pWC6L5mAHeOPwwRj+7kIbGT/PsxUXGjcMPy/lYOisVJKRPgV1ypqOcaWiLz2+/7Y+ce/xxaGz0pYqjR7O6aB/mTl4EnWAGqKCVHBUkpEeBXXJWK5zVnKlzfldoVZU/HLqkBC6/HEaNgv33B6Byx0M7SzBV0JJsUWDv4nJZK5yVnGljI0ye7AP6G29A795w001wxRWw557tHq5gKl2BesV0cblsXpZ2v/N4Nm/27XIHDIBvftO3zX3gAd/D5YYb4gZ1ka5CM/YuLtVa4UzTNhnPmGtr4f774b774D//gWOPhVtv9a1zi4o6/n6RLkCBvYtLJe8dlLapXrUu+/1cVqyAO+6ARx7xDbqGD4frrvMdF7Ow5V8kykJJxZjZ78zsIzNbHMb1JHdSaccblLZ5Ys7qUI/Ma2XuXF/VcvDBvo/LhRf6qpepU+GrX1VQF4kjrBz7o8CpIV2rywv70OhEUsl7B6Vngnq8pK2pCV54AY4/Hr70JfjrX2HMGN+o6+GH/UEXIhIolFSMc+5VM+sfxrW6unx0tEs2751K3/S0+nls3QpPPOFr0Jcuhb59fcfFyy6DXXZJ/XoiXVTOqmLMbKSZVZtZdW1tba5uGzmd+Yi9eGmboERISrXpdXUwbhzst58P4t27+81F77wDV12loC6SopwtnjrnxgPjASoqKtq+e8+KKDbp78wd7eLtljzxkN5MmleTXm366tVw113w0EOwcSOcfDL8/vcwdKhy5yIZKNiqmKg26e/sHe3ipW0q+u2e2gvowoV+Q9HEiX7H6Pnn+x7oAwdmefQiXYM5F87keUeO/QXn3OEdPbaiosJVV1eHct8gg8fNDMwHl+dw9p7qu4a2L0jgZ8Bpb+TpLJzzi6BVVfDSS9CrF3z/+z7Vsu+++R6dSCSY2TznXEVHjwtlxm5mTwEnAHua2RrgRufchDCuna5EqYtczd7TedeQzeZQeUlNNTT404luuw3mz4e994b/+R/4wQ9gt92ye2/JiSimPAtdaDP2VOR7xt5SkRkXHNs3rXM1O/qBDhpDeVlJc5vaRNcJ8xcm5+8ENm70pYl33ulz6QMG+HTLRRdBjx7h30/yomDfYXZSOZ2xd0bxGk7F0+gcj89ZDZBScE9mNp7MQmii3ZwtFyUzfZeRs/MjP/jA90B/4AFf7fLVr8K998Lpp0O38IqwNEvsHHQuaedUsE3AWm68ScZTc99L6frJlCUGLXi2/HzQdZ6a+15oZY9T5tcEvnsJrdpm+XKfM+/Xz5cuDhkC//d/8L//C2ecEXpQ//nkRdnb7SpJ68xVXF1ZwQZ28MF99pgh3HXewHb11201ppiSSuYHOpnt+kHXCRpPqr8wsSAYJOjFJ+ndr7NnQ2Wl3w36+ONw6aU+yE+a5HeNJhhXurtrO3Otf1eTzORFcq9gUzEttVyQDJq5FqVYN51MWWIyC6FB1ykyixvcE/3CtE1PnHhIb56a+17gi0RQvXm89NBVExcwduoSxp5xGJVf2Nv3aqmq8rPy3XeHX/4SrrwS9torcHyJrp9KmkmzxM5D55J2Tl0isMOn9de/nLKoOafe0gXH9k06bztlfg2bt21v9/l4P9Adbdc/8ZDePDFndat+KyXFRZx9dHlKG3/iBct4z7OlnsXdGDVxAVXTl7d6rvFmxAD1H29k/vW/ZuiSF+i1eqXfKXrvvfDd78LOOye8V0uZ5mU7e61/V6Ij/jqnLhPYY2ILpLGZbKwqpqLf7knNIuNVAQCUlRT72WwKP9BT5tcwaV5Nq6BuwNlHl3Nz5REpbfwJCsZBDFi/uQFo/1zbznzL6j/m2/Nf5JJ5L9B7cx1Lywdw6MSJMGIE7JT6j1CmM27NEjsXnUrV+XS5wA4+uLetgBk8bmZSs8igALpzj51S/uGOdy0HzFrme+m0nQ3FcsipdF6MxwjuyFg5qLx5Rtyn7gMuq/4T5731EqUNW5m1/9E8eOzZzO17BCu/eXrS92sr0xl3FGeJquKRXOqSgT2eZGeRYeZ3O7pWsrnoKfNr6BaQk28rKHcfu/7gcTO5sLiW/s//hmFLX6PRujH188cz/piz+Efv/gBJVxoFCWPGHaVZYlTbW0h0FWRgT2d2lOwsMtPZZsuxBQXj2LWSyUXHgkYyQT22cSTuIrJzHL/yTUa+PonBq95iS2kvHvvK2Txw5Df4cJc9W10j05RHFGfcmVCtt+RawQX2dGdHyc4iM5ltth1bvGDc8lpBM/rYzDrRi0Ns7SDoyLrYOIobGxi+9FW+//pzHFr7Lu/32oNfnXAprxx/Ji+NPYOyLKUQojTjzpSqeCTXCi6wpzs7SnYWmczjgt4xBOXni8xocq7dtYLeHRg0fz5opt7kXPM6Qmw8LStgqk7pz7u33MXZ//ss+2z8D8v27MfV3xjF84ceR0NRMbbl0+fbVQJwtqiKR3Kt4AJ7JrOjZINYosclescQNIYm51g57hvtPh/v3UG8hc94YkGj7Xi2v/ce//nRA1y8aDrFGz+h+oBBjPn6T3hlv6Na9UBPNehocTCYqngk1wousKcyO8pGMEr0jiHVmVvbjVWJFj5bahk0YuM5uPZdRr7+HGe8/QpFrolZRxzH0N/dxppue/P65EXQQdBJ9G+lxcHEutqaguRfwQX2ZGdHiYIRpP9LmOgdw53nDUx55tY2Jx4kbjrHOfq+NZf/njuZIf+qZnNxD54ceCoPf7GSmrK9WXn00VQm8Xw7CtxaHOyYUlqSSwUR2NvOJs8+upwXFr5PXb3fgNOzuH1LnKBgdNPzS9jS0JT27DPRrDzdmVtHm4/atUndvh2eeQaqqni6upp/l+7KbV/7No8POo26ks8ArUsWOwo6HQVuLQ6KdC6RD+zxZpMT33ivVSJ6/eaGpFvqxnZjtpTK7LOjdwzpzNwSBchWp0Ft2gSPPAJ33AErV8JBB7HgF+P4TuMh1LX4X51qfrejwK3FQZHOJfLdHePNJhsaHQ1NrXPRybbUDZLs7LNlu2DDB95MDx0IGmvswI7K8mK44QZ/xNyPf+xPKZo8GZYuZeCvfsbY8yoyGk9HHfyS6WIpIrkT+ROU9hszLakqEfAVJbHqk6CeL2b+eM62ykqK2bnHTnlZ/Ao6peaeip05+S9PwqOPwpYtvu/5ddcxpbR/qAt1yZySo6oYkezrMicoBaUBgh4bEws6Y6cuac7FQ/ygXtzN2LRte/Pjcl310TY3f/LHKxm7/EU+96u/QHExXHwxXHMNHHJIVipUklkb0OKgSOcR+Rl70My7m0HLbEysFW7bnZhBPdpbVpls3rY9bu697dmlicaY8Wy2qQleeMH3QH/tNX8Q9A9/+GnqZYdkz1kVkejpMjP2ykHlVK9a166neZEZu5bsRN3mhuZDJ9qeIXrVxAWB1225aWi/MdPiPmZtXX3coA2fzm7LSovZuGV7c84/5Rn0li3+ZKLbb4dly6BfP966dixX96rgnXr43KNvM3pYY0rnrIpIYYt8YAff5rbt+46GJkdp952Yf8MpQPy2vIm0TNsEpXvKSovbpT1G/3EhmF/AhQyqbNav9wdC33MPfPghDBoETz7JDd0P4bE31uJ2DKftC4UqVEQk8lUxkNwsNZUZa9uKjqCqD+doX5HT5JqDejpjZtUqGDUK+vaF66+HI4+EGTNg3jymHHKcD+ptvqVlxY8qVEQklMBuZqea2XIzW2FmY8K4ZiqSOVA3mRlrUDlgUAnjhvr2s/G0x7xgAVx4IRxwANx3H5x1lv/c9OkwdCiYUTV9eWAFUOyFIhvlliISLRmnYsysCLgfOBlYA7xhZlOdc29neu1kxdsUBLBp63amzK+hclB54GNiOlpcjFf1kehw7ESaZ9DO+dl4VRW8/DL06gU/+QlcdZWvSW8j0buOthU/CuQiXVcYM/ZjgBXOuX8557YBTwNnhnDdpMVmqbuVFrf6fF2933EaC+63jDiCspLidt+fbqoiXtqjuJtRXGQB3+EXdccNP4TKJbN83nzYMFi8GG65Bd57z+8ajRPUIfhdh+0Yi4gIhBPYy4H3Wny8ZsfnWjGzkWZWbWbVtbW1Idy2tcpB5ZR2b/8GpGX+uXJQOQtuPIW7zhsYSqoiXtqj6twjqTrnyHYvMgB7NG1lyrbXOfOc4+Cii2DbNpgwAd59F8aMgbKyhPeL90JiwIVf2lczdBFpFkZVTLzpabtUsHNuPDAefB17CPdtJ9lSvzBTFUHXqhxU3lwKuW3NWn685EXOn/ci3T/ZAF/7GvzmN3DaadAt+ddWtX/NDu2alUITRmBfA/Rt8XEfYG0I101ZUKmfAw74+YtccGxfbq48IuVf5HR/8StLPqFyxZPw2GPQ0AAjRsDo0XDssR1+b9A9lT8Pl3rJSyEKI7C/ARxkZvsBNcD5wLdCuG7KEi2QNjrH43NWs7J2I2+u3pD0L3LKv/jOwezZfkF06lTo2RMuuwyuvhoOPDCp56FgkzvqJS+FKOPA7pzbbmZXAtOBIuB3zrklGY8sDW1PHIpn9jvr2n0u0S9yol/82NfX1tXT5zPdua37uxw76XcwZw7ssQfceCP86EfQu3dKz0PBJne0U1cKUSg7T51zLwIvhnGtTMVSFf0D2gAESfUXvKaunlETF9C9YSvfWvxXLntjCvuvX8umPv1Y8bObGVU6iJWbHZ+bsCjlnK2CTe5op64UooJoKRBPsueDxiTa5BS3nUD9x1z85jQufvMF9ty8gQX7HMQVZ45hzpHHUd9k1G9OP42iYJM7OmhaClHBBvYLju3L43NWJ/XYRL/IbX/x+9R9wPfemMI3F82gtGErfz3gi4w/ZgRz+x7um7lvbWp3jVTTKEHB5sRDejN43ExVb4RIlUZSiAo2sN9ceQQAT819L+HMvbyDX+TY55+fMJWz/voUX18+m0brxpTDTuChL57FP3v3S2o8qaRR4gWbeN0ptaAaDlUaSaGJfD/2mEQliUGnLLU8USnuNd9cw2v3PMbZs57iy6sX8XH3Up4YdBqPHD2cj3bZI6XxxV5A0p0Zqs+6iHSZfuwQvzxw1MQFVK9ax82VR6Ses962jTdvuZ/D7r+bytpVrN1lT24+8VKePvJUNvYoTWuMJx7SO6MSRi2oikiyCqJtb7zyQAc8MWc1U+bXJN/KdsMGX3++//4cNfZqtmOM+sbVHH/5Qzx8zIi0g3pZSTGzltUmLJvsSDIdLEVEoEACe9Cs1UHzomXCVrZr1vgdoX37wnXXwYABXHLuTXz9u/fy3OFDaChq3/clWSXFRYw947CMZ9zqsy4iySqIVEyiA61b9ilvl/JYtAhuuw2efNLvGD33XB/gjzqKFeNmQoKga8RpiNNGkVnzC0jQpqlkZ9yq3hCRZBVEYB89bACjJi6IG2jbBU7nYNYsn3L5y1+gtBSuuMKfWtS/f6trBrUnKCku4pYRvuomFrDbBvrYY2KBN4x6aVVviEgyIlsV07YKpv8eJfz9nXWtgmss2JaXlXDdSQdw5r/mwK23wptvwl57+UMtfvhD2H33hPeoqatv3vAUVB6ZTKMwdREUkUwkWxUTycA+ZX4No59d2Ops0eIi47wv9mXWstpWM+iSbVv45qIZfP+NKfTZ8CEcfDBce63vh96zZ+ZPRkQkRwq63PGm55e0OzC6odEx7a33mX/DKQweN5OtNWu5eN4LXDT/RXbb8gnV5Ydy3/ArGPf7Xzb3QNcMWkQKUSQD+/rN8Q+RXr+5Af7xD370dBVnL/4rxY3bmXHQsTx4zNm82edQDBjXIqhnszWuXjREJF8iGdjjOapmKZfPnQS3zuXsop2YdPhJPPzFSv61R5/mx7RcSM1ma9yOXjQU9EUkmyIZ2EuKu1Hf0IS5Jk7+51xGvj6Zipql1PXsxaxzvs+WkT/kv1/5IGEFSjZ3cnbUw12HaIhINkUysPdsbODMBX5B9IB1a1jzmb0Ye9JIJn7hFOq794SXa9ittJgeO3VjQ31D3FlxNlvjJnrR0CEaIpJtkQrs0/62mJX/fRsvzf0TvTfVsfizB/Dj4aN58ZCv0tit9a7MWB5+t9Li5pl6y5a3bbslQnBdeaqpk0QvGur5IiLZFpnAPmV+DVuu+RlXvvkir+x3FA8eM4K/9zvS90BPYP3mBkY/uxAcNDT5Spqaunomzavh7KPLmbWstsPa81RTJ4k2I2W6A1VEpCORCexV05djx5zNo0d+nWV77ZfS97YtjQSf/pi1rLbDlrfppE462v6vE3tEJJsiE9jX1tXjdv1s6NdM9zFBvWligrb/q+eLiGRbZAJ7okZfmVwz3fsaPk2TTkBWzxcRyabIBPZETbk6UlxkrXLsED/98cuUEuzWAAAJSElEQVQpi5qP0isy44Jj+wY2GGvZEjhG9eki0hlEJrBXDiqnetW6Ds8wbWu30mJuHH4YkDj98cspi1odft3oXPPHQXdrmabJ9k5WEZFkZRTYzexcYCxwKHCMcy7cg0xbmDK/hknzalIK6nedN7BVUE0UYJ+a+17g58uTqHlXfbqIdBaZnqC0GBgBvBrCWBKKFzgTKS8rSSmgBr1gNDqX1OlFqk8Xkc4ioxm7c24pgHVQSx6GVAKk4Q+PTqRtPjzoRCTj0xeVRD3Zs7mTVUQkFTk789TMRppZtZlV19bWpvz9ZaXJnzvqgEnzapgyvybu12P58Jq6ehw0/zeebt2sOWA3Otc8U2/7bkBnkopIZ9FhYDezl81scZw/Z6ZyI+fceOdchXOuonfvxLPp+N+f2uNbNt1qK5m0TjeDnbsX0djU+sZB1+3wwGwRkRzpMBXjnBuai4F0ZEN9/B7siWSS995n19T7uqg+XUQ6g5ylYjKVTq466HuSuVYs9x7WWEREciWjwG5mZ5nZGuDLwDQzmx7OsNobPWwAqSzRxvLbU+bXMHjcTPYbM43B42b681Lj5MPb2rWkmM3btgdeV0Sks8oosDvnnnPO9XHO9XDOfdY5NyysgbVVOaicC7+0b8LgXmTWKr8NtFskjW0aiuXDgXbXLO5mbNq2vd0RfGUlxcqbi0inF5mdpwA3Vx7BytqNzH5nXdyvNznHynHfaP548LiZgZuGZo8Z0hyg25Y+bo4T1AF27rGTgrqIdHqRCuwAf/9X/KAO7XPfiToztmzg1XbRc78x0+J+nzYbiUgURGbxNCZR2WPb3HeiRc6fT14UWOeuRVMRibLIBfZEktk0FJOozl2bjUQkyiIX2IMWT2P90VtWwADNi6jxJEqt9Njp03+a3Uq1aCoi0RG5wB6UiXEEV8CUp5BaibUbqGuxIWpLQ1OGoxYRyZ3IBfagIF1kFlgBk0pqJVH7XRGRKIhcYB89bIA/EamF4iILbLu7tq4+pT4uar8rIlEXuXJHoF0+prHR0c2gKU5sj6Vbku3jova7IhJ1kZuxV01f3ursUoAm4gf1dCpZVBEjIlEXuRl7simRIrO0Kllij9eh1CISVZEL7EGpkraanEs7GKv9rohEWeRSMcl0ZgTlxEWk64pcYK8cVM7ZR5cn7PKonLiIdGWRS8UAzFpWG7hRKd5B0yIiXUkkA3vQAqoBs8cMye1gREQ6mUgG9s5ca962t7vePYhIrkUqsMeCZk1dPUbrfUqdIa8e6zMTa0nQsl+NgruI5EpkFk9jQTM2U3d82ukxUYuAXFKfGRHpDCIzY48XNB3+HNLOkldXnxkR6QwiM2MPCo519Q2BJyHlmk5eEpHOIDKBPVFwvOaZhc2Ha+QzyKvPjIh0BhkFdjOrMrNlZvaWmT1nZmVhDaytRMGx0blWh2vkK7in0h5YRCRbzCU6HbqjbzY7BZjpnNtuZr8GcM79rKPvq6iocNXV1Snfb9B/vcT6zQ0dPq68rKTT5N1FRMJiZvOccxUdPS6jGbtz7iXn3PYdH84B+mRyvY7cOPywpPrEaLFSRLqyMKtiLgUmhni9dtq21O1m8U9O0mKliHRlHQZ2M3sZ2DvOl653zv1px2OuB7YDTyS4zkhgJMC+++6b1mChdUvdthuCQIuVIiIdBnbn3NBEXzezS4DTgZNcgoS9c248MB58jj3FccalQzFERNrLKBVjZqcCPwOOd85tDmdIyWnbk+XO8wYqoIuIkHmO/T6gBzDDzADmOOd+kPGoOqCeLCIiwTIK7M65A8MaSCoS9WRRYBeRri4yO09bUk8WEZFgkQzs6skiIhIskoFdPVlERIJFpm1vSypzFBEJFsnADq03KomIyKcimYoREZFgCuwiIgVGgV1EpMAosIuIFJjILp627RWjqhgRES+SgV29YkREgkUyFZOoV4yISFcXycCuXjEiIsEiGdjVK0ZEJFgkA7t6xYiIBIvk4ql6xYiIBItkYAf1ihERCRLJVIyIiARTYBcRKTAK7CIiBUaBXUSkwCiwi4gUGAV2EZECY8653N/UrBZYlea37wn8O8Th5JOeS+dTKM8D9Fw6q0yeSz/nXO+OHpSXwJ4JM6t2zlXkexxh0HPpfArleYCeS2eVi+eiVIyISIFRYBcRKTBRDOzj8z2AEOm5dD6F8jxAz6WzyvpziVyOXUREEovijF1ERBKIZGA3s3PNbImZNZlZ5FbKzexUM1tuZivMbEy+x5MJM/udmX1kZovzPZZMmFlfM5tlZkt3/Gz9NN9jSpeZ9TSz181s4Y7nclO+x5QJMysys/lm9kK+x5IJM3vXzBaZ2QIzq87mvSIZ2IHFwAjg1XwPJFVmVgTcD3wd+DxwgZl9Pr+jysijwKn5HkQItgPXOOcOBb4E/CjC/1+2AkOcc0cCA4FTzexLeR5TJn4KLM33IEJyonNuoMod43DOLXXORfXk6mOAFc65fznntgFPA2fmeUxpc869CqzL9zgy5Zx73zn35o6/f4IPJJFs+O+8jTs+LN7xJ5KLaWbWB/gG8HC+xxIlkQzsEVcOvNfi4zVENIAUKjPrDwwC5uZ3JOnbkb5YAHwEzHDORfW53AVcBzTleyAhcMBLZjbPzEZm80ad9gQlM3sZ2DvOl653zv0p1+MJkcX5XCRnU4XIzHoBk4CrnHMf53s86XLONQIDzawMeM7MDnfORWodxMxOBz5yzs0zsxPyPZ4QDHbOrTWzvYAZZrZsxzve0HXawO6cG5rvMWTJGqBvi4/7AGvzNBZpwcyK8UH9Cefc5HyPJwzOuToz+xt+HSRSgR0YDJxhZqcBPYHPmNnjzrlv53lcaXHOrd3x34/M7Dl8WjYrgV2pmNx7AzjIzPYzs+7A+cDUPI+pyzMzAyYAS51zd+R7PJkws947ZuqYWQkwFFiW31Glzjn3c+dcH+dcf/zvycyoBnUz29nMdon9HTiFLL7QRjKwm9lZZrYG+DIwzcym53tMyXLObQeuBKbjF+iecc4tye+o0mdmTwH/BwwwszVmdlm+x5SmwcBFwJAd5WgLdswUo2gfYJaZvYWfSMxwzkW6VLAAfBZ4zcwWAq8D05xzf8nWzbTzVESkwERyxi4iIsEU2EVECowCu4hIgVFgFxEpMArsIiIFRoFdRKTAKLCLiBQYBXYRkQLz/w19mHUnN1bpAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the scaled\n",
    "plt.scatter(X_gdp_scaled[:,1], y_gdp_scaled)\n",
    "x_lin = np.linspace(-1,5, 100)\n",
    "y_lin = lin_reg_gdp_scaled.predict(x_lin.reshape(-1,1))\n",
    "plt.plot(x_lin, y_lin, \"-r\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1a2bc27080>]"
      ]
     },
     "execution_count": 207,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xd8lFX2+PHPyaTRwYAoJYKCBVRQsouIq4JtVxH1axcXV0W2iHVV9OvXddX9uSo27CIWUHRFsCDquiqWFQUFDQgoCirVAkgvKZPz++POhEkyM5lM5pmW8369oskzz8xzJxPO3Dn33nNFVTHGGJP9clLdAGOMMclhAd8YY5oIC/jGGNNEWMA3xpgmwgK+McY0ERbwjTGmibCAb4wxTYQFfGOMaSIs4BtjTBORm+oGhGrfvr1269Yt1c0wxpiMMXfu3LWq2iGWc9Mq4Hfr1o05c+akuhnGGJMxRGRZrOdaSscYY5oIC/jGGNNEWMA3xpgmwgK+McY0EZ4GfBG5TEQWiMhCEbncy2sZY4yJzrOALyL7AxcBvwb6AENEpKdX1zPGGBOdlz38/YBZqrpNVSuB94FTPLyeMcaYKLwM+AuAw0WkSESaA8cDXT28njHGZJaqKnj9dbjjjqRczrOAr6pfArcDbwH/BuYBlbXPE5GRIjJHROasWbPGq+YYY0z62LYNHn0UeveGE06Ahx+GHTs8v6yng7aq+riqHqyqhwO/AN+EOWecqpaoakmHDjGtDjbGmMz0ww/wf/8HxcXwpz9BixYwaRJ8/TUUFnp+eU9LK4jIrqr6s4gUA/8DDPDyesYYk5ZKS+Gee+C556CyEk46Ca64An7zGxBJWjO8rqUzVUSKgArgYlVd7/H1jDEmPVRVwWuvuUD/7ruuN/+nP8Gll0KPHilpkqcBX1V/4+XjG2NM2tm6FSZMgHvvhW++gS5d3KDsRRdB27YpbVpaVcs0xpiMtWoVPPCAG4xdvx5+9SuXwjn1VMjLS3XrAAv4xhjTOHPnurTN88+7NM4pp7j8/KGHJjU/HwsL+MYY01B+P0yfDnffDR98AC1bwsUXw2WXQffuqW5dRBbwjTEmVlu2wFNPufz80qWwxx5w111w4YXQpk2qW1cvC/jGGFOfFSvg/vvhscdgwwYYMAD++U+XvsnNnDCaOS01xphk+/RTl7Z54QVQdQOwV1zhAn4GsoBvjDGh/H545RUX6GfOhFat3Nz5Sy+Fbt1S3bpGsYBvjDEAmzfDE0/A2LHw3XcuuN9zD1xwAbRunerWJYQFfGNM07ZsGdx3H4wfD5s2wcCBMGaMK3+QQfn5WGTXszHGmFjNmuXSNlOnuvnyp5/u8vO//nWqW+YZC/jGmKajshJeeskF+lmz3FTKq66CUaOga/Zv12EB3xiT/TZuhMcfd6mbZctgr73cNMs//MEtmmoiLOAbY7LXd9+5IP/4425Q9vDD3aKpE08Eny/VrUs6C/jGmOyiCh9/7NI2L70EOTlw5pkuP9+vX6pbl1IW8I0x2aGy0g3A3n03fPIJtGsH11zj8vOdO6e6dWnBAr4xJrNt2OCmVN53nyuB0LMnPPggnHee23TEVLOAb4zJTEuXuiD/xBOuqNmRR7p69EOGuDSOqcMCvjEmc6jChx+6FbAvv+wWRp11lsvPH3RQqluX9izgG2PSX0UFTJ7sAv3cuVBUBP/7v/CXv0CnTqluXcawgG+MSV+//ALjxrlUzapVsO++8Mgj8PvfQ/PmqW5dxrGAb4xJP99844qYPfkkbNsGRx/tAv9vf2v5+UawgG+MSQ+q8P77Lm3z6qtu4+9zznH5+QMPTHXrsoKnAV9ErgBGAAp8AZyvqju8vKYxJsOUl7sNwO++G0pLoX17uOEG+POfYbfdUt26rOLZZyMR6QxcCpSo6v6ADzjLq+sZYzLMunVw662u7vzw4S7wjxsHy5fDTTdZsPeA1ymdXKCZiFQAzYHVHl/PGJPuFi929WwmTIDt2+HYY12u/thjXZli4xnPAr6qrhKRO4HlwHbgP6r6H6+uZ4xJY6owY4bLz7/2GhQUwLnnwuWXw/77p7p1TYaXKZ12wElAd6AT0EJEzg1z3kgRmSMic9asWeNVc4wxqVBWBk89BX37upk2n34Kf/+7S9uMH2/BPsm8nN90NPCdqq5R1QrgReDQ2iep6jhVLVHVkg4dOnjYHGNM0qxZA7fc4vLz55/vNgYfP97Vor/xRth111S3sEnyMoe/HDhERJrjUjpHAXM8vJ4xJtUWLXL5+aefhh073Lz5K690vXvLz6eclzn82SIyBfgMqAQ+B8Z5dT3TNKzbUsbK9dvp0q4ZRS0LUt0cAy4//9ZbLj//739DYaGbdXP55bDffqlunQnh6SwdVb0RuNHLa5im45XSVYyeOp+8nBwqqqq449QDGdrX6pynzI4dMGmSC/QLF7pplP/4B/zxj24uvUk7ttLWZIR1W8oYPXU+Oyqq2EEVANdMnc/AHu2tp59sP/0EDz8MDz3kcvV9+riB2bPOcrNvTNqygG8ywsr128nLyakO9gB5OTmsXL/dAn6yLFjgevOTJrnZNyec4PLzgwZZfj5DWMA3GaFLu2ZUVFXVOFZRVUWXds1S1KImQhXefNOVPXjrLWjWDC64AC67DPbZJ9WtMw1kZedMRihqWcAdpx5IYV4OrQpyKczL4Y5TD7TevVe2b4fHHoPeveF3v3O9+1tvdVsIPvSQBfsMZT18kzGG9u3MwB7tbZaOl3780QX0hx+GtWvdLlITJ8KZZ0J+fqpbZxrJAr7JKEUtCxIW6LN5imeDn9v8+S4//+yzbnepIUNcfv6IIyw/n0Us4JsmKZuneMb83Kqq4PXXXaCfMcPtIHXRRS4/37Nn8htuPGc5fNPkhE7x3FxWyY6KKq6ZOp91W8pS3bRGi+m5bdvmtgns1QtOPNFVr7ztNpeff+ABC/ZZzHr4pslZuX47vlppilRN8Ux0Winq9NVN6+DBB12w/+UXKClxUyxPP93tLmWyngV80+QsWLWRreX+GseCUzyTmdf3Iq0Ubvpqj9XfsO/oiTDlBaishJNPdvn5gQMtP9/EWMA3WStc8F63pYxbXltU59wbTujFh0vWJi2v79XK4eD01dFTSjl6yaecO+sl+i+bDy1bui0DL70U9torUU/DZBgL+CYrReo9h0t5tCjw0XWX5ox8ek7SSjd4tnJ461aGznyZE/51L76lS/B36QJjxsCIEdC2bQJabjKZDdqarBNt4DJcysNfpYCSl1Pzn0MwAHsh4SuHV66Ea6+Frl1h1Ch87YvgX//C9913cNVVFuwNYAHfeGTdljLmrdgQdeZLLOfEI9h7DhXaew63Yrd3pzZJLd2QsJXDc+e6rQK7d3c9+aOOgo8+glmz3GKpXPsQb3ayvwaTcLEMRno5Dz5S77lFvo95KzYwsEd7Zo4ezMr122mR76sewL3j1AO5plabvBy4jXvlsN8Pr77q6tv897/QqhVccon76t7ds/aazCeqmuo2VCspKdE5c2xTrEy2bksZA2+fwY6KnQG3MC+HmaMH1xg4re+cxppWuqo6eJf7qzhq311556ufyfftDOYKdd500rp0w5Yt8OSTMHYsLF0Ke+zhBmFHjIDWrVPdOpMiIjJXVUtiOdd6+CmWbcv7YxmMTEap42DvedLs5Tww42teX/AjAGWV7ppXT5kHCGWVNQdpZ44eTJ+uaZbvXr4c7r/fFTPbuBEGDHALpU4+2VI2pkHsryWFsnF5fyyDkcksdfzQe0uoNeUeAJ/kQK0p6GlXX/+TT1zaZsoU9/Opp8IVV8Ahh6S2XSZj2aBtimTr8v5YBiOTVeo43OBtkF+rArNzdkqL+vp+P0yd6hZF9e8Pb7zh9oZduhSef96CvWkU6+GnSDbv4BTLYGQySh2H+yQBUJCbw5jTDgRI6iBtVJs2wRNPuPz899/Dnnu6788/3w3KGpMAFvBTJNt3cIqljHFDSh3HM9YR/CSxc/DWz6hBPTmnf3H1Y6R8kHbZMrjvPpef37wZDjvMpXGGDgWfL/ntMVnNAn6K1A5GKe9hprHGjHUEP0ksXL0JUHp3alMnvZSS3/nHH7uyxFOnQk4OnHGGy8+XxDTZotGybbJAqmTa79GzgC8i+wDPhxzaE/ibqt7r1TUzje3gVL9E1JxJZo2cqCor4cUXXaCfNcutfr3qKhg1yq2QTZJXSldxzZR5+CQHv1Yx5rQ+GT9ZIBUycdKFZ4O2qrpYVfuqal+gH7ANeMmr62WqopYF9Ona1oJ9BNFWzcYiVYPjNVYRb9zo0jR77eVWv65d66ZZrlgBt9+e1GC/bksZf51cSlmlsq3CT1mlcuXk0oyfLJBsmTrpIqYevoj4gI6h56vq8gZc5yhgqaoua1jzTFPX2LGOVAyOB3t+3Tf+xFmzX2bYgrfJ3bbVbRd4331u+8AU5ecXrt5EZa1x7Moqd/zwvTukpE2ZKFMnXdQb8EXkEuBG4CeofnYKHNiA65wFPNfg1pkmryFjHeHyqckeHF+3eQfP3/Mc98x6iWO/mUWVCK/1OpwjHr6Vtoelw5TKSCvr02fFfSbI1EkXsfTwLwP2UdV18VxARPKBocB1EW4fCYwEKC4ujucSJsvFMtYRKZ+atMHxigqYMoVmt43h2fmfs6GwJY/0P5WJB5/Atva78cwe+5IO63d7d2pDnk+o8O8M8Hk+oXenNilsVebJ1EkX9dbSEZF3gWNUtTKuC4icBFysqsfWd67V0jHxiLV+jyeD4+vXuymV998PK1fi36sH/+h5LP/abzDb8wvDtiXVppWu4uop8/HlCP4qZcxp6T/YmK7SYZZOomvpfAu8JyKvAdUjEqp6d4ztORtL5xgPxZJPTfj0yyVL3MKoJ5+ErVth8GB4+GF8xx/PQfN/4Lmp82mVpj0/mx2WOCmb1hunWAL+8sBXfuArZiLSHDgG+GPDm2ZMbJKWT1WFDz5w0yqnTXOFy845x82f79On+rRMCKiZFqhMYtQb8FX1JgARaeV+1C2xPriqbgOK4m+eyTSp+IjreT61vBwmT3aB/rPPoKgIrr8e/vIX2H33iG2ygGrSTSyzdPYHngZ2Cfy8Fhiuqgs9bpvJMKlciOJJr/qXX2DcOJefX70a9t0XHn0Ufv97aJbeszGMCSeWlM444EpVfRdARI4EHgMO9bBdJsMkYkVsYyWsV/3113DvvTBhAmzbBsccA+PHw3HHuTIIxmSoWAJ+i2CwB1DV90SkhYdtMhko3oUo6TDLAXD5+ffec2mb6dMhLw+GDXOliQ9syJITY9JXTLN0ROQGXFoH4FzgO++aZDJRPAOn0VJASXsjKC+Hf/3LBfrSUujQAW64weXnO3b07rrGpEAsAf8C4CbgRdweQR8A53vZKJN5GjpwGi0FlJRiZ+vWwSOPwIMPwg8/QK9ebj79sGGWnzdZK5ZZOuuBS5PQFpPhGjJwGikFtHD1Rm/HAr780uXnJ06EHTtcXv7JJ+HYY0Gk/vsbk8EiBnwRuVdVLxeRVwlTaENVh3raMhO3VObFYx04jZQCAkl8USpVeOcdV7HyjTegoMDNtLn8cujdO77HNCYDRevhB3P2dyajISYxMqVGd6QUUO9OrRO3iGrHDnjuOZef/+IL2HVXuOkm+NOf3PfGNDERA76qzg1821dVx4beJiKXAe972bBkSpuZIo2UDlMjGyJSCqjRi6jWrIGHH3b5+Z9/hgMOcGmbs892vXtjmqhYBm3PA8bWOvaHMMcyUqb0iGORiTW6w6WA4l5EtXChy88//TSUlcHxx7uyB0cdZfl5Y4iewz8bOAfoLiLTQm5qBcRVKjndhOsRXz1lHm2b59XZ+zQVbWtowMvUGt3hxLyIShXeesvl5998EwoL4Q9/gMsug/3287ydxmSSaD38j4AfgPbAXSHHNwPzvWxUsoTrEZdVKn965jOqVFPW24/3U0em1uiOy44dMGmSy88vXAi77Qa33OLy8+3bp7p1JoWyJUXrhWg5/GXAMhEZBqxW1R0AItIM6AJ8n5QWeihcjxhgW7kfSE3+u7F5+GRUakzpP6iffnL5+Ycecrn6Pn1cCYQzz7T8vMmqFK0XYikMMhkIjYp+4AVvmpNcwR5xYV4OzfPq7jGaI8KEj75nyU+bE3bNGptbh9GQTbsjPVZRywK6tGvGyvXbE76p8iulqxh4+wzOHT+bgbfPYFrpqoQ+fkQLFsCFF0JxsZtp078/zJgBn38Ow4dbsDcZu7F4MsUyaJurquXBH1S1PLBtYVYI9ogXrt7ERRPnUBayw/O2cj/3zVjCfTOWMHxAMTefdECjrhVL7yPWPHy0x/Kql5P0WUBVVS4vf/fd8PbbbgXshRe6/Pw++yT+eiEsLZB5MnHSQrLF0sNfIyLVi6wCWxau9a5JyVfUsoDD9+7AmNMCvf38ur39iR8vb1RPP9beR+injlYFuRTm5dTJw0d7LC97OQtXbySn1myXSJ8+GmXbNleGuHdvN9Nm0SK49VZYscKlcjwO9in7FGMaJZsmLXgllh7+n4BJIvIArpbOCmC4p61KkWBvf8JH33PfjCV1bi9dsYEeHVvF9dgN6X3Ul4eP9ljB7xPdy3mldBXXTJlHWWXNRdex/IOKubf8ww9u7vwjj7haN/36wTPPwOmnQ37sHyob0zvPtLUMZqcmNWkhTrHU0lkKHCIiLXGbnicuoZ2GiloWMLRPp7ABv2/XtnE/bkN7H9GmJdb3WI3p5dQOluu2lLFw9cawwb4gt+6nj9pqp5duGNKL/Tu1qRmMS0vdbJvnnoPKSjjpJDd//je/afD8+camsywtkNkyYXvJVIqlh4+InAD0Bgol8A9QVW/2sF0p1aNjK4YPKGbix8urjw0fUBx37z4YRG84oRe3vLao0b2PYE/m6inz8eUI/iqt8Vjx9nJqB8sz+nVh8tyV5IjUCfbN83w88vt+9O7UmnkrNoT9xxWut3z9Swtoke+jqsrPU7uupf8rE93ga4sW8Mc/uvx8jx4N/p1Eul5De+eWFsh8tr1kZLFscfgI0BwYBIwHTgM+8bhdKXfzSQcw/JBulK7YQN+ubcMG+1hSBzH1cOOgwf+qULu2XTy9nHDBcuKs5RHPr0JZ8cs2Rj49J2JvOlxvuVn5Dk75fAbnz3mFvX5Zhb9zZ3x33AEjRkC7dg35FdSRiN65pQVMNoulh3+oqh4oIvNV9SYRuQtXGz/r9ejYKmKvPpbUQbggesv0RcwcPbhRAST4uK7XHX7NQLCXE5y6GU+54nCa5/moQqs/rUTrTYf2ljtuXsvwz15jWOkbtN2xhdLde3L1/1zLuWOupM+eHeL+XYRKVO88m9ICNtvIhIol4O8I/H+biHTClVXo7l2T0l+sqQOv8sGxPm5D8tkt8n2U+aMH+4LcnOo0Trg2+HKEd7/6mUH77lr9hvNoL2HDrXdx/KL/kqNV/KfnITz+q5OY07kXhfk+rt21ddy/h9oS2TvPhrSALUIytcUS8F8VkbbAGOAzXP7gMU9bleZiDbiJzAeH9tRiedyG5LODgUHUpYYK89xs3TNKujB5zsoaAePwvXf2xmu3YWuZnxunLeSGl+fzVPuf6f/yBI54/320ZUvW/mEEHx1/DqNLt5CXk0OhR6mSbOqdN4bNNjLhRCuedrqqvgA8o6obgKkiMh0oVNWNsTx44I1iPLA/7o3iAlX9OAHtToh4P+7GGsgT1eMM11Or73FjfVMKDQxBVVXK65f+hh4dW3HZUXuH/R2FPjdfjrC1zE+z8h38z2dvc8GcV+i+/gf8Xbviu/NOZMQIOrRpw0nAYcd5n2LIht55Y9lsIxNOtB7+dbgSClOBgwFUtQxoyAqescC/VfW0wOrc5vE2NNEa83G3IYG8sT3OSD21maMHM3P04IiPG+1NKfSNLlxgKMj1sTVQTyha8Aw+t1nvlfLjrXdy2mdv0KZsK5/vvg9XnXo+vx9zBX261yxkZsE4OWy2kQknWsBfJyLvUrc8MlD/Foci0ho4HFc7n0B5hvJo90mWRHzcbUggjyXIRfq0Ea2n1qdr24iPG+lNqfYG4TcM6RVzYKjTxk8/peieezh+8mSqqpR/7z2Ax0tO5rMu+1GYl8N1HeKbxmoaz2YbmXCiBfwTcD37p6lZHjlWewJrgCdFpA8wF7hMVbeGniQiI4GRAMXFxXFcpuES9XE3Ub3VaJ82GtNTq/2mBDDw9hl1Zg3Fsj4g2MYClMO/+oi/L32Los8/gVatkEsvZcbRZ/DXj9cnPD9vs0ziZ+MZprZo5ZHLgVkicqiqrgEQkRygpapuivGxDwYuUdXZIjIWuBa4odZ1xgHjAEpKSupslu6FeIKoV4Gnvk8bje2phU7PfPern8nNqVsLZ//ObaKmh9ZtKePmZ2dx9mdvcv6caRRv/ImVbTqy9bYxtPjzSGjdmmOAmYfH/juKZw2DzTJpOEuhmVCxzNIZKyJ/wk34ngu0EZG7VXVMPfdbCaxU1dmBn6fgAn7KNTSI1hd4GvNmEMunjcb21ILt94lU5+aDgm90EQPDsmX4b72T9yY8QauybXzauRe3DrqAj/c/jInnHEqf1junVcYaXOJdw2CzTIxpnFgCfi9V3RTYCOV1YDQu8EcN+Kr6o4isEJF9VHUxcBSwqNEtjlPtoBxaFhmU3p3aRLxftMDT2F5oQ2b8xBPows3CAWhR4KtTkqGGWbNcfZupU+kAvL7PYYzrN5R5nVylykIk7umlqVzDYExTFkvAzxORPOBk4AFVrRCRWFMvl+AqbeYD3wLnx9nORokUlGsPYIYL1vVVpmxsL9TrwbVw7W+R7+OmE3tXL5CqVlkJL73kAv3HH0ObNnDllcioUfh/8bF46nxaNbKNqVjDYIxxYgn4j+K2M5wHfCAiewCx5PBR1VKgJO7WJUCkHmWv3VvHFKyjBZ5E9UK9HFwL136/as1gv2kTPP44jB0Ly5bBnnvCffe5zcBbuZk2Q4tJSBuTvYbBGLNTLOWR7wPuCzm0TEQGedekxIoUlEtXbIgpWNcOPOV+Pxcf6ao5xjrXPZYg5dXgWtTA+f33LrCPHw+bN7tyxPfcA0OHsm57pWu/lNWpz+NZe2qxWSbGJJaohs/OiMi5qvqMiFwZ7nZVvTvRjSkpKdE5c+Yk9DHXbSmrnooYVJiXw/RRhzHkgQ/rHI9U2GzdljImzV7Og+8uId+3M1ABdYKXQtrNLql+A2pbSNEXn7nA/uKLkJMDZ5zh6s+XuA9jyZgdY9MtjUkMEZmrqjFlUqL18FsE/h9u9UxSpk8mQqQeZY+OraL2NMMFpIfeW0JZZVX1vrfhVrxC3bnu8c4uSWRQLCr0UfTRmy7Qz54NbdvC1VfDqFHQpUuNayZjdoxNFzQm+aLNw3808O3bqjoz9DYRGehpqxJs54ycjYDQu1PrGsdrB9VwPdw9ilrEtOJ1XoyponBCA3wsA8ox2bDBpWzuvx+WL3ebizzwAJx3HrRsWed0mx1jTPaKZdD2fgK1dOo5ltYiBdDaPc1wPdyrXpjHsyP6xzTYGGsly2hvMuX+KvxVVVRWEX8ve+lSl59/4gnYsgWOPBLuv591Rx7Dwh+3wOpt9O6U16AaPMaYzBatWuYA4FCgQ608fmvA53XDEqkhaYpwPdxyv3LO459w1q/qlguuff/6BiXDfXoY2KN9nfbVFlMvWxU+/NClbV5+GXJz4ayzXH7+oIN4pXQVf71tBoGMFHk+4a7T+9T45GCzY4zJXtF6+PlAy8A5oXn8TbhtDjNGQ9IU4Xq4AOWVVUyes5Lpow5ja7k/al49Uqoo0hvPuN/3q3e3qai97IoKmDIF7r4b5syBXXaB666Diy+GTp2qr33NlHnVwR6gwq9cPaXuG5/NjjEmO0XL4b8PvC8iT6nqsiS2KeEakqYI9nCvemEe5f6aY9N5OTlsLffTp2vbeq8ZblAy0hsPSNg3maCC3Jzwvez162HcOJeTX7kS9t4bHnrI5eeb16xEvXL9dnySQ3BLxCBfjoR947NBVWOyT04M52wTkTEi8rqIzAh+ed6yBAoG8cK8HFoV5FKYl8MNQ3qxcv121m2pW95/aN/OvH7pb8jPrfnraWwuO9IbT+9Orbnj1APrXA/cHrKPDS+pOWC7ZAlccombXXPttbDPPjB9Onz5Jfz5z3WCffDafq37puKvUsvPG9NExBLwJwFf4faxvQm36vZTD9vkiaF9OzNz9GCeGdHflQOevohzx89m4O0zmFa6qs75PTq24s7Tar5JNDaXHe6NJ/iYQ/t25vVLDiPfV7OaZRXqZhWpwvvvw0knuZ78uHFu/vy8efD223DCCW5OfZRrjzmtD6HvKXk+Ycxplp83pqmIuPCq+gQ3qb+fiMxX1QMDx95X1SMS3RgvFl7VFmkhVrQFV4nOZUd7zGmlq2oMmI4Zui8nLp7p8vOffw5FRfCXv7iv3XaL69qhBeMs2BuT2RK18CqoIvD/H0TkBGA10CXK+WmtofPMvchlx7Jt4A/frmLPqc/Q/OQRsHo17Lef69Wfey40iz8FU9SyoMZG5CZ5bHWxSbVYAv4/RKQN8Ffc/PvWwBWetspD4fLoZZWVfL78F1rk++jRsfHb8jXqH/bixRTdey9FEybA9u1wzDFu4dRxx0VN2SS0DSbhbDMXkw7qTekkUzJSOlAzbbKlrLJGnYjhA4q5+aQDqn9uaOCM6x+2Krz7rkvbvPYaFBTAsGFw+eVwwAHR75uoNhjPNDSNaExDNCSlE7HLKCIXiUjPwPciIk+KyEYRmS8iByWqsakQHMD95/8cUKco0MSPl7Pkp82AC5wDb58RdXA3VOg8+81lleyoqOKaqfPDzgQCoKwMnnoK+vaFo46CTz+Fv//dlUB4/PG4gn2D22A8F0wjhgrdU8GYZImW0rkMeCrw/dnAgbiNyQ/ClUv+jact81hRywJ2VPjD3la6YgPtWuQ3uIhYpPGBhas30qZZPi3yfWwt99PVv5VdnnkSHnwQfvwR9t8fHn+cdUNPZeV2pUvzZhTF+bysFk76sXIVJl1EC/iVqhocsB0CTFTVdcDbInKH903zXt8IC6j6dm0bV+AM9w97R6WfiybOQUTo/MMyLpr7CvssmAGV5fDsehxTAAAXzElEQVS737myB0cfzSvzVjN67EeNTsNYcEk/Vq7CpItoAb9KRHYH1uP2o/1/IbdlRfTo0bEVwwcUM/Hj5dXHzijpzNZyPy3yfQ0OnHU3S6nC7/fzq29LGfHpyxz53Vx25ObzYu9BPDPgFJ4e8weKWhYktCSxBZf0ZOUqTDqIFvD/BszBFUqbpqoLAUTkCNz+tFnh5pMOYPgh3ShdsYF1W8u55+2veeOLn6ioquKMkvqLpdUW/Ie96of1FLzwPDlj76Xnz9+zpkVb7jpsGJMOOp5fmrehVUFu9aeFhas3kiM1F1w1Jg0TLbjY7J3UsXIVJtWi1dKZHti/tpWqrg+5aQ5wpuctS6IeHVvRrkV+nY1LYi2WFrRuSxk/frOc7lMmcuD4cfDzz3y5a3euOv5ypu13BOW5edXnlvvdp4j73vmGB2Z8TXmt4YTGpmHCBRebvWNM0xZ1Hr6qVuJSOqHHtnraIg9F691GytnHWixtxtQZrP3HHZz0xQwK/BWsGjiYlo9fxYkz/VSq1Dm/sgqOu/cD/GFmxdYulpaIXnmydrIyxqSvWBZeZYVg79YnQoW/ihtP7M2wQ/aovj2uwU5V+M9/KL/zLga//Rbbcwt44YCjebJkKEuLuvJnXzea5S9nc1llnbv6q8Kvf2ie5+OR3/erXg2bqF65zd4xxnga8EXke2AzriZvZayLAxIttHcbdP3LC0BgWH8X9Bs02Ll9O0ya5DYaWbQI2XU3xg46jycPOI4NzVpXnzb+v9+S04DVsRBSLI3E9spt9o4xpt5oFFh0da6I/C3wc7GI/LoB1xikqn1TFewhWAu+blrlplcX1ViQFFpRc+bowXV70j/+CH/7GxQXw0UXQX4+TJjApi+/5sFDz6wR7AHyc32MGtSDwrwcWuTXv0lYQa7UeJNJ5IKdaJU6jTFNQyw9/IeAKmAwcDOuxz4V+JWH7UqoLu2aUeGvWws+z1d384+wMym++ML15idNcrtLDRni5s8feSSIUATceGIvrn9pQY27+VU5p38x5/QvZuX67SxYvZFbpi9yqZVKP6pKs7xcyv1VjBrUg3P6F9e4dqJ75TY10JimLZaA319VDxaRzwFUdb2I5Mf4+Ar8R0QUeFRVx8Xb0MYoalnAjSf2dmmcEFE3/6iqgn//29W3eecdt6nIiBGuvk3PnnVOH9Z/D1C46dWF5Ply8KvW6EEXtSygT9e2/Lb3btUBF4gafL2YU29TA41pumIqjywiPlzwRkQ6QJTNV2saqKqrRWRX4C0R+UpVPwg9QURGAiMBiouLY295DEJntww7ZA8Ql8bJ8wn+KheQAeat2LAz6G7bBhMnwr33wuLF0Lkz3HabS+HsskvU6w07ZA9+u/9u9Qbx2p8oorFeuTEmUWLZAGUYbt79wcAE3Abm/6eqLzToQiJ/B7ao6p2RzklktcxXSldxzZT5+HJccB9zmpvdEvom8OGStdUzYNpuXMvjWz9h7xefgV9+gX79XNrmjDMgL6/+CxpjTAokZAMUEemuqt+p6iQRmYsrryDAyar6ZQyNaAHkqOrmwPfH4sYAPLduSxlXvTCPipBJ7n99YV717JbQcgZ7rlzChXNe5sRFH5Bb5ad8yInkX3MVHHYYhBnoNcaYTBUtpTMF6Cci76jqUbh9bRuiI/CSuKCZCzyrqv+Or5kNs3D1xhrBHqDCryxcvZHD994VqqrYNPlFJjxzO/2/n8fWvEImHfQ7Jg84hduuOSXsQqtklCSwsgfGGC9FC/g5InIjsLeIXFn7RlW9O9oDq+q3QJ9Gti9O4XvmOdu2wUMPwdixdP/6awpad+D/HXkBz/c5lk2FLSnMywk7iJuMkgRW9sAY47Vo8/DPAnbg3hRahflKW707tSY35Jl13LyW0e8/xaGDD4aLL4Y2beC555j79ic8fdhpaJu2EeelJ2NDkVivsW5LGfNWbLDNTIwxcYlWPG0xcLuIzFfVN5LYpkYralnAOb8u5rOXZ3Dhpy8z5Kv/kqPKj4OOo9PN18Ohh4IIJwKH7rd71DRKMkoSxHIN+wRgjGmsaIO256rqM0AvEdmv9u31pXRSxu9n0+QXGXLtzdy8fAGb85sx8eAhPNnvRH5otxtv9jiQrSs3Vgf4+ualJ6MkQX3XsMJnxphEiJbDbxH4f8swt6XPzudBW7bAk0/C2LG0XrqUzm125ZZBFzK5z7FsLgg8FYXfjf2Awrxcyv1+Rg3qWWd1a+2B03gWPzV08LW+a1jhM2NMItQ7Dz/snUQuV9V7E92YuObhb90KN90E48bBxo1U/PoQXjvqDP5auRf+nFjq1+RUz9GPljaJNYg3JvUS6RrrtpRV1+oPKszLYebowRbwjWniGjIPP96Av1xVE7ssljgDvt8P++0HBx3EBycM46Kv8yirjHUhsFOYl8P0UYcx5IEPGxVUvQzM00pX1fkEYDl8Y0xCFl7Vd40475d4Ph/Mn8+6SmHk7TMiBvs8n9SZm199W04OpSs2NDpt4mXqxUosGGMaq2HF2ndKrxx+YWHYUsKhbh7amwJf+Pepiqoq+nZt2+jB2XCDr+V+f8IGeIMF2CzYG2PiETFCishmEdkU5msz0CmJbYxJuGAbVJiXQ6vC3LDvUsHtBHt0bNXoevHBwdfQNQBVCjOXrI14H5tbb4xJlrhy+F5pbPG0aaWruHrK/Dppndwc8OXkIAI7KqpcT1+IaZZOJEt+2kzpig307dqWHh1b1bj/obfVTC1FyuPb3HpjTGMlI4eflgb2aM9jw0v4eOlanpj5Hfk+H+V+P1VKjQCsIrx+yWE1AnVQLPXi//byF0yctbz65+EDirn5pAMAl8fP9+XUuF64PL7NrTfGJFvWBPzaveW/ndib/Tu1YeP2ci6e9DkV/p0biRf4ctha7o/rOkt+2lwj2ANM/Hg5ww/pRo+OrWJeqGVz640xyRbvoG1aCVeL5qZpC2mR76N3pzYJXSlbumJDvccvPrIHBbkSdSzANhU3xiRbVgT8cDN0yv3K8fd/yMwla2sMxhbk5nDxkT0a9PihA6t9w5ROBujbtS2vlK5i4O0zGPfBt4Aw8vA9w2+Gjm0qboxJvqwYtF23pYxD/vlO2Hn2wQFTgEmzl/Pgu9+Q7/PFPEhaO1V0w5BefPTNWl5b8GP1OcMHFHPZUXvHtejKauAbYxqjSQ7aRnrjCubFu7RrxkPvLaGsUimrdPn8+gZJww2sXv/SAlrk+8j3wen9ijl/oMvdz4tz4ZZtKm6MSZasSekU5Iavm1Pud3nxcGmfYECO9rjhFnNtLfdT7oepn6+kXYt8wHLyxpj0lxUBv0u7ZlT4wy+6umBgN4paFsQVkKMt5oKabxiWkzfGpLusCPhFLQu48cTeYW97YuZ3TCtdVW9ADrfiNfQ+LQrqfoKo/YYxtG9nZo4ezDMj+kccrDXGmFTJikHboEmzl3HTtEWU1+rthw6ehhskrW/Fa/A+C1Zt5JbXFjWqfLIxxiRSkxy0BRjWfw+6tmvO+U98QuiyKq3S6sHT2oOksax4Dd6nT9e2/Hb/3Rr8hmGMMekgK1I6oabMXUHtNbRlfqVFfvhB3XADszkiLFy9Mez5tStWJmOTc2OMSQTPA76I+ETkcxGZ7vW1lvy0mWnzfqhzPN9HxFIK4QZmt5X7uWjiHKaVrqr3mvHM/jHGmFRIRg//MuDLJFwnYtmDKiXibJzgwGxBbs1fRVmlxtRTt+mYxphM4WnAF5EuwAnAeC+vExSp7MEJB+wedSB1aN/OPDa8hOZ5NdM+sfTUbTqmMSZTeD1oey9wDVC3DrEHenRsxRklnZk8p2Yq5s1FP7FuS1nEWToAvTu1pqrWFimx9tQH9mjPuN+XAErvTm0s2Btj0pJnAV9EhgA/q+pcETkyynkjgZEAxcWN3xd9WP9uvDb/xxo5+2BP/cMlayPOpgn21GtvFF5f8LYZOsaYTOFlD38gMFREjgcKgdYi8oyqnht6kqqOA8aBm4ff2It2adcMv9btqbfI99U7/bKhG4XbJibGmEziWQ5fVa9T1S6q2g04C5hRO9h7IXQQtnm+r3rP2q3l/phm0zRko3CboWOMySRZNw8fCGTiNfCNsnlHJRu3V9RZgdvY2TQ2Q8cYk0mSstJWVd8D3kvGtYJplrJKhcASrOtfXkDLAh/+qiryfEJhri/mHH008eb9jTEmFbKqtAKE3ysWYEuZC/4FufDgsIMSNpumoXl/Y4xJlawL+PWVNM73+WjTLD+hgdk2MTHGZIKsy+HXKGkcpn6O5diNMU1V1vXwYWeaZeHqTXz87Vqe+PC7GvvYWm/cGNMUZWXAB2ossgJh5OF7ck7/Ygv2xpgmK+tSOlC3ZHFZZRUPvrck1c0yxpiUysqAbwuijDGmrqwM+Jm2ICrcfrrGGJNoWRXwg4ETyJiSxa+UrmLg7TM4d/xsBt4+I6ZNV4wxJh5ZM2gbrmrlzNGD03pBlBVfM8YkU1b08MPtK3v1lHkAMRdCSwUbazDGJFNWBPxwgbOsUnl29vIUtSg2mTbWYIzJbFkR8Lu0a0a5v+4m5Q+8u4R1W8rSdlDUtkc0xiRTVuTwi1oWMGpQT+566+sax/N9OUyavZyH3luStjtSWfE1Y0yyZEUPH+Cc/sUU5NZ8OuV+Pw++u6RGbv+aqfPTsqefzmMNxpjskDUBv6hlAWNOq5keGTWoJ/k+GxQ1xhjIkpROUO30CFCnpIINihpjmqqs6eEHhaZHbFDUGGN2yqoefjg2KGqMMU7WB3ywHamMMQayMKUTlK5z740xJlWysocfrq5OOs29N8aYVPCshy8ihSLyiYjME5GFInKTV9cKFa6uTjrOvTfGmGTzMqVTBgxW1T5AX+C3InKIh9cDrCCZMcZE4llKR1UV2BL4MS/wpV5dL8gKkhljTHieDtqKiE9ESoGfgbdUdbaX1wMrSGaMMZF4Omirqn6gr4i0BV4Skf1VdUHoOSIyEhgJUFxcnJDr2tx7Y4ypKynTMlV1A/Ae8Nswt41T1RJVLenQoUPCrmkFyYwxpiYvZ+l0CPTsEZFmwNHAV15dzxhjTHRepnR2ByaIiA/3xjJZVad7eD1jjDFReDlLZz5wkFePb4wxpmGytrSCMcaYmizgG2NME2EB3xhjmoisDfhWLdMYY2qyapnGGNNEZF0P36plGmNMeFkX8K1apjHGhJd1Ad+qZRpjTHhZF/CtWqYxxoSXlYO2Vi3TGGPqysqAD66nb4HeGGN2yrqUjjHGmPAs4BtjTBNhAd8YY5oIC/jGGNNEWMA3xpgmQlQ11W2oJiJrgGVx3r09sDaBzckE9pyzX1N7vmDPuaH2UNWYNgRPq4DfGCIyR1VLUt2OZLLnnP2a2vMFe85espSOMcY0ERbwjTGmicimgD8u1Q1IAXvO2a+pPV+w5+yZrMnhG2OMiS6bevjGGGOiyPiALyK/FZHFIrJERK5NdXsaSkS6isi7IvKliCwUkcsCx3cRkbdE5JvA/9sFjouI3Bd4vvNF5OCQxzovcP43InJeyPF+IvJF4D73iYgk/5nWJCI+EflcRKYHfu4uIrMDbX9eRPIDxwsCPy8J3N4t5DGuCxxfLCLHhRxPu78JEWkrIlNE5KvAaz2gCbzGVwT+pheIyHMiUphtr7OIPCEiP4vIgpBjnr+uka5RL1XN2C/ABywF9gTygXlAr1S3q4HPYXfg4MD3rYCvgV7AHcC1gePXArcHvj8eeAMQ4BBgduD4LsC3gf+3C3zfLnDbJ8CAwH3eAH6XBs/7SuBZYHrg58nAWYHvHwH+HPj+L8Ajge/PAp4PfN8r8HoXAN0Dfwe+dP2bACYAIwLf5wNts/k1BjoD3wHNQl7fP2Tb6wwcDhwMLAg55vnrGuka9bY31f8QGvnLHgC8GfLzdcB1qW5XI5/TK8AxwGJg98Cx3YHFge8fBc4OOX9x4PazgUdDjj8aOLY78FXI8Rrnpeg5dgHeAQYD0wN/zGuB3NqvK/AmMCDwfW7gPKn9WgfPS8e/CaB1IPhJrePZ/Bp3BlYEglhu4HU+LhtfZ6AbNQO+569rpGvU95XpKZ3gH1XQysCxjBT4GHsQMBvoqKo/AAT+v2vgtEjPOdrxlWGOp9K9wDVAcC/KImCDqlYGfg5tY/XzCty+MXB+Q38PqbQnsAZ4MpDGGi8iLcji11hVVwF3AsuBH3Cv21yy+3UOSsbrGukaUWV6wA+Xp8zIaUci0hKYClyuqpuinRrmmMZxPCVEZAjws6rODT0c5lSt57aMeL4BubiP/Q+r6kHAVtzH8Egy/jkHcson4dIwnYAWwO/CnJpNr3N9Uv4cMz3grwS6hvzcBVidorbETUTycMF+kqq+GDj8k4jsHrh9d+DnwPFIzzna8S5hjqfKQGCoiHwP/AuX1rkXaCsiwR3YQttY/bwCt7cBfqHhv4dUWgmsVNXZgZ+n4N4AsvU1Bjga+E5V16hqBfAicCjZ/ToHJeN1jXSNqDI94H8K9AyM/OfjBnumpbhNDRIYdX8c+FJV7w65aRoQHK0/D5fbDx4fHhjxPwTYGPhI9yZwrIi0C/SujsXlOH8ANovIIYFrDQ95rKRT1etUtYuqdsO9XjNUdRjwLnBa4LTazzf4ezgtcL4Gjp8VmN3RHeiJG+BKu78JVf0RWCEi+wQOHQUsIktf44DlwCEi0jzQpuBzztrXOUQyXtdI14gulQM7CRowOR43s2UpcH2q2xNH+w/DfUybD5QGvo7H5S/fAb4J/H+XwPkCPBh4vl8AJSGPdQGwJPB1fsjxEmBB4D4PUGvwMIXP/Uh2ztLZE/cPeQnwAlAQOF4Y+HlJ4PY9Q+5/feA5LSZkVko6/k0AfYE5gdf5ZdxsjKx+jYGbgK8C7XoaN9Mmq15n4DncGEUFrkd+YTJe10jXqO/LVtoaY0wTkekpHWOMMTGygG+MMU2EBXxjjGkiLOAbY0wTYQHfGGOaCAv4Ju2ISEcReVZEvhWRuSLysYicErjtSBHZGChRsFhEPgis3g3e9+8iskpESsVVaRwa5vH/ICJrAucsEpGLGtne3UTkXyKyNPB4r4vI3nE+1ngR6RX4/n8b0y5jarOAb9JKYIHJy8AHqrqnqvbDLaoJXXH4X1U9SFX3AS4FHhCRo0Juv0dV+wKnA0+ISLi/8+cD5xwJ3CoiHWNsX26tnwV4CXhPVfdS1V7A/wIxPV5tqjpCVRcFfrSAbxLKAr5JN4OBclV9JHhAVZep6v3hTlbVUuBmYFSY274EKoH2kS6mqj/jFrXsISItxNU3/zTwCeIkqP5E8IKIvAr8p9ZDDAIqarW3VFX/KyItReQdEflMXE3z4ON1E1cXf4K4uuhTRKR54Lb3RKRERG4DmgU+hUwK3PZy4BPPQhEZWe9v0phaLOCbdNMb+KyB9/kM2Lf2QRHpj6vIuSbSHUVkT9zqzyW4FZ0zVPVXuEA+RlxVS3DleM9T1cG1HmJ/XBXIcHYAp6jqwYHHuyvwiQBgH2Ccqh4IbMLVg6+mqtcC21W1r7rSEwAXBD7xlACXikhRpOdlTDi59Z9iTOqIyIO48hPlgUAc9rRaP18hIucCm4EzNfxy8jNF5DCgDPijqv4iIsfiCrtdFTinECgOfP+Wqv7S0Obj0kWH4954OrMz1bNCVWcGvn8Gl5q6s57HuzQ4loErttUTWNfANpkmzAK+STcLgVODP6jqxSLSHleHJpKDgC9Dfr5HVesLns+rau00kACnquriGgfdJ4WtUdp7WoTbhgEdgH6qWiGuQmhh4Lbab0JRa5yIyJG4CpQDVHWbiLwX8ljGxMRSOibdzAAKReTPIceaRzpZRA4EbsAVpWqsN4FLgmkXETkohvvMAApCZ/qIyK9E5Ahcid+fA8F+ELBHyP2KRWRA4PuzgQ/DPHaFuNLZBB5rfSDY74vbIs+YBrGAb9JKIP1yMnCEiHwnIp/g9oMdHXLab4LTMnGB/lJVfScBl78FyAPmi9uU+pYY23sKcExgWuZC4O+4uuWTgBIRmYPr7X8VctcvgfNEZD5uG8CHwzz8uEBbJgH/BnID598CzIrvKZqmzKplGpNk4raynK6q+6e4KaaJsR6+McY0EdbDN8aYJsJ6+MYY00RYwDfGmCbCAr4xxjQRFvCNMaaJsIBvjDFNhAV8Y4xpIv4/rIWaiouGoNgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Bonus - unscaled the values using scaler.inverse_transform\n",
    "gdp_happiness.plot.scatter(x=\"GDP Per Capita\", y=\"Life Satisfaction\")\n",
    "x_lin = np.linspace(0,100000, 100)\n",
    "\n",
    "# This is rather tedious, find a better way?\n",
    "y_lin = yScaler.inverse_transform( lin_reg_gdp_scaled.predict(Xscaler.transform(x_lin.reshape(-1,1))) )\n",
    "plt.plot(x_lin, y_lin, \"-r\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stochasic Gradient Descent\n",
    "* Pick only one point of data at random, and use it to train (rather than whole set of data)\n",
    "* Suitable to have a large initial learning rate then smaller (simulated annealing, learning schedule)\n",
    "* Much less regular, cost will bump up and down - may be suitable to jump out of local minima\n",
    "* random m times in 1 round (called __epoch__)\n",
    "\n",
    "Note\n",
    "* Training instance must be independent (not groupped). shuffle if needed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Try it out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 278,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.00524856],\n",
       "       [0.73889146]])"
      ]
     },
     "execution_count": 278,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# t0, t1 is learning schedule hyper parameters\n",
    "def learning_schedule(t, t0=7, t1=1):\n",
    "    return t0 / (t + t1)\n",
    "\n",
    "def sgd(X,y, initial_theta, epochs=200):\n",
    "    m = X.shape[0]\n",
    "    theta = initial_theta\n",
    "    errors = []\n",
    "    for epoch in range(epochs):\n",
    "        for i in range(m):\n",
    "            idx = np.random.randint(m)\n",
    "            xi = X[idx:idx+1] # Use ranges, so we got a 2D vector instead of 1D (array)\n",
    "            yi = y[idx:idx+1]\n",
    "            gradients = find_gradient(theta, xi, yi) # same as batch, but only 1 element here\n",
    "            eta = learning_schedule(epoch * m + i)\n",
    "            theta = update_theta(theta,eta,gradients)\n",
    "\n",
    "        # Error at end of epoch\n",
    "        errors.append( np.sum((X.dot(theta) - y)**2) ) \n",
    "            \n",
    "    return theta, errors\n",
    "    \n",
    "sgd_theta, errors = sgd(X_gdp_scaled, y_gdp_scaled, [[0],[0]])\n",
    "sgd_theta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 279,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1a2b807898>]"
      ]
     },
     "execution_count": 279,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl8XOV97/HP78yifbMkL1g2XsAGh8WAIZAECBDSQCkkuWmKb2/CTRdy03Qh9zZt0jTpcm9vE9KkN71dUlrSpm1CCC0kJC0BLkkIhBhijA023vdVlrxoH8323D/OmdFIaGYkY410xPf9euml0dGM5+czo68e/c7znGPOOUREJPy86S5ARETODgW6iMgsoUAXEZklFOgiIrOEAl1EZJZQoIuIzBIKdBGRWUKBLiIySyjQRURmiWgln6ytrc0tWbKkkk8pIhJ6L774Yrdzrr3c/Soa6EuWLGH9+vWVfEoRkdAzs/0TuZ9aLiIis4QCXURkllCgi4jMEgp0EZFZQoEuIjJLKNBFRGYJBbqIyCwRikB/amsnf/3DXdNdhojIjBaKQP/h9i7+/pm9012GiMiMFopA9wyyupi1iEhJoQh0MyOTVaCLiJQSikCPeIYG6CIipYUi0NVyEREpLySBrpaLiEg54Qh0tVxERMoKR6Cr5SIiUlZIAt3IKNBFREoKTaA7B06hLiJSVNlAN7OVZrax4KPXzO4xs9Vmti7Ytt7MrpqyIs0A1EcXESmh7DVFnXPbgdUAZhYBDgOPAH8H/JFz7jEzuxW4F3j7VBTp+XlOxjk8bCqeQkQk9CbbcrkJ2O2c2w84oDHY3gQcOZuFFfKCRNeBURGR4sqO0Me4E3gguH0P8LiZ/Rn+L4a3nM3CCqnlIiJS3oRH6GYWB24HHgo2fQT4mHNuEfAx4P4ij7s76LGv7+rqOqMiI0GVWlwkIlLcZFoutwAbnHOdwdd3AQ8Htx8Cxj0o6py7zzm3xjm3pr29/cyKNLVcRETKmUygr2Wk3QJ+z/z64PaNwM6zVdRYlg/0qXoGEZHwm1AP3cxqgZuBDxds/lXgS2YWBRLA3We/PF8kmNiSVaKLiBQ1oUB3zg0CrWO2PQtcMRVFjaVZLiIi5YVipWiu5aLl/yIixYUi0COatigiUlYoAj23UlQtFxGR4kIS6EHLRQdFRUSKCkege2q5iIiUE45AV8tFRKSskAS6Wi4iIuWEI9A9rRQVESknHIEetFx0xSIRkeJCEuhaWCQiUk6oAj2bneZCRERmsJAEuv9Zs1xERIoLSaDr5FwiIuWEItAjmuUiIlJWKALd1HIRESkrFIE+clBUgS4iUkwoAl0tFxGR8kIR6Gq5iIiUF4pAV8tFRKS8UAS6Wi4iIuWFItC1sEhEpLxQBLouEi0iUl4oAn3kItEKdBGRYkIR6Do5l4hIeeEI9KBKtVxERIoLR6Cr5SIiUlbZQDezlWa2seCj18zuCb73G2a23cy2mNm9U1akadqiiEg50XJ3cM5tB1YDmFkEOAw8YmY3AHcAlzjnhs1s7lQVGcm1XJToIiJFTbblchOw2zm3H/gI8Fnn3DCAc+742S4ux3Q+dBGRsiYb6HcCDwS3VwDXmtnzZva0mV15dksbMdJDn6pnEBEJvwkHupnFgduBh4JNUaAFuBr4OPBNyw2lRz/ubjNbb2bru7q6zqjI3Dx0tVxERIqbzAj9FmCDc64z+PoQ8LDzvQBkgbaxD3LO3eecW+OcW9Pe3n5GRepsiyIi5U0m0Ncy0m4B+BZwI4CZrQDiQPfZK22E56nlIiJSzoQC3cxqgZuBhws2fwVYZmabgW8Ad7kpmige0blcRETKKjttEcA5Nwi0jtmWBP7LVBQ1ls62KCJSXihWipoWFomIlBWKQM9f4EKJLiJSVCgCXS0XEZHyQhHoarmIiJQXikBXy0VEpLxQBLpaLiIi5YUk0NVyEREpJ2SBrkQXESkmJIHuf1YPXUSkuJAEupb+i4iUE45A99RDFxEpJxSBDn7bRReJFhEpLkSBbrrAhYhICeEJdM/UchERKSE8ga6Wi4hISSEKdLVcRERKCU2gR0wtFxGRUkIT6GZaKSoiUkpoAj3imQJdRKSE0AS6Zwp0EZFSQhPoph66iEhJoQn0iKeTc4mIlBKaQFfLRUSktJAF+nRXISIyc4Un0NVyEREpKTyBrpaLiEhJZQPdzFaa2caCj14zu6fg+79tZs7M2qa0ULVcRERKipa7g3NuO7AawMwiwGHgkeDrRcDNwIEprBHwT86lKxaJiBQ32ZbLTcBu59z+4Os/B34HmPKk9cx0tkURkRImG+h3Ag8AmNntwGHn3KazXtU4PDOy2Uo8k4hIOJVtueSYWRy4HfikmdUCnwLeOYHH3Q3cDbB48eIzLNO/wIVaLiIixU1mhH4LsME51wksB5YCm8xsH9ABbDCz+WMf5Jy7zzm3xjm3pr29/cwL1QUuRERKmvAIHVhL0G5xzr0CzM19Iwj1Nc657rNaXQHNchERKW1CI/SgxXIz8PDUllOc5+mKRSIipUxohO6cGwRaS3x/ydkqqBhPF7gQESkpVCtFleciIsWFJtAjuki0iEhJoQl0XVNURKS00AS6Wi4iIqWFJtAjWlgkIlJSaAJdLRcRkdJCE+haWCQiUlpoAj3ima5YJCJSQmgCXQuLRERKC02gm1ouIiIlhSbQI6aWi4hIKaEJdM9Ty0VEpJTwBLqZAl1EpISQBfp0VyEiMnOFKNDVchERKSU8ge6p5SIiUkp4At2MbHa6qxARmblCFOhquYiIlBKaQI+o5SIiUlJoAt3MyKjlIiJSVGgC3TNwGqGLiBQVmkCP2OgLXGw4cIrTg8lprEhEZGYJTaBbwblcnHOsvW8d/7Ju/zRXJSIyc4Qm0AuvKZrMZBlOZ+kfzkxvUSIiM0hoAj3ikW+5DKf9o6NpHSUVEckLTaAXnpwrmQt0ndxFRCQvWu4OZrYSeLBg0zLgM8BC4OeAJLAb+JBz7vRUFBnUkT85Vy7QUxqhi4jklR2hO+e2O+dWO+dWA1cAg8AjwJPARc65S4AdwCenstCIR/6gqAJdROS1JttyuQnY7Zzb75x7wjmXDravAzrObmmjjWq5ZHI9dLVcRERyJhvodwIPjLP9l4DHXn85xY3bclEPXUQkb8KBbmZx4HbgoTHbPwWkga8VedzdZrbezNZ3dXWdcaERM8Bvuwyn/emKmuUiIjJiMiP0W4ANzrnO3AYzuwu4DfhFV2RdvnPuPufcGufcmvb29jMv1M9zss7lpy2m1HIREckrO8ulwFoK2i1m9i7gd4HrnXODZ7uwsbwg0bOucNqiRugiIjkTGqGbWS1wM/Bwwea/BBqAJ81so5l9eQrqy/NyLRfnNMtFRGQcExqhByPw1jHbzpuSiooobLnkZrmo5SIiMiJUK0XBb7kMp7T0X0RkrPAEejBEz2RHRuha+i8iMiI8gR60XNyoHroCXUQkJ0SBPs4sF7VcRETywhPoarmIiJQUnkAvaLnkFhblRuoiIhKiQI8UznLJLf3XwiIRkbzQBHquh54pOCiqsy2KiIwITaBbbmFRVitFRUTGE5pAjwRNdOd0CToRkfGEJtBHtVx0gQsRkdcITaBb4elzg6X/yUyWImftFRF5wwlNoI+0XEZG6ODPSxcRkRAFer7lkh09/1x9dBERX4gC3f9ceD500EwXEZGcEAX6yAUuhgtCXAdGRUR84Qv0MS2XlFaLiogAYQr0oFL/ItGZ/HadQldExBeeQB/nmqKgU+iKiOSENtBr4xFAI3QRkZwQBrq/oKg27l/fWmdcFBHxhSfQcz304ORc9VX+CF2zXEREfOEJ9IJzuQyns9RV+SN0zUMXEfGFLtDTGUcm66jLt1w0QhcRgRAFeiSoNJHypyzWBi2XlC5DJyIChCjQLRihJ4IAz43QUxqhi4gAEwh0M1tpZhsLPnrN7B4zm2NmT5rZzuBzy5QWmgv0ZDBCj+cOimqELiICEwh059x259xq59xq4ApgEHgE+ATwlHPufOCp4OspE8mP0P1AHzkoqhG6iAhMvuVyE7DbObcfuAP4arD9q8C7z2ZhY+UucDEUjNDrqzQPXUSk0GQD/U7ggeD2POfcUYDg89yzWdhYuZbL0JiDopqHLiLim3Cgm1kcuB14aDJPYGZ3m9l6M1vf1dU12fryclcsSqRGHxRNqocuIgJMboR+C7DBOdcZfN1pZgsAgs/Hx3uQc+4+59wa59ya9vb2My80aLnkpy3GNUIXESk0mUBfy0i7BeBR4K7g9l3At89WUePJT1tMjT4oqh66iIhvQoFuZrXAzcDDBZs/C9xsZjuD73327Jc3ItdyGUpplouIyHiiE7mTc24QaB2z7QT+rJeK8F4zy0Xz0EVECoVmpag3ZqVorc7lIiIySngC3RvdQ88dFE3qXC4iIkCYAn3MLJfqWATPdFBURCQnNIEeGTPLJR7xiEY8TVsUEQmEJtBtzErReNQjHvFKznL51X9az5/8+6sVqU9EZLpNaJbLTDAyy8VvsVRFPaIRK9pycc6xbs8JBobTlSpRRGRahWaEnpuHPjCcxjOIRjyi3mtH6B+4/3n+6ge7ODWYoi+RpmcoNR3liohUXGhG6IUtl1svng9ALGKj5qF39Q3zzM5uEqkMb1nuT5tXoIvIG0VoAr0q6hGLGFcva+WL718NQDRioy4SvX7fSQC2H+tj/4lBQIEuIm8coQn06liEx++5jo6WWuJRv1MU87xRl6B7fq8f6L2JdP52XyJNJuvyLRsRkdkqND10gGXt9fkwB4hFvFEtlxf2nqQhOMfLU1s789v7Ehqli8jsF6pAHysasfw89J6hFFuP9fLeyxcCcLxvOH8/tV1E5I0g5IE+0nJ5cf9JnIN3XbSAtvo4AK11/mcFuoi8EYQ60GPeyCyX3ccHAFh1TiMr5jUAcHFHEwC9Q6Pnom8/1sfGg6crWKmIyNQLdaAXznLp7h8mHvVorI7mA/2SjmbgtSP0z31vG598+JXKFisiMsVCHeixgqX/Xf3DtNdXYWasnO8H+upF/gh9bKCfGEjSVdBjFxGZDUIzbXE8sYiXX/rf3Z/M987fc9lCauMRrl42/uKi3qEUpwaTZLMuf1peEZGwC/UIPeqNzHI50T9Ma30V4M9Zv2P1QmpiEWIRe02g9wylyGSdDpaKyKwS6kD3Wy4jPfTcCD3HzGiqiY0KbudGgvzEQLJyxYqITLFQB7p/tkVHNus40Z+kLRihF2qsidFbEOj9w/7KUYCTCnQRmUXCHeieRyqdpTeRIp11+ZZLobEj9MLbJ/p1YFREZo9QB3osYqSyju4gmMe2XMAP9N5EkUDXCF1EZpGQB7p/LpeuPj+Y2ycyQh8sHKEr0EVk9gh1oOfO5XJiwB+hj9dyaawu3nI5OaCWi4jMHqEO9FjEI5XN0t1XpuUylCKbHTmJF0BNLEK3Wi4iMouEOtBz89C7+5N4Bi214wd61kF/0j+fSy7Ql7bVcVItFxGZRSYU6GbWbGb/ambbzGyrmV1jZqvNbJ2ZbTSz9WZ21VQXO1Y04pEODorOqasad9VnU00MGOmd9wyliHpGR0tNvlUjIjIbTHSE/iXge865C4BLga3AvcAfOedWA58Jvq6oWBDgR3sS47ZbwJ+HDiMj89NDKZpqYrTWV2keuojMKmUD3cwageuA+wGcc0nn3GnAAY3B3ZqAI1NVZDFL2+sA+PGubtobXntAFPzWCsCOzj7AD/am2hht9XFODiTzvXURkbCbyAh9GdAF/IOZvWRmf29mdcA9wOfN7CDwZ8Anp7DOcf3sxQt4x4Xz/EVFdeOP0M+bW09tPMLLh3oA/8RcTTUx5tTFyTp/xC4iMhtMJNCjwOXA3zjnLgMGgE8AHwE+5pxbBHyMYAQ/lpndHfTY13d1dZ2lsvP/Nve+7xIWz6nlooVN494n4hkXLWxi0yH/ghY9BYEOpVeLDqcz3PzFp/nGCwfOqL6HNxzi/mf3ntFjRUQmayKBfgg45Jx7Pvj6X/ED/i7g4WDbQ8C4B0Wdc/c559Y459a0t7e/3npfY05dnB/+9tv5lWuXFb3PpR1NbDnSSyqT5fSgH+i5876UWi36xJZOdh7v56EXDwFwvDdBIpWZcG1/+/QePv/4NoaSE38M+L9k/nndfj772Lb8ycdERMopG+jOuWPAQTNbGWy6CXgVv2d+fbDtRmDnlFQ4AeXOaX5JRzPJdJbtx/roGUrRXBPL99yPnB4q+rhvrj8IwIYDp9jT1c87vvg0//O7r06opv7hNDuO95FIZXlm58T/MslmHe/56+f49Lc28+Wnd/PQev+XyWAyzd8+vZvvbT424X9LRN5YJnqBi98AvmZmcWAP8CHg28CXzCwKJIC7p6bE1+/S4FJ0Gw+epjfhj9CXt9fTWB3l+T0nee/lHaPuv/VoL6cHUzy7q5ubV83jyVc7+bWvbaA3kebRjUf49G2rqI5F8vf/zqYjPLe7m3jE43fedQF1VVE2H+7BBcdbH9/SyTvfNH9CtW7v7OPAyUH++I438a2XDvMXT+2ksSbK//ruVo71Joh4xn0fuIKbLpx3dnaOiMwaEwp059xGYM2Yzc8CV5z1iqbAojk1tNTGeHZnN875UxkjnnHN8lZ+vLub04NJPviVFzh/bgPD6Qzffflo/rGfuW0VWw73sO1YH+c0VXOkJ8ETr3Zy+6XnADCUzPCJf3sZM6N/OM3yufV88JolbAouQn39inae2tZJOpMlGinf4Xp+zwkAbrxgLufPbWDt363j17/+EhcuaOTe913C5x/fzke/voHHfuu6/AweEecc3335KL2JFGuvXKwrcb1BhfoSdBNlZly3op1vb/RnVuYWG731vDYe39LJ5763jZcP9bDreD+pTJbfuul8lrTVUhOLsmhOLTddOI9/XrefL7x/Nf/jmxv5xgsHaKuL86aFTfxw+3EGkhm+cffV/Mm/b+Xrzx/gA1efy6ZDp1k0p4a1Vy3i6R1dPL2ja0Kj6nV7TtLRUkNHSy0dLbV8+Ppl1MejfPj65cSjHivnN3D953/A//3+Tr74/tVTut+m2rZjvfQl0qxe1EzEDDP/tZLJ6RlK8fvf2sx3Nvnv7+9sOsKKeQ3Ma6zmv75lCXVVIz/mz+3u5tGNR+hoqeHyxS0sbq1l3Z6TPLOzi73dA1xxbgvOwbGeBL9w5SKO9AzxnU1HuGHlXK5cOodEMsPl57aM+gtVZo43RKAD/PEdF7H5cA+7uwZoDk4R8JblbQA88MJB3npeK/d9YA3JdJaWMVMgP3rDeVy6qJmrl83h3Zct5K9/uJvndp9geXsdrfVVLGyu4aolc7jzqkV86pHNbDrUw6aDPVy2uJm3r5zL0rY6/uDRLbx5WSv1VVGO9SQ4eGqQK5fMGfU8zjle2HeSG1bOzW/75C0XjrrPvMZqfvHN5/KPz+3jN288nyWvY5TeM5TiWy8dZtfxfj58/TI6WmpHff/kQJJXj/TSl0hRHY+QTGdpb6hidUfz6xoB9iVS/OX3d3HfM3twzj8NcjrriJjRVl/FqnMaedeb5vPzazped8A753hmZzeZrOMt57Xy413d7OkaIOIZa86dw5vOacTzjMFkmiOnh+gfzrC0tY6m2tjret6zbTidYV/3IKcGk6yY18DxvgQ/3XuSVMZx/7N7Odab4OM/s5KW2jiffWwrrx7ppTeR5qvP7eM/v3kxbz2vjT1d/fz+tzYTi3gMjjlQ31YfZ1l7PV9//gARz6iNR/neFv94zcLmGv70sW35+7bWxXnfFR1cv6I9//rMb6qmLh7xT2fdN0zE81/L+U3VldtJgjlXuYU1a9ascevXr6/Y84114MQgn39iO5+5bRXtDVU457jmT7/Psd4Ef//BNbxjVfkRdG8ixeObj5HJOn7vkVfIOvjoDcv5+M9cQF8ixZv/91MsbK5h5/F+fv9nL+RXrl3G+n0n+fm//Qm3XDSfX7l2Gb/2Lxs41ptg7VWL+fUbz+OcpmrMjB2dfbzzz3/Eve+7hPevWVS0huO9Cd527w9Y2lrHjRfO5d2rF7JyfgOnBpI8uukIe7sHaKqJ0d3v/2Ddfuk5XLigkdp4BDMjk3U88tJh/vg7W+hNpIl6RnUswkfevpy3ntfGc7u7eXxLZ75tNFZrXZxUJktbQxWf/tlVXHt+W76dlLvE36FTQxw6NURX/zDDqQw7O/vZ091P1PPYePA0Q6kMa69azHXnt7Hx4Gmqov5pHI71JNh48DR7uge4YWU7a69azMUdTcxvrM6HRybr8MaM5p1zpDKORDpDIplhKJVhR2c/D/70AP9v63EA4hGP5JhZQ1VRj9a6OEd7ExT+KJw3t57Vi/xjL4PJNLXxKPVVUfqH0/xk9wk8zw+6TNYxr7GaixY2kck65jZUccW5LdRVRUmkMvQl0ixsrqGxJkZ/Ig0GyXSW04NJFjTXUF9VekyVSGV48tVO/ug7W+gucu6hxXNq+dKdq7lsccuo7RsOnOILT2znud0n8v+3yxc38w8fugrP4IW9Jzl4cpCrlrZywfwGPM9IprNEPP898h+vHKW+KspNF85le2cfh08NkXXw4E8P8MPtXaQnsCjvnKZqzpvXwMLmGhY2V5NIZekfTnPxwiYWNFfTl0jz4n5/0sGJgSRZBx0tNbxz1TwuWthEW10VyUyWVCZLJuvIOodz/qpG5xzVsQjD6Szbj/XS1Z9kOJWhripKQ3WUiBn7Tw7SG5zu46KFTcSjHgdPDtKbSHOsJ8He7gH2dg+QymRZNKcW5xzD6SyJVIZEKstwOkMynWVxay3L2+sB/5fXBfMbWDGvgbb6KqKeEYt61MYir2mrOufo7B1m69FeLlvcnB9MTpaZveicG9v2fu393kiBPp4/+PZmnt97kv/4zWsnPeq8/9m9/J8nd/Dob7wt389+8KcH+Mqz++jsS/DND1/DinkNAPzl93fyhSd34Jw/1fLWi+fzL+v8+e218QgGpLKOZDrLjz5+A4tba4s9LQAPrT/IPz63jx2dfaQyjsbqKL0J/wRkdfEIA8kMzbUxhlNZhoKplrGI0VjtX/AjlXFcuaSFT9+2ipbaOL/3yCs8s7M7/+9f2tHEzavmcfniFlrq4gwmM1RFPXYd7+dHO7qor47ybDDaBWisjtJUG+PUQIr+4fRr6m2ujbFiXgPpTJaV8xu488rFXBoE5ljOOf7pJ/v508e2kkj5AdxSG6O5Nk4ileFYb4LaWCT/dSKVIZHO5i8tWKgmFuFjN59PR0stP97VzXUr2rl6WSuJVIbndnfz6pFeuvuTLJ5Ty/K59VRHPXZ09rHhwGlePtRDPGLUxCMMJjP0D/u//K5Z3krE8zjWM0TU89h3YoCjPYmSr1cxDdVRBobT1FdFqQt+YVRFI1THPE4NJBkIRtKXdDTxy29bSnNtnG1He2msiXHt+W1URSO01MZKHp851pNg27FeIp5x5ZI5Z6Vd0ptIsWH/KeJRD+f8028kUhmintHeUEUm6zh8eogX959i/4lBDp8e4uRAkohnxCNe/j0JEI96LGuro63ePx+T/5qcvfMsRT0j6xxj3x4NVVGWtdexrL2eqGccPj2EZ0ZV1KM6FqEq5lEV9S80v7urn4Mnh3D4g45UZvzcrIlFcDiyWcg4N+o9ef9da854MoMCfYKyWUc664hHz+zEk8l0dsKP3dnZx4M/PcgvXLmI8+c1sKOzj3V7TrCvexDP/OmXHS01fPCaJRN+/lMDSf71xUPsPznA/MZqbrxgHqvOacwfhO0fTvP9bcc5enqI00Op/MKqCxc0ctvFC0b9EtvXPcBLB09x1dJWFjbXlH3u4XSGRzce4dCpIU4PJjk9lKKlNh4cA/CPA8xtqCIe9Wisjk36F+ZgMs3Wo31sOdLD1qO9DAxniEaMhc01DAxn6BlKUR3zf/hqYpH87dzH4jm1XNLRVJF+b28iRTzisf/EIC8fOk0q47+n6qsiHDw5RP9wmoZqfzQei3g018Y4cGKQEwNJ6qoi9CfSDCQz1FdFGU77o8PmWn+9xKI5tdx60fwJHVSfyYaSGWIRw8zYdbyfU4NJqqIeFy5oHPUaZbKOV4/0sqOzL3+fWMQj4hlecKzFC/46G05nMDMumN/AgqYa4lGPwWSavkQ6P+purI6RSGV45XAP2axjSVsdTTUxqqLeGbX0Upkse7sH2H6sz7/8ZcYfiA0k0/Qn0nhBnREPIma01ldx4YJG3nRO46jjGZOhQBcRmSUmGujh/pUvIiJ5CnQRkVlCgS4iMkso0EVEZgkFuojILKFAFxGZJRToIiKzhAJdRGSWqOjCIjPrAvaf4cPbgO6y96q8mVoXzNzaVNfkzNS6YObWNtvqOtc5V/aSbxUN9NfDzNZPZKVUpc3UumDm1qa6Jmem1gUzt7Y3al1quYiIzBIKdBGRWSJMgX7fdBdQxEytC2ZubaprcmZqXTBza3tD1hWaHrqIiJQWphG6iIiUEIpAN7N3mdl2M9tlZp+YxjoWmdkPzGyrmW0xs98Ktv+hmR02s43Bx63TUNs+M3sleP71wbY5Zvakme0MPreU+3fOck0rC/bJRjPrNbN7pmt/mdlXzOy4mW0u2DbuPjLfXwTvuZfN7PIK1/V5M9sWPPcjZtYcbF9iZkMF++7LFa6r6GtnZp8M9td2M/uZCtf1YEFN+8xsY7C9kvurWD5U7j3mnJvRH0AE2A0sA+LAJmDVNNWyALg8uN0A7ABWAX8I/PY076d9QNuYbfcCnwhufwL43DS/jseAc6drfwHXAZcDm8vtI+BW4DHAgKuB5ytc1zuBaHD7cwV1LSm83zTsr3Ffu+DnYBNQBSwNfmYjlaprzPe/AHxmGvZXsXyo2HssDCP0q4Bdzrk9zrkk8A3gjukoxDl31Dm3IbjdB2wFFk5HLRN0B/DV4PZXgXdPYy03Abudc2e6sOx1c879CDg5ZnOxfXQH8E/Otw5oNrMFlarLOfeEcy53cdZ1QMdUPPdk6yrhDuAbzrlh59xeYBf+z25F6zL/mnLvBx6YiucupUQ+VOw9FoZAXwgcLPj6EDMgRM1sCXAZ8Hyw6deDP5u+UunWRsABT5jZi2Z2d7BtnnPuKPhvNmAv75Z3AAACkElEQVTuNNSVcyejf8ime3/lFNtHM+l990v4I7mcpWb2kpk9bWbXTkM94712M2V/XQt0Oud2Fmyr+P4akw8Ve4+FIdDHu4rrtE7NMbN64N+Ae5xzvcDfAMuB1cBR/D/5Ku2tzrnLgVuAj5rZddNQw7jMLA7cDjwUbJoJ+6ucGfG+M7NPAWnga8Gmo8Bi59xlwH8Hvm5mjRUsqdhrNyP2F7CW0QOHiu+vcfKh6F3H2fa69lkYAv0QsKjg6w7gyDTVgpnF8F+srznnHgZwznU65zLOuSzwd0zRn5qlOOeOBJ+PA48ENXTm/oQLPh+vdF2BW4ANzrnOoMZp318Fiu2jaX/fmdldwG3AL7qg6Rq0NE4Et1/E71WvqFRNJV67mbC/osB7gQdz2yq9v8bLByr4HgtDoP8UON/MlgYjvTuBR6ejkKA/dz+w1Tn3xYLthX2v9wCbxz52iuuqM7OG3G38A2qb8ffTXcHd7gK+Xcm6CowaNU33/hqj2D56FPhgMBPhaqAn92dzJZjZu4DfBW53zg0WbG83s0hwexlwPrCngnUVe+0eBe40syozWxrU9UKl6gq8A9jmnDuU21DJ/VUsH6jke6wSR3/PwtHjW/GPGO8GPjWNdbwN/0+il4GNwcetwD8DrwTbHwUWVLiuZfgzDDYBW3L7CGgFngJ2Bp/nTMM+qwVOAE0F26Zlf+H/UjkKpPBHR79cbB/h/zn8V8F77hVgTYXr2oXfX829z74c3Pc/Ba/xJmAD8HMVrqvoawd8Kthf24FbKllXsP0fgf825r6V3F/F8qFi7zGtFBURmSXC0HIREZEJUKCLiMwSCnQRkVlCgS4iMkso0EVEZgkFuojILKFAFxGZJRToIiKzxP8HkVuX2olEI24AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Errors jumps around for SGD\n",
    "plt.plot(errors)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using Scikit Learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 297,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SGDRegressor(alpha=0.0001, average=False, early_stopping=False, epsilon=0.1,\n",
       "       eta0=0.01, fit_intercept=True, l1_ratio=0.15,\n",
       "       learning_rate='invscaling', loss='squared_loss', max_iter=1000,\n",
       "       n_iter=None, n_iter_no_change=5, penalty=None, power_t=0.25,\n",
       "       random_state=None, shuffle=True, tol=0.001, validation_fraction=0.1,\n",
       "       verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 297,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#.ravel ~= flatmap [ [0],[1] ] -> [0,1]\n",
    "# transform from column vector to array\n",
    "\n",
    "from sklearn.linear_model import SGDRegressor\n",
    "sgd_reg = SGDRegressor(max_iter=1000, tol=1e-3, penalty=None, eta0=0.01)\n",
    "sgd_reg.fit(X_gdp_scaled[:,1].reshape(-1,1), y_gdp_scaled.ravel())\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 298,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([-0.000493]), array([0.70833334]))"
      ]
     },
     "execution_count": 298,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sgd_reg.intercept_, sgd_reg.coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 299,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Mini batch ?? - https://github.com/ageron/handson-ml/issues/226"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}