05. Improvements to the RNN/5.06 Predicting Bitcoins price using LSTM RNN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predicting Bitcoins price using LSTM RNN\n",
    "\n",
    "As we have learned that the LSTM models are widely used for sequential datasets, that is dataset in which order matters. In this section, we will learn how can we use LSTM networks for performing time series analysis. We will learn how to predict bitcoin prices using LSTM network. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# Import Libraries\n",
    "\n",
    "First, we import the required libraries as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline \n",
    "\n",
    "\n",
    "import tensorflow as tf\n",
    "tf.logging.set_verbosity(tf.logging.ERROR)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Preparation \n",
    "\n",
    "We will see how can we prepare our dataset in a way that LSTM needs. First, we read the input dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv('data/btc.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Display a few rows of the dataset as follows: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Symbol</th>\n",
       "      <th>Open</th>\n",
       "      <th>High</th>\n",
       "      <th>Low</th>\n",
       "      <th>Close</th>\n",
       "      <th>Volume From</th>\n",
       "      <th>Volume To</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5/26/2018</td>\n",
       "      <td>BTCUSD</td>\n",
       "      <td>7459.11</td>\n",
       "      <td>7640.46</td>\n",
       "      <td>7380.00</td>\n",
       "      <td>7520.00</td>\n",
       "      <td>2722.80</td>\n",
       "      <td>2.042265e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5/25/2018</td>\n",
       "      <td>BTCUSD</td>\n",
       "      <td>7584.15</td>\n",
       "      <td>7661.85</td>\n",
       "      <td>7326.94</td>\n",
       "      <td>7459.11</td>\n",
       "      <td>8491.93</td>\n",
       "      <td>6.342069e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5/24/2018</td>\n",
       "      <td>BTCUSD</td>\n",
       "      <td>7505.00</td>\n",
       "      <td>7734.99</td>\n",
       "      <td>7269.00</td>\n",
       "      <td>7584.15</td>\n",
       "      <td>11033.72</td>\n",
       "      <td>8.293137e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5/23/2018</td>\n",
       "      <td>BTCUSD</td>\n",
       "      <td>7987.70</td>\n",
       "      <td>8030.00</td>\n",
       "      <td>7433.19</td>\n",
       "      <td>7505.00</td>\n",
       "      <td>14905.99</td>\n",
       "      <td>1.148104e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5/22/2018</td>\n",
       "      <td>BTCUSD</td>\n",
       "      <td>8393.44</td>\n",
       "      <td>8400.00</td>\n",
       "      <td>7950.00</td>\n",
       "      <td>7987.70</td>\n",
       "      <td>6589.43</td>\n",
       "      <td>5.389753e+07</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Date  Symbol     Open     High      Low    Close  Volume From  \\\n",
       "0  5/26/2018  BTCUSD  7459.11  7640.46  7380.00  7520.00      2722.80   \n",
       "1  5/25/2018  BTCUSD  7584.15  7661.85  7326.94  7459.11      8491.93   \n",
       "2  5/24/2018  BTCUSD  7505.00  7734.99  7269.00  7584.15     11033.72   \n",
       "3  5/23/2018  BTCUSD  7987.70  8030.00  7433.19  7505.00     14905.99   \n",
       "4  5/22/2018  BTCUSD  8393.44  8400.00  7950.00  7987.70      6589.43   \n",
       "\n",
       "      Volume To  \n",
       "0  2.042265e+07  \n",
       "1  6.342069e+07  \n",
       "2  8.293137e+07  \n",
       "3  1.148104e+08  \n",
       "4  5.389753e+07  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As shown in the above data frame, the column Close represents the closing price of bitcoins. We need only the column Close to make predictions, so we take that particular column alone as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "data = df['Close'].values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we standardize the data and bring it to the same scale,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "scaler = StandardScaler()\n",
    "data = scaler.fit_transform(data.reshape(-1, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot and observe the trend of how the bitcoins price changes as follows, Since we scaled the price, it is not a bigger number:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XmcXFWd9/HPr7Ze00k66XR2spEAAUKgZRc6qGwK6sjzKIOO28iMozOOzjgjo88zyuj4jM7ouKEyjoO4EJ0BR0WQEaFBdhIgkIVA9oVsnaT3rZbz/FG3KtWdXqq7q+pWdX/fr1e/UnXrVtWvbpJvnT733HPMOYeIiEx8Ab8LEBGRwlDgi4hMEgp8EZFJQoEvIjJJKPBFRCYJBb6IyCShwBcRmSQU+CIik4QCX0Rkkgj5XUCmmTNnukWLFo3puZ2dnVRVVeW2oAJS/f4p5dpB9fvN7/rXr1/f7Jyry2bfogr8RYsWsW7dujE9t6mpicbGxtwWVECq3z+lXDuofr/5Xb+Z7c52X3XpiIhMEgp8EZFJQoEvIjJJKPBFRCYJBb6IyCShwBcRmSQU+CIik4QCP4+Od/Zx74uv+V2GiAhQZBdeTTQfunMd63Yf58IlM5hZXeZ3OSIyyamFn0ebD7QBYD7XISICCvy86uqLAxB3zudKREQU+AWRSPhdgYiIAr8g1MIXkWKgwC+AeFyBLyL+U+AXgFr4IlIMFPgFEE8o8EXEfwr8AkiohS8iRUCBXwBq4YtIMVDgF4ACX0SKgQI/TxIZIa8uHREpBgr8PIlmXG0VUwtfRIqAAj9PYhlj7xMKfBEpAgr8PMkMfPXhi0gxUODnSSyjS0cXXolIMVDg50lmv70mTxORYpD3wDezoJk9b2b35vu9ikk0rha+iBSXQrTwPwZsKcD7FJX+ffhq4ouI//Ia+GY2H3gz8L18vk8xyuzD/8Ad6/q1+EVE/GAuj90NZvZfwBeBKcBfO+feMsg+NwM3A9TX15+3du3aMb1XR0cH1dXV46g2t/a2J/g/j3en73/7jZVUhIZe7LDY6h+tUq6/lGsH1e83v+tfs2bNeudcQzb75m0RczN7C3DYObfezBqH2s85dztwO0BDQ4NrbBxy12E1NTUx1ufmw8b9rfD4Y+n7F150CdOrIkPuX2z1j1Yp11/KtYPq91sp1Z/PLp1LgOvNbBewFrjCzH6Ux/crKgO7cP7pNy/TF1O3joj4J2+B75y7xTk33zm3CHgX8JBz7t35er9iM3A6hbXP7uW/1u/zqRoREY3Dz5vYIMsaarSOiPipIIHvnGsa7ITtRLLtcAdf+Z+tpE6CxwYJ90Bg6JO2IiL5phZ+jvzVf27g6w9tY/uRTmDwFn5IgS8iPlLg50hVJAjA/pbkUMzBxt0HAzrcIuIfJVCOBL3We6qffrAZMoM62iLiI0VQjqRWtUp15UQHDXwdbhHxjxIoR1LnaE8Ef3JDMKPfXn34IuInBX6OpGbETHXdp1r6377p3PQ+AVPgi4h/FPg5MnA4Zp+X/OXhYHqfoFr4IuIjBX6OpLrsU106vd40ClVlwaGeIiJSUAr8HEmNykl16fTG4gBURk7MT5fQQigi4iMFfo64dB9+MvF7o14LPyPw8zkVtYjISBT4OTLwpG1vLEEkFCAUPNFvP8hITRGRglHg50gq6OPpPvw4ZaFAvxO16tIRET8p8HMk1V2TSDg27m9l68F2ykLBfoE/2NW3IiKFkrcVryab9AVXCcdbvpFc6WretIp+F1upgS8iflILP0dSrfdERis+HDR16YhI0VDg50gqy+MZoR4MDAz8QlclInKCAj9HUq33zHVrQwGdtBWR4qHAz5FUy769J5reFggYoYwZMjUOX0T8pMDPkdRsmV198fS2UMDInD5HXToi4icFfo6kVrg63tWX3hYIGGbGo59cA2hYpoj4S4GfI6nAf2bnsfS2DXtbAKj0JlBTl46I+EmBnwOJhON4V7Lvvrmj76THU/Pgq4EvIn5S4OfANx/eNuzjqX58jdIRET8p8HPgVxteG/ZxUwtfRIqAAj8HRsrx1Fh89eGLiJ8U+AWQ6tLRKB0R8ZMCPwcWTK8AoLYqMujjOmkrIsVAgZ8D5yyYDsCHL1866OOmk7YiUgQU+DlwoLUbgPLw4Icz1cJXH76I+ClvgW9m5Wb2jJltMLNNZva5fL2Xn3Y2d7L22b0AhIL9D+ely2YC6tIRkeKQzwVQeoErnHMdZhYGHjOz+51zT+XxPQtu//Hu9O3MmTF/9icXcf7iWkDj8EWkOOQt8F2y/6LDuxv2fiZc4mVOjhbOWLA8FfagcfgiUhzy2odvZkEzewE4DPzWOfd0Pt/PD5kLnqS6bgYTDFi/1bBERArNCnEi0cymAT8H/tw5t3HAYzcDNwPU19eft3bt2jG9R0dHB9XV1eMtddTWH4rxjed7AXjfygh3bErOpXPH1VX99vvgA51cszjMDcsHH7rpV/25Usr1l3LtoPr95nf9a9asWe+ca8hm34IsYu6cazGzh4GrgY0DHrsduB2goaHBNTY2juk9mpqaGOtzx+Po+n3w/AYAFi5eBps2A5xUS/DB+5m/YCGNjacN+jp+1Z8rpVx/KdcOqt9vpVR/Pkfp1Hkte8ysAngT8HK+3s8vL+5rSd/ujsaH3C9gGpYpIv7KZwt/DvADMwuS/GL5mXPu3jy+ny86ek+E/LTK8JD7Bcw0SkdEfJW3Fr5z7kXn3Grn3NnOuTOdc7fm6738lBniN75u4ZD7JQO/EBWJiAxOV9qOU+aEaIHA0KN0zDQOX0T8pcAfp7hzzJ9eweZbrxp2v4BpWKaI+EuBP06JhKMiHKQyMvzpkGBAXToi4i8F/jjFE67flApDCahLR0R8psAfp4Rzw15hm2I6aSsiPivIhVcT1bce3saDWw5z1ryp6W3Xr5o76PBMjcMXEb8p8Mfhyw9sBfqPzvn6jasH3Vfj8EXEb+rSyYHgyD06GocvIr5T4OdANidtzdCwTBHxlQI/B7I5aasuHRHxmwI/B7Jp4Wscvoj4TYGfA1l36aiFLyI+UuDnQLZdOsp7EfGTAj8HdKWtiJQCBX4O6KStiJQCBX4OBLM4imZGPJH/WkREhqLAH6PMefCz6dIpCwVo64nmsyQRkWEp8Mdg77Euth3uSN/Ppktn1fypbNzfms+yRESGpbl0BojGE3zuV5u46YJTOH1OzaD7vP5LD/e7Xx4Ojvi6M6rL6OqL0xdLEAnpe1ZECk/JM8Dze1r40VN7+PO7ns/6OVlMpcPUiuQMmurWERG/KPAH2N/SBSTnvXHOsX738RGnNT7S0Tvi66YCv7VbgS8i/lDgD9AbTQ6liSYSPLH9KO/49hN89bevDPucKeUnz38/kAJfRPymwB+gJxoHIBpz9MWS4f/1h7YNuf8bT6/n1utXjvi6NQp8EfGZAn+AHi/k++KJfkMvY0MMor/x/AVMr4qM+LrpPnwFvoj4RIE/QKqFf6yzr1/ffDSeDP+Bc9qXhUYeoQMwszr5pbCruSsXZYqIjNqoAt/MKvNVSLHoiZ5oyb/W0p2+3Rc/0befyZHddAnTKiMsmlHJ1kNtOahSRGT0sgp8M7vYzDYDL3v3V5nZbXmtzCe9sXj69rHOvvTtVH9+zGvpX7dqLqfPqRlyrP5gKiKh9G8KIiKFlm0L/6vAVcBRAOfcBuCyfBXlp8wW/tM7j6VvR+P9A3/1gmnc/7HXM7O6LOvXjgQt/cUhIlJoWXfpOOf2DtgUH3THEtcbPfGxMqdPSAV+qmsnnM3K5QOEg4H064iIFFq2gb/XzC4GnJmFzeyvgS3DPcHMFpjZw2a22cw2mdnHxl1tAfTEBv8eS3fpeH34oWymyBwgElLgi4h/sk2tPwU+AswD9gPnePeHEwP+yjl3BnAh8BEzO2OshRZKb3TwQO4b0KUTHkPgh4MB+tSHLyI+yWryNOdcM3DTaF7YOXcAOODdbjezLSS/MDaPtshC6onFOe+U6azffbzf9r5Ybrp01IcvIn7JdpTOD8xsWsb96Wb2/WzfxMwWAauBp0dbYKH1RBOUhwO8s2FBv+3Hu5IjdtbtSp7IHcviVZGQqUtHRHxjI00MBmBmzzvnVo+0bYjnVgOPAF9wzt0zyOM3AzcD1NfXn7d27dpsa++no6OD6urqMT03098/0c20MmNGhfHQnlh6+ztXRLhmcZgPP9hJdwyuWxLmHctHvsI203c29PDUgTj/fmXlSYum5Kp+v5Ry/aVcO6h+v/ld/5o1a9Y75xqy2Tfb+fADZjbdOXccwMxqs3mumYWBu4EfDxb2AM6524HbARoaGlxjY2OWJfXX1NTEWJ+bKbS+iXmzpzCtMgJ79gBgBvXzFtLYuIL39bzMt5u285l3XUZ9TfmoXvuLzz8KtNM2/VTevnp+Xur3SynXX8q1g+r3WynVn+2Zx38BnjSzfzCzzwNPAF8a7glmZsC/A1ucc18ZX5mj99lfbuJ/Nh0c9fN6YwnKQ8F+c+dUhIN09SVH70S8k7WzpmQ//j5dkzfJWuZwTxGRQskq8J1zdwJ/ABwCDgJ/4Jz74QhPuwR4D3CFmb3g/Vw7rmqzFI0nuOOJXdz8w/X0xRLsOJJ9wPZEE5SFg8S8OXM+smYplZEgXalZNOMJQgHDsljWcKCLls5g3rQKDrT2jPq5IiLjNWzgm1mN92ctyaD/ifdz0Ns2JOfcY845c86d7Zw7x/u5L1eFD+dgRqD+2+93cMW/PMLTO46O+Lx4wtHdF6M8HKC9J9l/37ColopIkG6vhR9LuDENyUyZWhHWjJki4ouR+uF/ArwFWA/9Zgkz7/6SPNU1Lpkt6Ce3J4N+25EOLlgyY9jnnff539LZF6csFORvr17BnKnlvH7ZTCrDIbr6kl8AfbEEoTEMyUypqQjR1h0beUcRkRwbNvCdc2/x+uIvd87tKVBN43ag9cQsl49ta876eS1dyZZ3eTjAsllTuPWtZ6bvd0dPXGkbGWcLX1Mki4gfRkwulxy3+esC1JIzn/nvjSdtG824+fJw/znuI6EAUe+Cqfae2Lha+FVlITr71MIXkcLLtqn6nJm9Lq+V5FCq/z3TaK6TWlE/pd/91KRnu5o7+cULr3GobeRFy4dSET5xPkBEpJCyHYd/AfBuM9sFdOL14Tvnzs5XYWM11qkLNr3Wmr69cEb/dV4ioQCdvTF2NI9/OGVlJEh3VIEvIoWXbeBfldcqcig1BcKtb13J//3FphMPjNCn09xxYrGT6rL+hyUcDKTH549XRTgZ+M65MQ3tFBEZq5GGZZab2V8CnwSuBvY753anfgpS4SilunOmVoTZcuvV6e3/5xebGGwaic/+chOLPvVrnsg4uVs1IPCDZrx8sJ1IaPxLAJdHgjiXvMBLRKSQRkqwHwANwEvANSSvuC1qqUXIK8JBKiL9W+Q7mztP2v+OJ3YB8N1Hd6S3VQ44afsb74rdR185Mu76KrzX7lG3jogU2EiBf4Zz7t3Oue8CNwCvL0BN45KaTiEV9tv/8VoW1ib75Dt7Rw7ZJXVVBAKDd7WkhnhecdqsMdeXCnz144tIoY0U+OlLQp1zRT+W0DnH1x/aBpwI1mDA+OIfnAWQvngq09SKcL/7P3j/+UO+/nN7WgD4yJplY64x9UXUpZE6IlJgI520XWVmbd5tAyq8+6lROjV5rW4UNjbHuP17J6bbzxxLXz5Mq7oyEqQ1Y6qDbPrpx7L4yUm1KPBFpMCGTTfnXNA5V+P9THHOhTJuF03YA/zzul6e2H5ivpzM+W4qvVb1ay09/U7cHm7rOWkis8Guov3SDf1Hn4YCYz95m6pFffgiUmjjH3ZShC5bXscpGWPpU907f/fzl/jx0ydmiDjcnryAav70ivS28CAt/HMXTut3fzwt/FQte45pegURKawJGfh3fuD8fl06U8pP9Fw9/PLh9O3U3DmZXw6DtfAjwf6jdsYzW2aqrk/8bAMPbj405tcRERmtCRP4VeGhH5tRfWKxkswROC3dyYutZtdktPCzaL2PZy6d2qoTyyI+u/vYmF9HRGS0Jkzge+uVEBpiSGVKMOPq1lQLv9b7tnj/JYsGvfp14Eie8bTwM5dFrAiP/8pdEZFsTYjAd87RE4P/dd58nvq7Nwy77282HSThfTukZtWsrUr+BpDaPtDUyjDfuPHEeu0jfakMJxgwvvrOVQAczZjOQUQk3yZE4Hf1xXHAslnVzKwefK3ZP750cfp2ar6dlBWzkyvOL5019MrzdRlr2IbG0cIH0guY//Cp3ezVyVsRKZAJEfgdvckLqqrLh76s4DNvOSN9AVY03r8lf8Vp9dz94Yt4z4WnDPn8yoxpGsazAMpAo1mgRURkPCZE4KfG0g+c5XKgVN97NJ6cuKy2KsK7L1wIwHmn1A47e2Vmf/t4TtqmfPra0wGYXhkZYU8Rkdwo+cDv7I3xtm89Dox89WpqBE6fF/h9scRJQy6HMrXyxInb8fThp1xxenI+nt5YnCPtY19QRUQkW9nOh1+0MrtaXr+8bth9y7yLqg619fC1B1+lozeW9ZTHs6acGF2Ti3nsU78xfP+xnXxs7Qusqgvy+sscwRx8mYiIDKbkW/hmxqOfXMMXL61g3rSKYfdNdenc9vB2frnhNSC7uXPyIRX4G/YlV9racCQ+6PTNIiK5UvKBD8klCedUZzPpWXKfzAZ6mU+BP3ChdIDmDnXtiEj+TIjAz1Yq8DMvnMrliJvRGOyLRn35IpJPkyrwU903D2XMp1Me9ucQBAJ20nurhS8i+TS5An+Q1nzmBVWFNnBqBbXwRSSfJlXgz5t+8kndmvJhZl0b4A8vWMjSuqqc1ZMK/FDACJpa+CKSXyU/LHM0MmeqhGTgXrBkRtbP/8e3n5XTesq9IaUr59bQ0tZOs+bWEZE8mlQtfIAHP3FZ+vaWf7ja13Hv5aFk4FeVhZgaMXXpiEhe5S3wzez7ZnbYzDbm6z3GYmldNZctr+O2m871u5T0+YPqshBTy0xdOiKSV/ls4d8BXJ3H1x8TM+POD5zPtWfN8bsUVs5NLgs8q6aMmkgy8DPX3BURyaW8Bb5z7lFASzoNIzUnT21lhKllRjTuaO2O+lyViExUls8WpZktAu51zp05zD43AzcD1NfXn7d27doxvVdHRwfV1UPPZ1+Mfrq1j/t3RrlheZhq6+OOrcYXLq1gXhZXDRebUjz+KaVcO6h+v/ld/5o1a9Y75xqy2df3UTrOuduB2wEaGhpcY2PjmF6nqamJsT7XL7/v2Aw7d7J82VLiR3YBPRwum89Njcv9Lm3USvH4p5Ry7aD6/VZK9ZdeU3ICiXtLKgYDARbWJP8q9h7r4lsPb2P7kQ4/SxORCcj3Fv5kFksk5+UPBYyqsHHKjErueX4/AI9va+YnH7rQz/JEZILJ57DMu4AngRVmts/MPpiv9ypVZ8yZCiTX4gXYffTE+rb7W7r56bN7NGpHRHImby1859yN+XrtieLG8xeweuE0Tp9TQ9O+/o/tPtrF3979EqfWT+HchdP9KVBEJhT14fvIzDh9Tk36/nffc95J+zTr6lsRyREFfhG5auXsk7btb+n2oRIRmYh00rbIXLdqLkGDN5xez1/+9AX2H1fgi0huKPCLzDduXJ2+/aUHXuZop2bQFJHcUJdOEasIB+mJxv0uQ0QmCAV+ESsPB+lW4ItIjijwi1h5OEh3nwJfRHJDgV/EKsJBemKJYfdJJBxfvH8LOzQVg4iMQIFfxMrDAXpGaOHvb+nmu4/s4I/vXFegqkSkVCnwi9jUijBbD7Wz7XD7kPu09STnzz/Spgu0RGR4Cvwidt2quQA88krzoI8f7ejl+4/tAqC9N1aoskSkRCnwi9ily2ZSXRZi77GuQR//1wdf5e7nTkzCc6itp1CliUgJUuAXMTNjYW0lu4920jFIC/7VAV09mkNfRIajwC9yy2ZV8/DWI5z59w/w7K7+SwSnhmy+96JTAPjkf77IruZOGr/8MM/vOV7wWkWkuCnwi9yVK+vTt3/94oH07cPtPWzY18q7L1zI31+3klNnVbO/pZvrv/kYu4528fbbnvCjXBEpYgr8InfFabPSt+94Yhd7jnbRF0vwH4/vwgzed/FiAgHjV39+KWbQ1qOTtyIyOAV+kauMhHjm02+gcUUdAI++eoTln7mfbzdtZ/WCaenVssrDQWZUlfV7rlbLEpFMCvwSMGtKOZ958xkA3JMxKmdmdf+Av+bM5Hz6552SXCGrpStaoApFpBRoeuQSUVsVAeC5PS0nbUv51DWncd2quRxp72X97uMcbOth+oB9RGTyUgu/RAwMd4Ap5f2/r6vKQpy/uJbZU5Mt/4OtGpcvIico8EtYT3TwidVmT60A4KAuxBKRDAr8EvK1d53T7340Pnjgz5pShhkcUAtfRDKoD7+EvPWceZw2u4ZHXznCF+7bQmKIUTjhYIC66jKe3Xls0MdFZHJSC7/ErJg9hamVYQCGaOADcLi9lyd3HO13sZaITG4K/BIUMAOGH2f/4calAHzvsR0FqUlEip8CvwRdvHQGAH94wcIh9/nbq08D4PmMYZwiMrkp8EvQ3GkV7Pp/b6ZhUe2w+6Xm0xcRAQX+hHba7CkA9ESzXwj9oZcP0fD53w46HbOIlDYF/gSWujDr3x/biXOObz28jd9sHP4k7u2P7qC5o48rv/KIQl9kgslr4JvZ1Wa21cy2mdmn8vlecrI53gVYX35gK4tvuY8vP7CVP/3Rc4Pu2xON09LVxym1VQC81trDD57YVahSRaQA8jYO38yCwLeANwH7gGfN7JfOuc35ek/pL3VydyiJhCMQMLYf6eAN//LISY+nFlgRkYkhny3884Ftzrkdzrk+YC3w1jy+nwxQVRbiGzeuJmAnP/aJn73Akr+7j8/+ctOgYQ/JqZhFZOLIZ+DPA/Zm3N/nbZMCum7VXH72Jxel7y+eWcXh9h7ueW4/kFxUZaDPXb8SgBf3tXK4vYc9R7tIJDS3vkips3wtkmFmNwBXO+f+2Lv/HuAC59xHB+x3M3AzQH19/Xlr164d0/t1dHRQXV09vqJ9lM/6nXPsbE3w8N4YG5vj1FUarxxPMKfKONCZ/PtfMCVAT8xxpNvxF6uTc/F87bleXj8vxO/3x/ijMyJcsTDsS/35Vsq1g+r3m9/1r1mzZr1zriGbffM5l85+YEHG/fnetn6cc7cDtwM0NDS4xsbGMb1ZU1MTY31uMch3/WuAX3zrcY73tnC8Nxny9/zFGnY2d3K4rZdLls2kPBzgd1sOc92quQQDxtee+zW/358cqdNVMYvGxlW+1Z9PpVw7qH6/lVL9+Qz8Z4FTzWwxyaB/F/CHeXw/GcFlp85kw97klbd/+cZTmTO1Ij2SJ+VtqwfvdevUEE2Rkpe3PnznXAz4KPAAsAX4mXNuU77eT0b28TcuT99+z4WnjLj/Tz50Qfr2/RsPatSOSInL6zh859x9zrnlzrmlzrkv5PO9ZGSBjOE60ytHXvrw4qUz+dd3nsM7zp0PwN0Z6+mKSOnRlbaTVGCwsZqDeNvqefzz/zqb2TXlNG09zIHWbuIasSNSkrQAyiRz5wfOP2kt3JGYGW86o54fPrWbB7c8xLVnzea2m87LU4Uiki9q4U8yly2vY/XC6aN+3nsvXpS+fd9LBzms9XJFSo4CX7KybFY1j3yykcUzk3Pt3HLPS7oYS6TEKPAla6fMqOKhv7qc8xfX8ruXD/OTZ/b4XZKIjIICX0bFzPjRBy9gSV0V33joVV5r6fa7JBHJkgJfRi0SCvCFt51Fc0cfb/vW46zbdYzEMFN09ETjPPTyIQ6393DLPS/y+Xs3ExtuBXYRyQuN0pExuWjpDH7w/vP56F3PccN3nuSUmgB/9+TvCAaMC5fM4LPXn8GU8uTcOz96ajef//WWfs//3mM7+ZurV/Bnjcuyej/nHM5lP5xURE6mFr6M2aWnzqTprxu59a0rae9zdPbG2N/Szd3P7ePttz3BS/ta+fTPXzop7D/2hlMB+NJvtvKl37xMTzTOhr0t7DnaxVCT+f35Xc9z3ud/m/fPJDKRqYUv4zKtMsIfXbSIhb27aGxspKM3xj/8ajM/XbeX6775WHq/8xfXsmRmFR+6bAlL66p55+sWcPMP13Fb03Zua9qe3m/Nijq++s5zmDbgSuB7X0wuzXjLPS/ywUsXs2zWlMJ8QJEJRIEvOVVdFuKfbjibt62ex43/9lR6+4dev4Q3nVGfvj93WgX//WeXJEf7PL2HZ3cdY2ldNU2vHOGcW0+05D/xpuU8vfNo+v5dz+zlrmf28tE1y/jrq1YU5kOJTBAKfMmLi5bO4MlbrmBncycXLp4xaN97KBjgqpWzuWrl7PS2+186wG82HWTj/la2H+nkK799BYBgwPjVRy9ly4E2/uo/N/DNh7dxqK2HBbWVzKwuI2CwpK6apXVVHOvsIxQMsGhGJWaGc45YwhEO+teD2bT1MJ29ca49azZmOg8h/lDgS94MNv3ySK45aw7XnDUHgNdauln7zB72tXRz61vPpLosxBlza7hyZT1/cdfz3L/xIB3DTNu8sLaSeMKx3xs6evHSGbz34kVctHQGsbgjYJzUdTRard1RNu5vpS+e4GBrD73ROKFggHW7jnGsK8oVK+p4aX9beuK5y5fX8R/ve51OPosvFPhStOZOq+ATV57cbTOlPMx/vP98EglHbyxBc0cvx7v6ePlAOx29MSojQfYd72bDvhbKQgGqy0JsPdTOE9uP8sT2o/1ea87UchLRXqY89wihgBEMGKFggLbuKPGEY9msal63qJayUIDW7ihTykO09cQ42tHLsc4+7t94cNjP8Ogr/dcFfuSVI3z2V5tYvXAaV62cTWVE/wWlcPSvTUpWIGBURIIsqK1kQW0lZ8+fNuz+xzr7eHDLIY6091IZCbKruZP2nhivHTzIjJlTiMYTxBPJ7p+pFWEqwgGe2nGMh14+3O91zJLTSx/r7APgu+85jynlIeZNq6AyEuLVQ+0c6ejl3IXTeWbnMX7+/H5WL5zGn16+lOu++Rh3PrmbO5/cDWzgzWfP4eNvXE5rd5TeWJyeaJxQIEBVWYiDrT3U15Rxav0UaspDWXcFxRMOA3qAiLsOAAALcElEQVRjCSoiwWH37YnG2X6kgwW1ldSUh+mJxtlxpJPuaJzeaJwlddXMnlqe1ftK8VPgy6RRWxXhfzcsOGl7com6cwd9TjSeoLM3hnNQUxGmoydGKGhUlYVYt+sY63cf73cOAqBuSln69oLaSt5x3vz0/d994nI6++I8vq2Zf3t0B79+8QC/9kYgDScYMKZXhqmMhDCDoBmhoFEWCtLS1s3MzY9TFQmxv6Wbnc2dhAJG3DlWzZ/GRUtnMLO6jM7eGMGA8eqhdja+1saeo130ZXEBXOOKOtasmEVNRYjLl89iemWYhIMtB9q4f+MBjnVG+fibTmXWFH0xDKa9J0pf7ERjIhZ3xBIJjnf1seVAO+GgUV0W5s1nz8l7LQp8kWGEg4F+/fxTK08s5N6wqJaGRbWjej0zo7osxFUrZ/PG0+t5asdRDrf3UB4Kpl87EgzQ0RujN5YgaMbO5k5auvs41hmlJxon4RzxhCMaT9AbSxCKJl+zrTvK7JpyrjlzNm09Ubr64mx+rY1vZwx7BSgPB+iJJoN+YW0lH7hkEce7oiSc40h7L7VVERbNrOJIey/3bzzAi/taadp6JF1bRSRIa3e032ve9cwe3tmwgBWzp1A3pYzFM6uYXhWhuixEJBggFDRCAUv/lhKNJ/rdB2jtirL5QBtHOno53NZDR2+MTu841FWX0RtL8OrhdlbMrmF5fTW1lRHmTqugsixIbzTB1MowNeXh9EV6CecIeudK4glHwiV/O0s4RyQY6PfeXX0xDrX10tkbY2pFmN5Ygj3HOtl6sIOWrj66+uLMmVbO9MoIr1s0nfnTKwE43tWXfg3nHE/uOMrSumrqa8p5fs9xvnj/yzyz89iI/y7qppQp8EUmsmDAuGTZzHG/TvI3lAuGfLy7L05nX4x4wlEeClJTEcI56PNCNzTM6KWPrFlGTzTOK4faOd4V5b4XDxAOGVMrwrx6qIM/bVxKc3sv335kO/e9dICfrts7bK2p8yS9sQTBgFEZDtITjVHR9ABtPSefgC8PBwgFkl+AZuAcPLDp0JCvn9pnJNVlofTorWmVYQ619Q67/5TyEO2D1Acwo9xIPPo/HO+KpmuYURWhuSP5ZbC8vpqbLjgl/aUXDAQIBYzycIAz500Fsqs5FxT4IhNcRSR4Ul++GZQHhu/fTykPB9PnRy5fXjfoPleunI1zjqOdfew/3s3+lm7auqN09MaIxh2xeIJoIvlnPOGoKgvRG4vT1Rfntf37qJ8zj3nTKjhtTg2za8oJBozFM6sIBpLDajt6Y1RFQgQCxoHWbprb+zje1ceB1m6i8WSL/WBbD7F4AjMjYJZuzScSjlAwQDBg9HlfNMc6+7zXhpbuPmZNKWdJXRU15WGOd/URDgaYWR1hZnUZK+fWYGa0dkdp7uhl/a7jNHcmfxvYuL+NaMdxZs2qIxIKsKSumt1HO7nrmeQX3/Wr5vL1G1eP428vtxT4IpITZsbM6jJmVpexasHwJ9AzNTUdobHxzGFfNzUvE4xtuG8uTK0IM7UizNK66n7bk79h9Q/1z11/Jl/57Su89+JTClniiBT4IiI5FgkF+NQ1p/ldxkk0eZqIyCShwBcRmSQU+CIik4QCX0RkklDgi4hMEgp8EZFJQoEvIjJJKPBFRCYJG2rRaD+Y2RFg9xifPhNozmE5hab6/VPKtYPq95vf9Z/inBt8zosBiirwx8PM1jnnGvyuY6xUv39KuXZQ/X4rpfrVpSMiMkko8EVEJomJFPi3+13AOKl+/5Ry7aD6/VYy9U+YPnwRERneRGrhi4jIMEo+8M3sajPbambbzOxTftczGDNbYGYPm9lmM9tkZh/zttea2W/N7FXvz+nedjOzr3uf6UUzG3yF7QIzs6CZPW9m93r3F5vZ016dPzWziLe9zLu/zXt8kZ91ezVNM7P/MrOXzWyLmV1UKsffzD7u/bvZaGZ3mVl5sR97M/u+mR02s40Z20Z9vM3svd7+r5rZe32s/cvev50XzeznZjYt47FbvNq3mtlVGduLL5uSC/6W5g8QBLYDS4AIsAE4w++6BqlzDnCud3sK8ApwBvAl4FPe9k8B/+Tdvha4HzDgQuBpvz+DV9cngJ8A93r3fwa8y7v9HeDD3u0/A77j3X4X8NMiqP0HwB97tyPAtFI4/sA8YCdQkXHM31fsxx64DDgX2JixbVTHG6gFdnh/TvduT/ep9iuBkHf7nzJqP8PLnTJgsZdHwWLNJl/fPAd/MRcBD2TcvwW4xe+6sqj7F8CbgK3AHG/bHGCrd/u7wI0Z+6f387Hm+cDvgCuAe73/nM0Z/wnSfxfAA8BF3u2Qt5/5WPtULzRtwPaiP/5e4O/1Qi/kHfurSuHYA4sGhOaojjdwI/DdjO399itk7QMeezvwY+92v8xJHf9izaZS79JJ/WdI2edtK1rer9irgaeBeufcAe+hg0C9d7sYP9e/An8DJLz7M4AW51zMu59ZY7p+7/FWb3+/LAaOAP/hdUl9z8yqKIHj75zbD/wzsAc4QPJYrqd0jn2m0R7vovl7GOADJH8jgRKrvdQDv6SYWTVwN/CXzrm2zMdcshlQlEOmzOwtwGHn3Hq/axmjEMlf0b/tnFsNdJLsUkgr1uPv9XO/leSX1lygCrja16JyoFiP90jM7NNADPix37WMRakH/n5gQcb9+d62omNmYZJh/2Pn3D3e5kNmNsd7fA5w2NtebJ/rEuB6M9sFrCXZrfM1YJqZhbx9MmtM1+89PhU4WsiCB9gH7HPOPe3d/y+SXwClcPzfCOx0zh1xzkWBe0j+fZTKsc802uNdTH8PmNn7gLcAN3lfWFAitaeUeuA/C5zqjViIkDxJ9UufazqJmRnw78AW59xXMh76JZAaefBekn37qe1/5I1euBBozfhVuOCcc7c45+Y75xaRPMYPOeduAh4GbvB2G1h/6nPd4O3vW2vOOXcQ2GtmK7xNbwA2UxrHfw9woZlVev+OUrWXxLEfYLTH+wHgSjOb7v2mc6W3reDM7GqSXZrXO+e6Mh76JfAub3TUYuBU4BmKNZv8PomQg5Mr15Ic9bId+LTf9QxR46Ukf319EXjB+7mWZN/q74BXgQeBWm9/A77lfaaXgAa/P0PGZ2nkxCidJST/cW8D/hMo87aXe/e3eY8vKYK6zwHWeX8H/01y1EdJHH/gc8DLwEbghyRHhBT1sQfuInnOIUryN6wPjuV4k+wv3+b9vN/H2reR7JNP/f/9Tsb+n/Zq3wpck7G96LJJV9qKiEwSpd6lIyIiWVLgi4hMEgp8EZFJQoEvIjJJKPBFRCaJ0Mi7iEw8ZhYnOQQwTPLKyTuBrzrnEsM+UaSEKfBlsup2zp0DYGazSM4CWgP8va9VieSRunRk0nPOHQZuBj7qXe25yMx+b2bPeT8XA5jZnWb2ttTzzOzHZvZWM1tpZs+Y2QvefOmn+vVZRIajC69kUjKzDudc9YBtLcAKoB1IOOd6vPC+yznXYGaXAx93zr3NzKaSvOLyVOCrwFPOuR97l9EHnXPdhf1EIiNTl47IycLAN83sHCAOLAdwzj1iZreZWR3wDuBu51zMzJ4EPm1m84F7nHOv+la5yDDUpSMCmNkSkuF+GPg4cAhYBTSQXLEo5U7g3cD7ge8DOOd+AlwPdAP3mdkVhatcJHtq4cuk57XYvwN80znnvO6afc65hLeOajBj9ztITkp20Dm32Xv+EmCHc+7rZrYQOBt4qKAfQiQLCnyZrCrM7AVODMv8IZCauvo24G4z+yPgNyQXTAHAOXfIzLaQnHEz5X8D7zGzKMmVnP6xAPWLjJpO2oqMgplVkhy/f65zrtXvekRGQ334IlkyszcCW4BvKOylFKmFLyIySaiFLyIySSjwRUQmCQW+iMgkocAXEZkkFPgiIpOEAl9EZJL4/+v9xd6q5+MTAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(data)\n",
    "plt.xlabel('Days')\n",
    "plt.ylabel('Price')\n",
    "plt.grid()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We define a function called get_data which generates the input and output. It takes the data and window_size as an input and generates the input and target column.\n",
    "\n",
    "What is window size? We move the x values window_size times ahead and get the y values. For instance, as shown in the below table with window_size = 1, Y values are just 1-time step ahead of x values.\n",
    "\n",
    "\n",
    "![image to be added](images/9.png)\n",
    "\n",
    "The get_data() function is defined as follows: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def get_data(data, window_size):\n",
    "    X = []\n",
    "    y = []\n",
    "    \n",
    "    i = 0\n",
    "    \n",
    "    while (i + window_size) <= len(data) - 1:\n",
    "        X.append(data[i:i+window_size])\n",
    "        y.append(data[i+window_size])\n",
    "        \n",
    "        i += 1\n",
    "    assert len(X) ==  len(y)\n",
    "    return X, y\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We choose window size as 7 and generate the input and output:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X, y = get_data(data, window_size = 7)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consider the first 1000 points as a train set and the rest of the points in the dataset as the test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#train set\n",
    "X_train  = np.array(X[:1000])\n",
    "y_train = np.array(y[:1000])\n",
    "\n",
    "#test set\n",
    "X_test = np.array(X[1000:])\n",
    "y_test = np.array(y[1000:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The shape of X_train is shown as follows:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1000, 7, 1)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "What does the preceding shape mean? It implies that (sample_size, time_steps, features). LSTM requires input exactly in this format, for example:\n",
    "\n",
    "* 1000 implies the number of data points (sample_size)\n",
    "\n",
    "* 7 specifies the window size (time_steps) \n",
    "\n",
    "* 1 specifies the dimension of our dataset (features) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining Network Parameters\n",
    "\n",
    "Define the network parameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "batch_size = 7\n",
    "window_size = 7 \n",
    "hidden_layer = 256 \n",
    "learning_rate = 0.001 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining Placeholders\n",
    "\n",
    "Define the placeholders for our input and output:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "input = tf.placeholder(tf.float32, [batch_size, window_size, 1])\n",
    "target = tf.placeholder(tf.float32, [batch_size, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining weights\n",
    "\n",
    "Let's define all the weights we use in our LSTM cell.\n",
    "\n",
    "Weights of the input gate:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "U_i = tf.Variable(tf.truncated_normal([1, hidden_layer], stddev=0.05))\n",
    "W_i = tf.Variable(tf.truncated_normal([hidden_layer, hidden_layer], stddev=0.05))\n",
    "b_i = tf.Variable(tf.zeros([hidden_layer]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Weights of the forget gate:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "U_f = tf.Variable(tf.truncated_normal([1, hidden_layer], stddev=0.05))\n",
    "W_f = tf.Variable(tf.truncated_normal([hidden_layer, hidden_layer], stddev=0.05))\n",
    "b_f = tf.Variable(tf.zeros([hidden_layer]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Weights of the output gate:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "U_o = tf.Variable(tf.truncated_normal([1, hidden_layer], stddev=0.05))\n",
    "W_o = tf.Variable(tf.truncated_normal([hidden_layer, hidden_layer], stddev=0.05))\n",
    "b_o = tf.Variable(tf.zeros([hidden_layer]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Weights of the candidate state:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "U_g = tf.Variable(tf.truncated_normal([1, hidden_layer], stddev=0.05))\n",
    "W_g = tf.Variable(tf.truncated_normal([hidden_layer, hidden_layer], stddev=0.05))\n",
    "b_g = tf.Variable(tf.zeros([hidden_layer]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Output layer weight:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "V = tf.Variable(tf.truncated_normal([hidden_layer, 1], stddev=0.05))\n",
    "b_v = tf.Variable(tf.zeros([1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define the LSTM cell\n",
    "\n",
    "\n",
    "\n",
    "We define the function called LSTM_cell, which returns the cell state and hidden state as an output. Recall the steps we saw in the forward propagation of LSTM:\n",
    "\n",
    " $$ i_t = \\sigma ( U_i x_t + W_f h_{t-1} + b_i)  $$\n",
    "\n",
    "$$ f_t = \\sigma ( U_f x_t + W_f h_{t-1} + b_f)  $$\n",
    "\n",
    "$$  o_t = \\sigma (U_o x_t + W_o h_{t-1} + b_o)  $$\n",
    "\n",
    "$$ g_t  = tanh (U_g x_t + W_g h_{t-1} + b_g)  $$\n",
    "\n",
    "$$ c_t = f_t  c_{t-1} + i_t {g_t}  $$\n",
    "\n",
    "$$ h_t = o_t tanh(c_t)  $$\n",
    "\n",
    "The LSTM cell is implemented as shown in the following code. It takes the input, previous hidden state, and previous cell state as inputs, and returns the current cell state and current hidden state."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def LSTM_cell(input, prev_hidden_state, prev_cell_state):\n",
    "\n",
    "\n",
    "    it = tf.sigmoid(tf.matmul(input, U_i) + tf.matmul(prev_hidden_state, W_i) + b_i)\n",
    "\n",
    "    ft = tf.sigmoid(tf.matmul(input, U_f) + tf.matmul(prev_hidden_state, W_f) + b_f)\n",
    "\n",
    "    ot = tf.sigmoid(tf.matmul(input, U_o) + tf.matmul(prev_hidden_state, W_o) + b_o)\n",
    "\n",
    "    gt = tf.tanh(tf.matmul(input, U_g) + tf.matmul(prev_hidden_state, W_g) + b_g)\n",
    "\n",
    "    ct = (prev_cell_state * ft) + (it * gt)\n",
    "\n",
    "    ht = ot * tf.tanh(ct)\n",
    "\n",
    "    return ct, ht"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining forward propagation\n",
    "\n",
    "\n",
    "Now, we will perform forward propagation and predict the output, $\\hat{y}_t$. \n",
    "\n",
    "$$ \\hat{y}_t = Vh_t + b_v$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#initialize the list called y_hat for storing the predicted output\n",
    "y_hat = []\n",
    "\n",
    "#for each batch we compute the output and store it in the y_hat list\n",
    "for i in range(batch_size): \n",
    "  \n",
    "    #initialize hidden state and cell state for each batch\n",
    "    hidden_state = np.zeros([1, hidden_layer], dtype=np.float32) \n",
    "    cell_state = np.zeros([1, hidden_layer], dtype=np.float32)\n",
    "    \n",
    "    \n",
    "    #compute the hidden state and cell state of the LSTM cell for each time step\n",
    "    for t in range(window_size):\n",
    "        cell_state, hidden_state = LSTM_cell(tf.reshape(input[i][t], (-1, 1)), hidden_state, cell_state)\n",
    "        \n",
    "    #compute y_hat and append it to y_hat list\n",
    "    y_hat.append(tf.matmul(hidden_state, V) + b_v)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining backpropagation\n",
    "\n",
    "After performing forward propagation and predicting the output, we compute the loss. We use mean squared error as our loss function and the total loss is the sum of losses across all the time steps as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "losses = []\n",
    "\n",
    "for i in range(len(y_hat)):\n",
    "    losses.append(tf.losses.mean_squared_error(tf.reshape(target[i], (-1, 1)), y_hat[i]))\n",
    "    \n",
    "loss = tf.reduce_mean(losses)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To avoid the exploding gradient problem, we perform gradient clipping:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "gradients = tf.gradients(loss, tf.trainable_variables())\n",
    "clipped, _ = tf.clip_by_global_norm(gradients, 4.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use Adam optimizer and minimize our loss function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "optimizer = tf.train.AdamOptimizer(learning_rate).apply_gradients(zip(gradients, tf.trainable_variables()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training the LSTM model\n",
    "\n",
    "\n",
    "Start the TensorFlow session and initialize all the variables:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "session = tf.Session()\n",
    "session.run(tf.global_variables_initializer())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set the number of epochs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "epochs = 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0, Loss: 0.191305011511 \n",
      "Epoch 10, Loss: 0.0463331267238 \n",
      "Epoch 20, Loss: 0.0246299374849 \n",
      "Epoch 30, Loss: 0.0175942592323 \n",
      "Epoch 40, Loss: 0.0129416966811 \n",
      "Epoch 50, Loss: 0.0149279581383 \n",
      "Epoch 60, Loss: 0.00840948987752 \n",
      "Epoch 70, Loss: 0.00914164073765 \n",
      "Epoch 80, Loss: 0.00813645124435 \n",
      "Epoch 90, Loss: 0.00760477548465 \n"
     ]
    }
   ],
   "source": [
    "for i in range(epochs):\n",
    "    train_predictions = []\n",
    "    index = 0\n",
    "    epoch_loss = []\n",
    "    \n",
    "    #Sample some batche of data and train the network\n",
    "    while(index + batch_size) <= len(X_train):\n",
    "        \n",
    "        #sample batch of data\n",
    "        X_batch = X_train[index:index+batch_size]\n",
    "        y_batch = y_train[index:index+batch_size]\n",
    "        \n",
    "        #predict the prices and compute loss\n",
    "        predicted, loss_val, _ = session.run([y_hat, loss, optimizer], feed_dict={input:X_batch, target:y_batch})\n",
    "        \n",
    "        #store the loss\n",
    "        epoch_loss.append(loss_val)\n",
    "        \n",
    "        #store the predictions\n",
    "        train_predictions.append(predicted)\n",
    "        index += batch_size\n",
    "        \n",
    "        \n",
    "    #print the loss on every 10 iterations\n",
    "    if (i % 10)== 0:\n",
    "        print 'Epoch {}, Loss: {} '.format(i,np.mean(epoch_loss))\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Making predictions using the LSTM model\n",
    "\n",
    "Now, we will start making predictions on the test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "predicted_output = []\n",
    "i = 0\n",
    "while i+batch_size <= len(X_test): \n",
    "  \n",
    "    output = session.run([y_hat],feed_dict={input:X_test[i:i+batch_size]})\n",
    "    i += batch_size\n",
    "    predicted_output.append(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print the predicted output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[array([[-0.60501254]], dtype=float32),\n",
       "  array([[-0.60223514]], dtype=float32),\n",
       "  array([[-0.6013318]], dtype=float32),\n",
       "  array([[-0.6003017]], dtype=float32),\n",
       "  array([[-0.6000293]], dtype=float32),\n",
       "  array([[-0.6016976]], dtype=float32),\n",
       "  array([[-0.5977682]], dtype=float32)]]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predicted_output[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see above, the predicted values are in a nested list. So we will just flatten them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "predicted_values_test = []\n",
    "for i in range(len(predicted_output)):\n",
    "  for j in range(len(predicted_output[i][0])):\n",
    "    predicted_values_test.append(predicted_output[i][0][j])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now if we print the predicted values, it is no longer in a nested list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.60501254]], dtype=float32)"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predicted_values_test[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we took the first 1000 points as a training set, we make predictions for time step greater\n",
    "than 1000."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "predictions = []\n",
    "for i in range(1280):\n",
    "      if i >= 1000:\n",
    "        predictions.append(predicted_values_test[i-1019])\n",
    "      else:\n",
    "        predictions.append(None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot and see how well the predicted value matches the actual value:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6oAAAGtCAYAAAAf5omPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl4XOV99//PmX202pZkeZE3sMGA9wUwq8hCIBASQiiBJoGUZmmThj7Pk+QHoS1JSlLyC03TZmkfskHaYjdNAmQlDrFdFgeMDQYbL9gY4d2yLGsZjWY7c54/ZtGMJEsjabYjvV/X5cuznDnnlo7MxUff+/7ehmVZAgAAAACgXDhKPQAAAAAAADIRVAEAAAAAZYWgCgAAAAAoKwRVAAAAAEBZIagCAAAAAMoKQRUAAAAAUFYIqgAAAACAskJQBQAAAACUFYIqAAAAAKCsuEo9gEz19fXW3LlzSz2MM+rp6VFlZWWph4ES4N5PXNz7iYt7P3Fx7ycu7v3Exb0vjm3btrVZltWQy7FlFVTnzp2rrVu3lnoYZ7Rp0yY1NzeXehgoAe79xMW9n7i49xMX937i4t5PXNz74jAM461cj2XqLwAAAACgrBBUAQAAAABlhaAKAAAAACgrZbVGFQAAAABKJRqN6vDhwwqFQqUeiq35fD41NTXJ7XaP+hwEVQAAAACQdPjwYVVXV2vu3LkyDKPUw7Ely7J06tQpHT58WPPmzRv1eZj6CwAAAACSQqGQ6urqCKljYBiG6urqxlyVJqgCAAAAQBIhdezy8T0kqAIAAAAAygpBFQAAAADKyOOPPy7DMLRnz54hj3v44Yd19OjRUV9n06ZNuv7660f9+UIiqAIAAABAGVm7dq0uu+wyrV27dsjjxhpUyxlBFQAAAADKRCAQ0LPPPqsf/OAHWrduXfr1r33ta1q8eLGWLl2qu+++Wz/96U+1detW/emf/qmWLVum3t5ezZ07V21tbZKkrVu3qrm5WZK0ZcsWrVmzRsuXL9cll1yivXv3luJLGxG2pwEAAACAfr70y9e062hXXs95/owa3feeC4Y85oknntA111yjc845R3V1ddq2bZtaW1v1xBNP6IUXXlBFRYXa29s1ZcoUffvb39aDDz6oVatWDXnOhQsX6plnnpHL5dJTTz2lL3zhC/rZz36Wzy8t7wiqAAAAAFAm1q5dq7vuukuS9MEPflBr166VZVn66Ec/qoqKCknSlClTRnTOzs5O3X777dq3b58Mw1A0Gs37uPONoAoAAAAA/QxX+SyE9vZ2bdiwQTt27JBhGDJNU4Zh6Oabb87p8y6XS/F4XJKy9jH927/9W1111VV67LHH1NLSkp4SXM5YowpbsyxLnb3l/xshAAAAYDg//elP9eEPf1hvvfWWWlpadOjQIc2bN0+1tbX60Y9+pGAwKCkRaCWpurpa3d3d6c/PnTtX27Ztk6Ssqb2dnZ2aOXOmpEQDJjsgqMLWvv67vVr6pfXqDhFWAQAAYG9r167VjTfemPXaTTfdpGPHjumGG27QqlWrtGzZMj344IOSpDvuuEOf/OQn082U7rvvPt11111atWqVnE5n+hyf//zndc8992j58uWKxWJF/ZpGi6m/sLXvP/umJKk3aqra5y7xaAAAAIDR27hx44DXPvOZz6Qf33333Vnv3XTTTbrpppvSzy+//HK9/vrrA86xZs2arNfvv/9+SVJzc3PZTgOmogpbi8QSc/DNuFXikQAAAADIF4IqxoWYSVAFAAAAxguCKmzLsvrCadSMl3AkAAAAAPKJoArbOh3sa6AUY+ovAAAAMG4QVGFbxzp704+pqAIAAADjB0EVttXaFU4/ppkSAAAAMH4QVGFbx7tC6cdRmikBAABgHHA6nVq2bJkWLVqkm2++WcFgcNTn2rRpk66//npJ0i9+8Qs98MADZzy2o6ND3/3ud0d8jS9+8YvpfV3ziaAK2zre2RdUY0z9BQAAwDjg9/u1fft27dy5Ux6PR//2b/+W9b5lWYrHR/7/vjfccMOAfVgzjTaoFgpBFbZ1IqOiSjMlAAAAjDeXX3659u/fr5aWFp177rn6yEc+okWLFunQoUNav3691qxZoxUrVujmm29WIBCQJD355JNauHChVqxYoZ///Ofpcz388MP69Kc/LUk6ceKEbrzxRi1dulRLly7V5s2bdffdd+uNN97QsmXL9LnPfU6S9PWvf12rV6/WkiVLdN9996XP9ZWvfEXnnHOOLrvsMu3du7cgX7urIGcFiqAt0LdGlWZKAAAAyKvf3i0d35Hfc05bLF175um3mWKxmH7729/qmmuukSTt27dPjzzyiC6++GK1tbXp/vvv11NPPaXKykp97Wtf0ze+8Q19/vOf18c+9jFt2LBB8+fP1y233DLouT/zmc/oyiuv1GOPPSbTNBUIBPTAAw9o586d2r59uyRp/fr12rdvn7Zs2SLLsnTDDTfo6aefVmVlpdatW6ft27crFotpxYoVWrlyZX6+PxkIqrCtcCwul8NQLG7RTAkAAADjQm9vr5YtWyYpUVG98847dfToUc2ZM0cXX3yxJOn555/Xrl27dOmll0qSIpGI1qxZoz179mjevHlasGCBJOlDH/qQHnrooQHX2LBhg3784x9LSqyJra2t1enTp7OOWb9+vdavX6/ly5dLkgKBgPbt26fu7m7deOONqqiokJSYUlwIBFXYVjgWV6XXpc7eKM2UAAAAkF85Vj7zLbVGtb/Kysr0Y8uy9M53vlNr167NOmawz42WZVm655579IlPfCLr9W9+85t5u8ZQWKMK2wrH4qr0OCVJsVEsKAcAAADs6OKLL9Zzzz2n/fv3S5J6enr0+uuva+HChWppadEbb7whSQOCbMrb3/52/eu//qskyTRNdXZ2qrq6Wt3d3elj3vWud+mHP/xheu3rkSNH1NraqiuuuEKPP/64ent71d3drV/+8pcF+RoJqrCtSCyuCm9iUkCMiioAAAAmiIaGBj388MO69dZbtWTJkvS0X5/Pp4ceekjXXXedVqxYoalTpw76+X/+53/Wxo0btXjxYq1cuVK7du1SXV2dLr30Ui1atEif+9zndPXVV+u2227TmjVrtHjxYn3gAx9Qd3e3VqxYoVtuuUVLly7Vtddeq9WrVxfka2TqL2wrEjNVlQyqNFMCAADAeJCqYGaaO3eudu7cmfXa2972Nr344osDjr3mmmu0Z8+eAa/fcccduuOOOyRJjY2NeuKJJwYc8+ijj2Y9v+uuu3TXXXcNOO7ee+/VvffeO+TXMVZUVGFbETOuCk8iqNJMCQAAABg/CKqwrUgsrkpvYo1qlKAKAAAAjBsEVdhWONZXUY0x9RcAAAB5YFkUQMYqH99DgipsK7OiSjMlAAAAjJXP59OpU6cIq2NgWZZOnToln883pvPQTAm2FcmoqEbZngYAAABj1NTUpMOHD+vkyZOlHoqt+Xw+NTU1jekcBFXYkhm3FItbffuoUlEFAADAGLndbs2bN6/Uw4CY+gubisQSFVR/ao0qzZQAAACAcYOgCltKBVWPyyGXw6CZEgAAADCOEFRhS2HTlCR5XQ65nAYVVQAAAGAcIajCljIrqm6HQ1EqqgAAAMC4QVCFLaWCarqiSjMlAAAAYNwgqMKWwqmKqtMhp8PB1F8AAABgHCGowpaypv46aaYEAAAAjCcEVdhSxExN/XXSTAkAAAAYZwiqsCWaKQEAAADjV8GDqmEYTsMwXjYM41eFvhYmjqx9VGmmBAAAAIwrxaio3iVpdxGugwkkHEvso+pxOuRyOBSLx9UbMamsAgAAAONAQYOqYRhNkq6T9P1CXgcTT6rrr9edaKYUMS2d93dP6rbvPV/ikQEAAAAYq0JXVL8p6fOSKHMhryIZ29N43U6Fo4kK64stp0s5LAAAAAB5YFhWYdb2GYZxvaR3W5b1l4ZhNEv6rGVZ1w9y3MclfVySGhsbV65bt64g48mHQCCgqqqqUg8DkjYejOqRXRF9s9mv7++MKBi1dKAzEV4fvqYy79fj3k9c3PuJi3s/cXHvJy7u/cTFvS+Oq666aptlWatyOdZVwHFcKukGwzDeLcknqcYwjP+wLOtDmQdZlvWQpIckadWqVVZzc3MBhzQ2mzZtUjmPbyI58Oyb0q5dar7iMv2m9VW9dSoodXZLUkHuEfd+4uLeT1zc+4mLez9xce8nLu59+SnY1F/Lsu6xLKvJsqy5kj4oaUP/kAqMVmofVY/LIb/Hqe5QtMQjAgAAAJAv7KMKW8pco+pzOdXRmx1Uo2Y8fQwAAAAAeylKULUsa9Ng61OB0YrE4nI6DLmcDvncDgUjZtb7X/j5Dn360ZdKNDoAAAAAY1HINapAwUTMuDzOxO9ZfG7ngPcPtgfVGzUHvA4AAACg/DH1F7YUjpryuM4cVEOxuMx4YTpaAwAAACgsKqqwpYgZl/cMQdWyrPS+qgAAAADsh4oqbCkci2dUVB0D3gvH4ooXaI9gAAAAAIVFUEXZ+N8/2a7bf7glp2MjWUE1u6LaGzEVippM/QUAAABsiqm/KBs/f+mIpMTWMm7n0L9DicT6min5+wXVYDQRVPu/DgAAAMAeqKii7Ow40jnsMeFY5hrV7B/j3khMoWhcJlN/AQAAAFsiqKJsOB2GJGnfie5hj43E4vK6EhVTb/+KasRUKMbUXwAAAMCuCKooG6nmRxFz+IAZMTPWqLqyg2pXb0yWJcUJqgAAAIAtEVRRNlIzdWNmfNhjM5sp+T3ZQbU7FJUkpv4CAAAANkVQRdmJ5VBRDcfMdDOl/mtUu0MxSRIFVQAAAMCeCKooC9GMKmo0nltF1ZsMqHPrKrPe60pWVJn6CwAAANgTQRVlIRQ1049zqahmbk/jczv173deqMvm10uSAuFERZWpvwAAAIA9EVRRFnqzgmoOFdWMZkqSdPmCBt3/vkWSpEBy6i9dfwEAAAB7IqiiLIQimVN/c1mjmh1UJcnlTGxvk1qjSkEVAAAAsCeCKsrCSCuq4WjfPqopqanA3eFk118qqgAAAIAtEVRRFjKDanSYNapRM66IGVdlv21p3KmgGmKNKgAAAGBnBFWUhd5IRkV1mK6/wXDi2EqvK+t1d3IqcKqZEl1/AQAAAHsiqKIshGK5d/3tiSSCaKU3u6LqcmSvUaWiCgAAANgTQRVlIRTJfepvT7JiWuHpV1FNT/1NrFG1LMkirAIAAAC2Q1BFWUitUTWM4af+9iRDbVW/qb9OhyGnw0hvTyNJzP4FAAAA7IegirKQCqpVXtfwU3/TFVXngPfcTiMdZCU6/wIAAAB2RFBFWYjEElXUKq9L0WG2p0kF1f7NlCTJ7cj+kY4z9RcAAACwHYIqykI4GVQrvS7FhqmC9jVTGiSourJ/pKmoAgAAAPZDUEVZCEeTQdXjzKGiaqaP7c/tNLKeU1EFAAAA7IegirIQjplyOQx5Xc6c16gOVlF19Z/6O3TmBQAAAFCGCKooC5FYXF6XQy6nkXPXX797YEXV03/qLxVVAAAAwHYIqigL4VhcHpdDLqdj0H1Uv7Nxvx7Z3CIpUVGt9DjlcBgDjus/9Zc1qgAAAID9DJw7CZRAOGbK63LK7Ri8ovr13+2VJN1+yVwFI6b8nsF/dN3O7N+9WFRUAQAAANuhooqyEI7F5XUnp/4Os0Y1NU14MC4nU38BAAAAuyOooiz0rVF1KNKv62+83/TdqBkfMMU3xcPUXwAAAMD2CKooC6k1qm7HwIpqR28063ksHh8wxTel/+t0/QUAAADsh6CKspBao+pyOhTrV1Ft7wlnPY/ErJyDKlN/AQAAAPshqKIshKOJqb9up6Fov+m6bYFI1vOoGZf7DGtUU1OCU2tY4wRVAAAAwHYIqigLETO5RtUxsKJ6apCg2n8takqqolrpTXQF7r++FQAAAED5I6iiLISjqX1UB65RPR0cpKI6zNRfv9spiam/AAAAgB0RVFEW0vuoOh2K9uuA1BsxJUme5HTeiHnmNaopMyf7JdH1FwAAALAjV6kHAEjJfVRdDrkG6frbG00EVZcjMd03GjtzRfVTV83XpfPrVOV1a8ub7XT9BQAAAGyIiirKQiQWl9ftkNvpUCxuycqYspsKqqnGSFEzLo9r8DWq506r1i2rZ8vnppkSAAAAYFcEVZSFcCwuj9Op+iqPJOmp3a3p91JTf6NmX1AdbuqvI1l9ZY0qAAAAYD8EVZSFcMyU1+3Qn6yepZmT/PqvFw+l3wslK6pm3NK1//yMWk4Fhw2qTiMRVOn6CwAAANgPQRUlF49bipqWvC6HvC6nFjRW6Vhnb/r91NRfSdp9rEuShg+qqYoqQRUAAACwHYIqSq4nEpMkVXgSW8pMr/XrWGco/X5q6m+mM+2jmpIsqDL1FwAAALAhgipKKhiJ6au/2SNJmlSRWJ86o9an9p5IespvZkU1xZXj1F9yKgAAAGA/BFWU1PeeflNrtxyUJE3yuyVJ0ycl9kBNVVVDgwRVpv4CAAAA4xdBFWUjs6IqKb1OdbCK6nBTf+n6CwAAANgXQRUlVet3pR9PqkhUVKt9ib97wsmpv4OsUaXrLwAAADB+EVRRUpXevqBam5z663UnfiwjsbgkKRSNy+vK/lF1u4bZRzUVVMmpAAAAgO0QVFFSmWtIU0HVk6yWRkxTlmWpKxRNV1lThquoOhwDzw8AAADAHgiqKKmoGU8/9rkT29N4XH0V1U/8+zZ1h2KqyZgiLEnuYdaoppopxVmjCgAAANgOQRUlFTEHBsnMoLp+1wlJUldvNOuY4SqlqTWqVFQBAAAA+yGooqRSFdX3LJ2Rfi0VVMOxuJomJ7aq8bqcWZ+LDRJwMzmoqAIAAAC2RVBFScWSQfUbf7I0/VpqjWo4FtfsKRWSpC+/94Ksz0XjcQ0l3fWXoAoAAADYDkEVJZWa+uty9K05TTdTisUVMy2tOatO9VXerM+Zw1VU01N/8zlaAAAAAMVAUEVJRc24PE6HDKMvqDochtxOQxEzrogZl8tpyNWveVJ0mLWnqa6/7KMKAAAA2I9r+EOA/LMsS3/Y3areiDloB1+P05GoqMbjcjsd6SprimPopr/prr8mU38BAAAA2yGooiQ2vX5Sf/7jrZKkSRXuAe97XI701F+Xw1ClN/GjumBqlZrPbdDHLj9ryPPT9RcAAACwL4IqSqIjGEk/djsHzkBPBdWIGZfb5dCMSX795BNrtHRW7YAOwINJdf21qKgCAAAAtlOwoGoYhk/S05K8yev81LKs+wp1PdiLwxjYPCmTx+VQxExUVN3J0HnhvCk5nz9VUY1RUQUAAABsp5AV1bCkt1mWFTAMwy3pWcMwfmtZ1vMFvCZsoqs3mn7cv1GSlLFG1YzLNUiQHY7TydRfAAAAwK4KFlStxJzLQPKpO/mH1ABJUmdGUB186q9T4Vhc0bg16PvDSW13Q1AFAAAA7Keg29MYhuE0DGO7pFZJv7cs64VCXg/2MXxQdeip3Sd0sjs8aFfg4aS6/jL1FwAAALAfoxjNZgzDmCTpMUl/ZVnWzn7vfVzSxyWpsbFx5bp16wo+ntEKBAKqqqoq9TDGhR/sCOuZIzFJ0rxah+5b4896/yvP92pfR1ySdPUcl247zzui88ctS3/2u6BunO/We+d7xjxe7v3Exb2fuLj3Exf3fuLi3k9c3PviuOqqq7ZZlrUql2OL0vXXsqwOwzA2SrpG0s5+7z0k6SFJWrVqldXc3FyMIY3Kpk2bVM7js5NHD26VjpyQJE2ZVKvm5kuy3v/aK89IHV2SpHlzZ6u5+bwRnd+yLOl3v9GsOXPV3HzOmMfLvZ+4uPcTF/d+4uLeT1zc+4mLe19+Cjb11zCMhmQlVYZh+CW9U9KeQl0P9pI59TcSiw94P7PZktsx8h9TwzDkdBgy4wPPDQAAAKC8FbKiOl3SI4ZhOJUIxD+xLOtXBbwebCIet3SsM5R+3hOJDTimc5iuwLlIBNVRfRQAAABACRWy6++rkpYX6vywr/W7Tuhge1DvXjxNv9lxXMGwOeCYQLgvvI6m66+U6PxLRRUAAACwn4J2/QUG88bJxK5Ft104R5LUEx5YUc00mq6/UqKiStdfAAAAwH4Iqii6mJkIjw3ViU6+g039zeQaxRrVxOcM9lEFAAAAbIigiqKLxeMyDGlyhVuSNFiW/PVnLks/Hn1F1UFFFQAAALAhgiqKLha35HY4VON3n/GYC2bUavXcyZJGv0bV6ZBMk6AKAAAA2A1BFUUXM+NyOgz53M4hj3M6EpVU16ibKVFRBQAAAOyokNvTAIOKxa30ljP3v2+RFs+sHfS41NrUsTRTousvAAAAYD8EVRRdzLTkSlZLP3TxnDMe50geM6btaSioAgAAALbD1F8UXaKiOvyPXirMpv4eKSqqAAAAgD0RVFF0MTOeU/hMrVEdbVHU6TDSW+EAAAAAsA+CKoouc43qUJxG4pjR7oXqcrKPKgAAAGBHBFUUXSxupRslDcWZDLOj7dzrNAy6/gIAAAA2RFBF0eU69Td1THy0QdVBRRUAAACwI4Iqii7XZkqpqb+jrYom9lGlmRIAAABgNwRVFEU4ZqonHJOUe0X19kvmSpIum18/qms6HYbIqQAAAID9EFRRFO/7zmZdcN/vJOXeTGnprElqeeA6Tav1jeqaLqdBRRUAAACwIYIqimL3sa7045hpjXpv1JFgjSoAAABgTwRVFJ2ZY9ffsXI56PoLAAAA2BFBFUUXjcdzmvo7Vg6DiioAAABgRwRVFF2xpv4m1qgSVAEAAAC7IaiiqOJxS7G4JWcRpv46HQ4qqgAAAIANEVRRVBEzrpgZl7sIU38Ta1Tp+gsAAADYDUEVRRUx4zLjlpxF6vpLTgUAAADsh6CKogpH44rG43I7i9X1l6QKAAAA2A1BFUUVMeMyi9RMycE+qgAAAIAtEVRRVJFYXNG4VZTtadhHFQAAALAngiqKKhJLrFF1FaXrryHTJKgCAAAAdkNQRVGFY6aiZrwozZSoqAIAAAD2RFBFwWWuE43E4oqZVlG2p2EfVQAAAMCeCKoouEgsnvU4sT0NXX8BAAAADI6gioILx8y+x2Zqe5oi7aNqSZZFVRUAAACwE4IqCi6zohqKmLIsFWWNauoaTP8FAAAA7IWgioILZwTVYCRRXXU7i9P1VxINlQAAAACbIaii4LKDakxSYv1oobkIqgAAAIAtEVRRcJlTf3uSFdViTP09u6FKkvSH3ScKfi0AAAAA+UNQRcFlNlMKhhMV1WJM/X3bwqmaPaVCv9h+tODXAgAAAJA/BFXk3b4T3dp5pDP9PLOiGggXr6LqcBiaP7VKx7tCBb8WAAAAgPwhqE5gRzt602tG8+nLv9qlz/73K+nnmWtUj3f1SpKmVHryft3BNNZ4daIrXJRrAQAAAMgPguoE1ROO6ZIHNujSBzaoN2IO/4ERaO0K642TAcXMREANZpz/pbc6JElTq715veaZNFT7dKonnB4LAAAAgPJHUJ2gfr3jmCTpdDCqVw535PXcp3rCipqWDrYHtb+1Wz3JdalNk/3pabhTq315veaZNNZ4ZVlSWyBSlOsBAAAAGDtXqQeA0shcQ9rek78QZ8at9Pn+68VD+r9PH9B502skSavnTtHh00ckSQ1Fqqg2JgPxia6QptUWJxwDAAAAGBsqqhNUIByTJ9l599QZgupopgR3BCNKbVv6s5cOS5J2H+uSJF00b0r6OL/HOeJzj8bUmkQgbu1mnSoAAABgFwTVCaonHFPTZL8k6VRgYIjb/Eabzvu7J/XCgVMjOm9m6M2cbmsY0rWLp49ytKPXWNNXUQUAAABgD0z9naCCEVO1FW7V+FyDTv19ozUgSfr2xv1aOWeyXDnue9qWDL1elyOr22+F26lav1vvPL9R1b7i/djVVXrkMKRWgioAAABgG1RUJ6hAOKYqr0t1Vd5Bp/46HYkfjWf2tWn+vb/NeR1r6rjFM2uzXq/wJsLp9z6ySt/4k2VjGfqIuJwO1Vd5mfoLAAAA2AhBdYIKhk1VeJyqq/SofZCOuP33Vz3emVtFMrWu9b3LZmjmJH/69Spv6Yr3U2u8TP0FAAAAbISgOkEFwjFVel2aUunRie6BIa4nnN1IyZHjT0pquu+7Fk3Tc3e/TStmT5IkVRSpedJgGqt9OtFFRRUAAACwC4LqBNUTSUz9XTlnsg6c7NFbp3qy3g9GsyuqkYz1pkNJBVWfOxFMa/1uSVKlp5QVVZ+Od4VkWVbJxgAAAAAgdwTVCSox9del65fOkCT9fteJAe9nynWrmlA0cZzXlfjRSgXVCm/pKqoXzpus9p6IfvHK0ZKNAQAAAEDuCKoTUCQWV8SMq8rr1MxJfnlcDp3st0VNT781qr3R3IJqqqKa2qN1UoUn63kpvHfpTNVVerR5/8i22gEAAABQGgTVCagnnAihlckGRzU+l7pD/YJpvwpqKJrr1F9TXpdDhmFIkq48p0GSdLA9OKYxj4XDYaja51IollvYBgAAAFBa7KM6AaWqpal1ozU+t7p6o/2O6R9Uc6yoRuPpab+SdNXCqbrn2oVaPnvyWIY8Zj63M+fpywAAAABKi6A6AaU6+qYqqtWDVFSD4eznOQfVWFxed/Z61E9cefZoh5o3PrdToRwbQgEAAAAoLab+TkCB9NTfRKCs8bv1ZluP2jLWqQYjpqp9fb/HyH2NqplVUS0XPrdDISqqAAAAgC2UX6JAwQUj2WtUq30uHWwPatX9T2Ud01DtTT/PeY1qv6m/5cLvdrJGFQAAALCJ8ksUKLh0M6WMNaoDjomYaqjqC6ojq6iWbiuaM2GNKgAAAGAfIwqqhmFUFGogKJ7UGtWqZEXV6TDS76XWogbDMdVnVVRzX6Pqc5ff7z+oqAIAAAD2kVOiMAzjEsMwdknak3y+1DCM7xZ0ZCiYVNffiuQa1Y5gX8ffQ+1BmXFLPRFTU0cTVKPxsqyoet1O9UZopgQAAADYQa6lr3+S9C5JpyTJsqxXJF1RqEGhsFLNlFIV1a5QX1A92B5Mvz9zkl/fvm253E4j52mz4Zgpb5lWVMM5hm0AAAAApZVzorAs61C/l/i/fpsKhk05HUa66dF97zlfK+ck9jk93hVSdzK4Vvtcun7JDDVNrsjONR55AAAgAElEQVR5a5dwrDybKfncDqb+AgAAADaRa6I4ZBjGJZIswzDchmF8VtLuoT5gGMYswzA2GoaxyzCM1wzDuGvMo0VeBMIxVXicMozE2tT5U6v18EdXS0o0WkrtqVqdbLI0kkZEiaBaflN//W6noqalmMn0XwAAAKDc5RpUPynpU5JmSjoiaVny+VBikv6PZVnnS7pY0qcMwzh/tANF/vSEY+lpvympDsBHO0L64EPPS1J6H9Uqr1MdwUhO5w5Fy3Uf1UR4zrUyDAAAAKB0XMMfIlmW1SbpT0dyYsuyjkk6lnzcbRjGbiWC7q6RDhL5FYyY6T1UUxwOQxUep9a9eDC9Z2qqorps1iQ9svkt9UZM+T1nrpb+0+9f17HOkDzlGFST4+6NmANCOgAAAIDyYliWNfxBhvGIpLssy+pIPp8s6R8ty/qznC5iGHMlPS1pkWVZXf3e+7ikj0tSY2PjynXr1o1k/EUVCARUVVVV6mGM2T9uDSkQtXTfGn/W63dtDKo7Yime/JH46mV+zahy6NWTMX1jW1ifXeXTovozB9U7nuyRJJ03xaH/70L/GY8rhWcOR/WDnRF9/Qq/GipGHqTHy73HyHHvJy7u/cTFvZ+4uPcTF/e+OK666qptlmWtyuXYXEtLS1IhVZIsyzptGMbyXD5oGEaVpJ9J+uv+ITV5rockPSRJq1atspqbm3McUvFt2rRJ5Ty+XH1nz2ZVOx1qbr446/W6rZvU2daTfv6OKy7R1BqfFnaG9I1tf9DkWfPVfNGcM573wr1/1JY327Vg1jQ1N+f041E0gVePSjtfVnjK2Vq5bEa6Wpyr8XLvMXLc+4mLez9xce8nLu79xMW9Lz+5lpYcySqqJMkwjCnKIeQahuFWIqT+p2VZPx/dEJFvgbCpCs/A29d/OnAqzNX6E3939caGPG+Fxymvy6H7b1yUp5Hmjy/Z4OlvHt+p//s/B0o8GgAAAABDybWi+o+S/mgYxn9LMiR9QNJXhvqAkWgp+wNJuy3L+saYRom8CoSjqvZVD3i90ps9rdeX3A/V53bI7TTU2Rsd8JlMoaippU2TVDPCamUxzGuo1PRan451hrT9UMfwHwAAAABQMjlVVC3L+rGk90s6Iem4pPdblvXvw3zsUkkflvQ2wzC2J/+8e0yjtZmOYERfeGzHsAGv2LpDsXRH30z9mwyltq8xDEO1fre6QkN/HeFYXF53+TVSkqSzG6r0x3verg+unqWdRzuVy9psAAAAAKUxZEXVMIway7K6klN9j0t6NOO9KZZltZ/ps5ZlPatE9XXC+snWQ3r0hYPae7xb3//IKk2u9JR6SLIs64xBNTX1d+G0aj3yZxdmvVfjc6trmMAdjsZVV1l+e6hmumBmrda9eEjHOkOaMam8Gj4BAAAASBiu/JUKptskbc34k3qOIbQFEnuPbnvrtL67aX+JR5PQGzVlxq1BmwmlgmrTZL8aa3xZ71X73cNP/Y2ZZVtRTWmanAinxzpDJR4JAAAAgDMZMlVYlnV9cq3plZZlnZXxZ55lWWcVaYy2ted4d/pxIGxmvffSwdOae/evdeBkoKhjCoQSDZEG20u0O/needNrBryXmPo7dDOlcDQubxnuoZpparVXknSym6AKAAAAlKthU4WVWMz36yKMZdzZe7xL718xU3PqKhSMZIe8x18+Ikl6+vWTRR1TKmwOOvXXk5i2+4GVTQPeq/G5hp/6GzPlc5f31N+p1YlKcWt3uMQjAQAAAHAmuZa/XjIMY3VBRzLO9IRjOtEV1tkNVar1u9URzA55qcW7xWrps3FPq+be/Wsdag9K0qCdeb9w3Xl6/FOXak5d5YD3av25rVEt94rqlEqPHIbU2kVQBQAAAMpVrtvTXCTpQ4ZhtEjqUSJnWZZlLSnUwOyu5VSPJGlefaVqB1nfmeqoW6zms9/ZmFgj+9LB05IGr6jW+NxaNmvSoJ+vSX4NlmWlx95fyAYVVafDUH2VV61M/QUAAADKVq5B9V0FHcU41NKWqFzOratUjd+tI6d71RGMaPuhDjWfO1WprBcvUlLtjSbWyAYjib+rBgmqQ6n0OBWLW4qaljyugUHVTL5X7hVVSZpa42XqLwAAAFDGhtuexifpk5LmS9oh6QeWZQ3dUQeS+iqqc+sr0hXVrz25R2u3HMra+sWMFzeopgLaYF1/h5KqlPZGTXkGCaORWFyS5HWVd0VVkuoqvWrviZR6GAAAAADOYLiy2iOSopKekXStpPMl3VXoQdndN9bv1b9s2K+myX5VeFzpoLo32QX4Xzftl9ORqEqmKpyFFkpe5/XkGAab+jsUf7LRUihqqtY/MOSGkkHYV+bb00iJacwHk2t1AQAAAJSf4dLK+ZZlLZYkwzB+IGlL4Ydkf/+yIbEedElTraREI6JY3NJLBzskSc8faE8f2xMuToE6lKx47j3RreZzGwZtpjQUn6svqPb3woFTeuSPLZLsUVGtzqGDMQAAAIDSGS6opv9v3rKs2Jma6KBPagqsJFV7E2FwUkYF0uUwFMuY7tsTKXxQjcetrO1x3r9i4PYzw0lVVHsHCaq3PPR8+rEd1qjW+NzqDsWGbAwFAAAAoHSGSxVLDcPoSv7plrQk9dgwjK5iDNBujnX2ph//RfPZkqQLZtSmX7v6gsas4wPhwk/9/cbvX1co2hegp1R4RnwOvztVUY0PeM/j7PsxKveuv1Kiohox4wrHBn4tAAAAAEpvyKBqWZbTsqya5J9qy7JcGY9rijVIOzlyOhFUH/3YRZpbn9iPdNHMvm/V4pnZ278UY+rv9kMdWc8nV45s2q8keZNrT3sHWVNb4e0Lp7aoqCYr3F0hpv8CAAAA5WhkHXUwrMPJoNo0qSL9mmEY+tatyxWLx7OmBktSoAhB1e00tHhmrXYc6ZQkTakcS0V1YFCt9LjUEUyEPjtUVGuSjaS6QzFNrS7xYAAAAAAMUP7lL5s5GUhs/zK1xpv1+nuWztCNy5sGrIkMRhJrJfPheGdI4djAINkbNdNBU5Imj2bqr+fMQbXK2/f7jtGE4GJLNZKioRIAAABQngiqedYdisntNM44BbZpsj/r+c4jXVr6pfXafWxsS37jcUsX/8MfdO7fPKk/f+RF7Trad77eaFw+T19QHU3VM9X197WjA8eZOfX3rIbKEZ+72KozKqoAAAAAyg9BNc+6Q1FV+9xn7CZ7ydn1euwvL8l6rSsU0///5J4xXbczozr41O5W/WH3ifTzUMSUf4z7m6Yqqt/euF+/2XEs+z332EJwsbFGFQAAAChvBNU86w7F0hW7M1k+e7L+8H+u1C8+fWn6tRNd4TFdNzXlOOVoRvfhUCx76u9oZAbQN1oDWe9lbrdjB1RUAQAAgPJGM6U86wpF02sgh3J2Q5Usy9IX33O+Ht9+VKd6xhZU27qzP3+kI5R+3Bsx5XM7tfZjFysWH92WLL6Mimz/IJ7a5uXhj64e1bmLjTWqAAAAQHmjoponX/rla2r++kYdbA8OW1FNMQxDd1w6T0uaatXVO7bqXmZFdemsSTrW0VdR7Y0mguqas+t0+YKGUZ0/c6/U9p6I/ubxHenGTeGoqXee36jmc6eOcvTFVeFxyukwqKgCAAAAZYqgmgftPRH96LkWtZwK6sDJnpyDakqNz63uUHRM3X/bAhFJ0va/e6dWzJ6kox296fOFomZ6jeloZa65/ZcN+/Ufzx/US28l9meNxOK22D81xTAMVftcrFEFAAAAypR90kUZ69+xtzqHqb/Zx7sUt6SeyMCtX3LVFgjL7TRU63drRq1fPRFTXaGYYmZcUdMa8xrVwaS2qgnH4vK6yr+JUqZqn4uKKgAAAFCmCKqjdOfDL+pLv3xNUl9QdTsTVccRV1STXWi7x1Dh23u8W1OrfTIMI72H68nukELJ9aOFCKrtPYkqbjgWl3eMXYWLLVXFBgAAAFB+7JUuykR3KKo/7GnVj55rkSS9crhTjTVeXXxWnaTEGsiRGGsX2oOngtq4t1U3rZgpSWqoTgTV1q6wepNVWl8eguTP/mJN1vO+oGraauqvlPiej3VdMAAAAIDCsFe6KBPP7T+VfhyPW9q8v01rzqrTB1Y2SZJC0ZF11s3sQhuPW4qZI/v8rmOdsizp6gumSZKmVvskSa3d4fT03Hzsb7pyzpSs5+3Bvoqqx2ZBtcbnZo0qAAAAUKbYnmYUDrT17SO6paVdp3oiumxBg25YOkO9EVNvP69xROfLrKj+759s1+Pbj6rlgety/vyxzsRWNNNrEwG1MTn1t7U7pN5kUB1rM6XBnO6JyLKsZDMlu61RdbNGFQAAAChTBNVRaE922JWkDXtaJUmLZtbIMAx98MLZIz5fao1qZ29Uj28/KinRSTfXKuXxzpA8ToemVHokSVVel/xup9a/dkK/2XFcUuHWqKb2ULXl1N9QVC1tPQrH4ppa7dXk5PcPAAAAQGkRVEchtTZTkh56+oAkqWlyxajPN73WJ7fT0K6M7sEnA2HNnOTP6fPHOkOaVutLbyGTaqi09a3T6WPyPTV3eq1Pp4P2Dao1/kRFtfnBTZISv2j41V9dXtpBAQAAAJDEGtVRaQ9GtGhmjVbPnZx+rco7+sxf4XHponl16eqsJJ3oCuX8+eNdiaCaaWqyoVJqbHPrKkc9vsHMq69URzCqSCqoFqBiW0ip6dEpO490neFIAAAAAMVGUB2F9p6I6iq9+vv3LcrbOS+dX6/9rX1rX1tHElQ7Q+n1qSmphkoXzpuiHV+8WrOmjL7iO5jptX4FwjGFY4k1sHarqF4wozbructhlGgkAAAAAPqzV7ooE4mg6tH8hqq8nXNefXbF80RXOKfPWZal450DK6qpLWrqqzzpKcH5UF+VOG+N36VAKGbbqb8Lp1VnPY/FLQXCNFcCAAAAygFrVEehvSeiyZUeuZwOff0DS3RWw9in1c6tz654Hu3szXksETOuaTX9KqrJqa1+d35v8W/vulynesL6zY7jCkRi6X1a7RZUfW6nKjxOLZs1SR++eI7+4j9fUktbjxbNrB3+wwAAAAAKiqA6QqGoqWDETHfYvXnVrLycd3bG1NzFM2v1s22H9eGL5wzZpCkUNfW+7z4nSQOm/tYlxxe3rLyML6Wh2quGaq+eeb1NliWdTu6larftaSRp+99dLYchvXGyR5L0xskAQRUAAAAoA/Yqg5WBVMffujxvZVLh6fudwXuWTldbIKLbvvfCkJ/ZdaxLh9oTlddptdkdgl2OxK014/kNqilVyb1fTwVSQdV+P0oel0Mup0Nz6ipkGNKbbT2lHhIAAAAAEVRHLBVUC7Hn5r99aKV+9heX6NbkXqyt3UM3VMoslvavqJ43vUaSdPmC+vwOMinV5bgtkFhL63Xb90fJ53ZqRq1f33v6gC766lMKRlirCgAAAJQSU39H6FSBKqqSdM2iaenHt144W7/fdWLI40NRM/041eQo5fwZNXrx3neovir/45Sk6lRFtce+U38zndVQqWf29aonYupQe6/O7ddsCQAAAEDx2LcMViKnC1hRzeRzOxTOCKKDCSYbGT1481I5B9lepaHam9eOv5lSQXXz/jZJ9pz6m+ldF/T9kiAQjpZwJAAAAADsnS5KoJAV1Ux+t1O9wwbVxBTVZbOK3wCoyuuWJL1yuFOS/SuqH1w9S0uaEt/Hzl6CKgAAAFBKBNURiJiWHtncIsOQanzugl7L53YqFrcUNeNnPCY19dfvKf4M7lQzpRSPzSuqLqdD37xlmSSpq5c1qgAAAEApsUY1R8c7Q/qHLSEd7EwER8cgU23zyZdsThSKmnI7Bw+Bqam/Fe7iVzOnVGRXlO0+9VeSav2JXz5QUQUAAABKy/7pokj+4be79Wbnmaub+eZPhs9Q9MzX7E1XVIsfVP0ep/7x5qXp53bu+ptSM0hQNeOWnn79ZKmGBAAAAExI9k8XRfLZq8+VJK2YPUm//PRlBb+eNx1Us9ep/sfzb2n3sS5JUm/ElGGUrprZNLlv71bPGaq+duJ2OlTpcWYF1X/7nzf0kR9u0c62mF46eFoHTgZKOEIAAABgYmDqb45mTanQg1f6dd3bLy5KBdM/SFC1LEt/8/hOSVLLA9cpGDFV4XYWrLPvcCq9fT8+rnEQVKXE9N/MoLrvRLck6enDMT24dbOWNNXqF0X4RQUAAAAwkY2PdFEk9X5H0abZ+gaZ+tt/GnBv1CzJtN+UihJeu1Bq+gVV00r8/frpxPd+7/HuUgwLAAAAmFAIqmUqVVHN3KKmO5Td5Kc3UuqgOv4K8v2Daqqi3RFOJNZwLK6uEM2WAAAAgEIiqJapzK6/KV2h7G1TgpGYKtylC4sV3vFXUa32utQT7vs+n+gKDTimpa2nmEMCAAAAJhyCapnyDVJRDYSzg2pvNC5fKSuqJdgWp9Aq+wXVox19QbU6uXfs6ydoqAQAAAAUEkG1TPkGaaaUOfW3JxxTbyRW0rA4XhooZar0uhQIJ77nlmXpdDCSfu+ieVM0ucKtzW+0lWp4AAAAwIQw/pLGOJFaexqKmvrVq0cVjMQUCGVPST3VE9GkCnephjguVfv6KqrBiCkzbmlqtVeSZFnSZQsa9My+NlmWVcphAgAAAOMaQbVM+ZJ7o758sEOffvRl3fPzHerOCKpvnOzRwVNBzauvLNUQx6VKj0u9UVMxs69p0pXnNEiS3jzVo+WzJulkd1jtPZGhTgMAAABgDAiqZSrVUffVw52SpA17WtWdsXbyYz/eqljc0lkNVSUZ33hVmWwQ1RMx1dWb+H5fOr9eM6sM/e3152tufYUk6a32YMnGCAAAAIx3BNUy5fc4tXBatXYd65IkdYdiOj1IFe/shtJWVKfX+rR89qSSjiGfqryJXxD0hGPpimp9lVdfuaxCV507VXPqEt/vt07R+RcAAAAolPG3EeY48s7zG7XneHf6+X++8JYqPE4FI30NlhY0VpdiaGl/vOftJb1+vlUlO/sGwjF1JfdTrfa5dDr5ftNkvwxDammjogoAAAAUCkG1jN120Wx9a8P+9PPTwajm1FXoizdcoI5gRDcsnSmnwyjhCMefSm9fUE2tCa7xu9NB1etyakatXweZ+gsAAAAUDEG1jE2v9esXn75UJ7vDuvORrZKk//7kGk2t9pV4ZOPXYFN/a3zZ/0ym1/p0rLO36GMDAAAAJgqCaplb0jRJZrxvKxRCamFlBdX01N/sLYCm1fq080hn0ccGAAAATBQ0U7IBp8OQYUiXnF1X6qGMezX+RCg91N6rrlBMfrdTHlf2P5PptT4d7wqxlyoAAABQIFRUbWLP318jp8F61EKbUevTyjmT9YNn39TqeVNUX+0ZcMy0Wr9C0bg6e6OaVDHwfQAAAABjQ0XVJrwup1xOblehGYahd13QqONdIe043KGz6gfuUzu9NjH9+lhnqNjDAwAAACaEgiUfwzB+aBhGq2EYOwt1DaAQUmtSW04FdXbDwKA6LRlUjxNUAQAAgIIoZInuYUnXFPD8QEGkGipJ0tlTKwe8n6qoHu8iqAIAAACFULCgalnW05LaC3V+oFCqMrajmT2lYsD7DVVeOQym/gIAAACFYhSyc6lhGHMl/cqyrEVDHPNxSR+XpMbGxpXr1q0r2HjGKhAIqKpq4FRQjC/7Tpv6yguJEPqlS3yaU+MccO//emNQi+udunOxt1TDRJHw737i4t5PXNz7iYt7P3Fx74vjqquu2mZZ1qpcji1511/Lsh6S9JAkrVq1ympubi7tgIawadMmlfP4kB/TjndJLzwjSbr6yks0vdY/4N7Pee05yedSc/NFJRolioV/9xMX937i4t5PXNz7iYt7X35oIwv0k7lGdfIZtp9pqPLqmX1tWv/a8WINCwAAAJgwCKpAP9Ved/qxz+0c9JirL2iUJG3c21qUMQEAAAATSSG3p1kr6Y+SzjUM47BhGHcW6lpAPlV6Bw+nmf5k1SwtnFattkAk/dqh9qDufPhFfWfjfhVy7TcAAAAw3hVsjaplWbcW6txAIbmcuf3+pqHaq5Pd4fTzB57coz/sadUf9rTq0vn1WjZrUqGGCAAAAIxrTP0FRqm+yqu2QFi/2XFMh9qDemrXCV1xToMkaX9roMSjAwAAAOyr5F1/gXJ14bwpQ77fUO3V4dO9+sv/fEkzan0Kx+K6ZdUsPbPvpA6fDkqS4nFL7cGI6qvYxgYAAADIFUEVGMQbX323jGGOqa/q6wh8tDOx72rTZL8aq306fLpXkvTV3+zW9599U6/cd7Vq/e5BzwMAAAAgG1N/gUE4HYYcjqGjamONb8BrU2u8aprs1+HTQX3+p6/o+8++KUnafayrIOMEAAAAxiOCKjBK1yyapgdvXqqv3bQ4/VpdpVezp1TowMke/WTr4fTru44SVAEAAIBcEVSBUfK6nPrAyiYtaerr7utxObRq7hS1JrsBL5pZI0naRUUVAAAAyBlBFRijefWVWc8vX1CffvxPf7JMV57ToNeoqAIAAAA5o5kSMEY+tzPr+awpFfqb687TzEl+LWis1vkzarT5mQOKxOLyuPjdEAAAADAcgiqQBz/95JqswPrnl5+Vfnz+9BpFTUv7Wrt1wYzaUgwPAAAAsBWCKpAHq+aeec/VBY1VkqQ323oIqgAAAEAOmIcIFFhq/9RAKFbikQAAAAD2QFAFCqzalwiq3QRVAAAAICcEVaDAKtxOGYbUHYqWeigAAACALRBUgQJzOAxVeV3qoqIKAAAA5ISgChRBjc+tQJigCgAAAOSCoAoUQZXXxdRfAAAAIEcEVaAIqn0umikBAAAAOSKoAkVQ7XMVfOpvKGrqf/3Xdh1qDxb0OgAAAEChEVSBIqjyuQteUd38Rpsee/mI7vvFawW9DgAAAFBoBFWgCBJTfwu7RjUVhNt7IgW9DgAAAFBoBFWgCGbU+tQWiOhTj76kUNQsyDWOdPRKYr9WAAAA2B9BFSiC2y6aI0n69avHtGnvyYJc48jpRFA90tGrmBkvyDUAAACAYiCoAkUwpdKjpz93lSTp1cMdeTuvZVnpx3uPd0uSQtG49iQfAwAAAHZEUAWKZHZdhS6YUaNXD3fm5XyWZemabz6jSx/YoEc2t2jrW6f1toVTJUl3PvJiwaYYAwAAAIVGUAWKaOmsSXrlcIficWv4g4dxOhjV3hPdOtLRqy/98jU11nj10IdXyuUwdKIrrKdfL8wUYwAAAKDQCKpAES1tqlV3KKaWUz1jPlfmfqlxS3r7eY1yOR3a+NlmSdK+1sCYrwEAAACUAkEVKKIlTZMkSa/kYZ3qwYygKknvWzZTkjRrSoVmTvLr9ROsUwUAAIA9EVSBIlowtUp1lR6tfeGQLMvSpx59SRd+5Sn98Y1TIz5XKqj+9TsW6PY1c3ThvCl912ms0o7Dndp5pFMHTg6srD6x/Uh6OxsAAACg3BBUgSJyOR367LvO1ZaWdv2fn7yiX796TK3dYd37+I4RneetUz36p9+/rum1Pv31O87Rl967KOv99yyZoQNtPbr+W8/q/f+6WUczQume4126a912XfrABj2581hevi4AAAAgnwiqQJF9cPUsvW3hVP385SMyDOnTV83XgZM9au0K5XyO7z/zpiTpO3+6YtD3b1rZpG/ftlyfuOIsdQSj+um2w+n3/ntr3+NP/sdLo/wqAAAAgMIhqAJFZhiGPrh6liTp3MZqXXFOgyTp73+9W8/uaxv285v3t+nRLQf1gZVNWjF78hmPu37JDN3z7vM0f2qVXjnUtyb2xZb2rOMy92IFAAAAygFBFSiB5nOn6tYLZ+tbty7X+TNqJEm/fOWo/vq/Xh7yc92hqP7y0Zd0Vn2l7r3uvJyutbRpkrYf6pBlWfryL3cN2Me1tTs8ui8CAAAAKBCCKlACHpdD//D+xVrQWK0qr0v/6x3naGq1V6eDUZn99lj9x/V7082W9rUG1BGM6vPXLFS1z53TtS5fUK9TPRE9uH6vfvhcYsrw3dcu1FkNlZKkt04Fh/o4AAAAUHQEVaAM3PWOBfqrty+QGbfUFuircHb2RvWtDft16/eel6R0U6RZU/w5n/uGpTO0as5kfWfjG5Kki+ZN0a0XztYPb18tSWppG/uergAAAEA+EVSBMjGj1idJWdvG7DnWJUlyOw1J0rGORMOlGZNyD6oOh6E7Lp0rSbr4rCn6r0+sUa3frabJfnldDvZbBQAAQNkhqAJlYnptInx+9de70w2OXjuaCKoNVV5JiRBb7XWpJsdpvynXLpqub926XN/7yKr0ay6nQwsaq7SXoAoAAIAyQ1AFysTMZJV061untb81oGAkpn9//q2sY4529Gr6JN+Iz+10GHrP0hkD1rUunFaj3ccIqgAAACgvBFWgTNRWuHXTiiZJ0paWdm15s11vtvVo/tQqnQyEZVmWWrvDaqwZeVA9k4XTqtUWCOtUgM6/AAAAKB8EVaCM/O31iS1n7n1sp778q12SpHee36ioaenXO47peGdItf6RTfsdysJpia1x9h6nqgoAAIDyQVAFysikCk/68YGTPar0OLVoRq0k6dOPvqzjXaGct6XJxbnTqiVJuwmqAAAAKCMEVaCMzamr1NsWTtWXbrgg/VqNz5W38zdUe1Vf5dHe4115OycAAAAwVgRVoMxs/Gyz3r98piTpnMYq+T1O3X7JXNVVJqqt1XkMqolrVGvviUBezwkAAACMRX7/jxfAmM2rr9RX379Y710+U4tm1KRfdzoSe6nmc+qvlAiq/731kOJxS47kNQAAAIBSoqIKlCGf26krz2lQXXL/VCkzqOa/otoTMXWkozev5wUAAABGi6AK2ITDSATVmjxXVM+dViVJuv2HW/TjP7bk9dwAAADAaBBUAZtwOQtTUV3SNEmSdKCtR3/3xGt6+eDpvJ4fAAAAGCmCKmAThVqj6nY6dMuqWennLx/syOv5AQAAgJEiqAI24UxO/XUU4F/t/Tcu0qtfvFrVPpe+/KtdOrLXsaMAACAASURBVMp6VQAAAJQQQRWwiX+5dbluWDpDZzdU5f3cbqdDNT63Ll9QL0lau+Vg3q8BAAAA5IqgCtjEedNr9C+3LpfbWbh/tl9532JJUlsgXLBrAAAAAMMhqAJIm1zp0cJp1TrZHSn1UAAAADCBEVQBZGmo9upkmVRUD54KyrKsUg8DAAAARUZQBZClodqrtu7SB9UTXSFd8fWN+sJjO0o9FAAAABQZQRVAloZqr052h0teyTzYHpQkrd1ySKGoWdKxAAAAoLgIqgCyNFR5FTHj6ghG9b2nD+h3rx0fcExHMKJL/uEP+uIvXivYOE50hdKP/3jglK7+p//RD559s2DXAwAAQPkgqALIsnz2JEnSI39s0Vd+s1uf+Pdt2ri3VUeSe6u+dPC0ln359zraGdLDm1sKNo7jnX1B9c8f2arXTwT097/aVbDrAQAAoHwQVAFkWTlnii45u07ffGpf+rWP/uhF3fnwi1r/2nG9/7ub837N3ce6dPh0MOu1w6d75XU59LWbFsuM901DLvWUZAAAABQeQRXAALdfMnfAa3uOd+u5/W1Zr3lcY/tPyMPPvak/f+RFXfvPz+iGbz+Xfn39a8f18OYWhWNx3bJ6dtZnDp/uHdM1AQAAUP4KGlQNw7jGMIy9hvH/2rvzOLmqOu/jn3Nrr+rqvTvd6U46+x4SkkDCEhIisolEBREFZBBkXHj0cRy30UdFhxlFR9RRBxccBYPoACKDgAISBNkCAbISsqezdLo7vVd1rfc8f1Sls5CELL2R/r5fr3p13XtP3XtunTq361dnuWaDMeaLfXksEek9500exjffM41nvnAu99w4j0WTKgG4/5UdnDqymIUTK4DceNYjeWN3J0te2ErzIW53k8m6fPcvb/D42kYAWmIpvnjfCh5fs7snIK4qDALwk6tmMaeuBICVO9p75yRFREREZNDqs0DVGOMBfgxcBEwBPmiMmdJXxxOR3uNxDNfMq6O2JMy8MWV8633TAehMZJhUVchPrprF3NGlJDOHno03kc5y13NbuPgHT/PlP6ziituf65m59+n1TfzT71/l8bWNdCUzfO6CiVw6YzgA9yyr54Y7X6K9Ow3AvR8/A4CLp1ez5KNz8XkM//vaTjJZt4/fAREREREZSH3Zono6sMFau8lamwLuARb34fFEpI9UFgYJ+TwAnD2unLDfy8wRxXQmModM/42H1vD//riajGv56PzRbGqO8fHfvMx//30z19zxIvcv38HHfvMyJWEf1501iu9dMeOA129tiTNvTCm1JeGedQGvh8pokEdWNXDLw2v77mRFREREZMCZvpqYxBhzOXChtfaG/PI1wFxr7U0HpbsRuBFg2LBhs++5554+yU9v6OrqoqCgYKCzIQNAZQ9r9mTZ0+0yv9YHwIMbU9y/Ps0vzg/jdUxPuqxr+fgTcVJZGBF1+MaZQf53U5r71+daSSeXOqxtybWIXj7exyVj/QC8sCvDz1ckyeQvSfNrvFw//cCuxS81ZPjRq0kM8O/zQ1RF9v3W1p2xuBYiPkNvUtkPXSr7oUtlP3Sp7IculX3/OPfcc1+21s45mrTevs7MW7HW/gz4GcCcOXPswoULBzZDR7B06VIGc/6k76jsYeFBy1v9W7h//WpOPf1MyvJjVetb4mzdEyeVfYHvXTGDxTNr8DiGM+dnuf8rjwLwrQ+eweIf5yZO+vz7z6EyPw51IfCJ96aZ/a+Pk8q4nH3KOBYuGPumPFz7riQLv/Mkz3aU8MN3ndqz7UM/f57udJY/fOKsXj1vlf3QpbIfulT2Q5fKfuhS2Q8+fRmo7gBG7Ldcm18nIieBaDB3+ehMZCgrCLClOcbC7y7t2X5KbRGefEtrwOvhwZvOwu91mFRVyM+umc2m5lhPkLpvnz5+eOWprNrRzofPGHXI41ZEA7x3Vg33vbyDRDpL0Ofhtfo2nt24B8dAZyJNNOjrk3MWERERkf7Rl2NUlwHjjTGjjTF+4ErgwT48noj0o73B4EU/eJqsa7nzua37tgW8jC4/sPvMKbXFTKoqBOD8qVV87KDW0r0unFbFP18wkZDfc9hjXzC1iu50lu/8eR0bGjv5xTObAXAtvLKt7YTOS0REREQGXp+1qFprM8aYm4A/Ax7gl9ba1X11PBHpX6WRXKDanc7y0IqdPLmukfnjy7l8di3zxpT1tKb2hbmjywC445nN3JEPUq+YU8u9L2/npa2tnDOhos+OLSIiIiJ9r0/vo2qtfdhaO8FaO9Zae0tfHktE+tepI0r4zfVziQa8PLKygc3NMRZMqGDxzBqGHdSlt7f5vQ7jKw9ssX3H5GFMri7kpS0tfXpsEREREel7fRqoisjJy3EMZ48vp7Y0zHOb9gAwrrL/Zsv72Yfn8O3Lpvcsz6gtZk5dCa/Wtx32/q4iIiIi8vagQFVETkh1UZD27tytZ0aVRfrtuKPLI3zgtJHMH18OQFVRkPOnVhFPZfngz56nqTPZb3kRERERkd6lQFVETkhVUa6br9cx1JaE+v34v7h2Diu+fj4AZ44t4+LpVSzf1sbtT23s97yIiIiISO9QoCoiJ2R4PlAtLwjg9fT/JSXg9VCYn4HYGMNPrprN4pnD+d2yejoT6X7Pj4iIiIicOAWqInJCisN+AM6bUjnAOdnnurNG05XMcO/L2wc6KyIiIiJyHBSoisgJuXTmcG6+dCpfvWTqQGelx8wRxcyuK+FXz24h61oA1jV0csuf1miiJREREZG3gT67j6qIDA2FQR/XnjlqoLPxJtefPZpPLFnOabc8TjyVIZF2AUhlXG5ePG2AcyciIiIiR6IWVRE5KV00rYp/XDCGlliqJ0gF+PVzW3lk5a4BzJmIiIiIvBUFqiJyUjLG8KWLJvPrj5zOkhvmAvCu6dUUBLx88u7lNHfp9jUiIiIig5W6/orISW3BhAoAHvjkWUyqirJlT4wLv/80v352C589f+IA505EREREDkUtqiIyJMwcUUzQ52FSVSHnTKjgP/+6gXO/u5TfLds20FkTERERkYMoUBWRIeeX187h2jPq2NnWzRfuW8nfNzQPdJZEREREZD8KVEVkyPF6HG5ePI3XvnY+I0vD/PP/vMb21vhAZ0tERERE8hSoisiQFfR5uP3q2XQlMnxyyXJWbm/vue+qiIiIiAwcBaoiMqRNGV7IrZefwhu7u3j3j57hgu//jfbu9EBnS0RERGRI06y/IjLkXTS9mjPHlvPbZdv4zp/XcfEPnubqeXXMrithT7f71js4Dol0lvuWb2dKdSGTqgoJ+T0A3PX8VpY8v5VU1sXvcfju+2cwraaoT/IgIiIiMlgpUBURAYrCPj62YCzThhfx4yc38O1HXwfAAM90vMr759Qyd3QZGdfl1kfXsa6hk8nVUeaMKqW2JMTU4ccWTP7+pXq++sfVuWOHfCy5YS6bm2N87Y+rGF8ZJRr0sampi6vveIEvXzyZd0weRjrrEg16Cft16RYREZGTm77tiIjs5+zx5Zw9vpwtzTG2tsT57V+X8+iqBv7wyg4cA8YYsq6ltiTEMxua+fnTmwH4x3PGUBLx8w9njiLo87zlce5bvoPSiJ+bL53Kzf+7mkv+85mebb/7x3kUh/1s2xPnip8+x+fuXdGzrTIa4L6Pn8mI0nDvn/xbaO9OE/A6R3V+IiIiIidCgaqIyCGMKo8wqjyC3Rngto/M56+vN7J2VweutZw6soR3ThlGY0eC5dta+d5jb/DTv20C4MnXG7n+7NGcP7XqgP11JTPceOdL3LRoHJXRIK/Vt/GVd03m3TOG096d5isPrALgYwvGUhz2AzCyLMxj/3QOq3d28IunN5HMuDyzoZn5tz7JrZedwhWnjejX92TmN/7CjNpiHvjkWf16XBERERl6FKiKiLyFkN/Du06p5l2nVB+wvrIwyIXTqpk7uoxNzTE2NHby1T+u5sa7XubSGcMpDvtYPLOGWSOLufXR13l24x6Wb2vl8tm1OAYunTEcgMtn17Jiexs3zB/DhGHRA44RDfqYN6aMeWPKAFi+rZVvPfw6n79vBf/+yFoWTKjg0+dNYFRZmKauJAGPh4/8ehkFAS8fOG0EWdfyzinDTrgVtL4ljrXwan0bu9q7qS4KndD+RERERI5EgaqIyAkqifiZHfEzu66ExTNr+Oz/vMbT65uIpbI8vmY3751Vw53PbQUglXH5zfPbOHNsGZWFQSB3m5xbL59xVMeaNbKEu244nduXbmJrS4w/rdjFA6/uxO91SGVcwn4P8VQWgKfeaAIg5PNw9byR3LRoPEUh33Gd4983NPc8f8d/PMVNi8ZRXhAgmc5y9bw6jDHHtV8RERGRQ1GgKiLSi4I+Dz/+0CwAnlzXyEd+tYwfP7mRurIwT/zTAh5e1cCnfvsKHz6j7riPEfB6+PR54wH4zHkTeHztbrY0x6gpCfHk600MLw7xhYsmsmZnB+ms5eGVu7jjmc28sLmFi6ZVE/I5nDmunHUNnaxv7GJKdZRFk4bR3p2mvMDfE3TGUxk2NsZo6krwwKs7qCkOcdf1p/Ovf1rLrY+u68mPBT50+kherW9j1sgSHEdBq4iIiJwYBaoiIn3k3ImVPPLp+Xz/sfV8cO5IvB6HS2cMZ8H4CorCx9eyebARpWGuO2t0z/KN54zteV45Mddi+84pw5g/vpzP3buCFdvbj7i/r797Cv+Q399nf/8aj6xq6Nl207njGFNRwB3XzuHV+jYcY/jGQ2v46h9X98xg/OWLJ/PRc8b0yrmJiIjI0KVAVUSkD02qKuT2a2YfsK63gtRj8b5ZtbxzyjBcC7FkhodW7KQo5ON9s2r53bJ6Xt7ayh9e2cG3H13HA6/uxDGwfFsb02oK+dwFk3h9VwdXzMlN3mSM4dSRJQAsuWEuD762k9fq21jywja+9ejrOI7h3adUEwl4Cfs96hYsIiIix0yBqojIEBEN5gLkopDvgJbXq+fVcfW8Om5aNI4fPrGelliKzc0xAP79vacwvbaIBRMqDrnPoM/DFXNGcMWcEXzp4sl86rev8M2H1vDNh9YAUBrxE/Q6uBYKgl4KAl5GlIapLgqydU+MPV0ppgwv5P2zRzCqPExnIsOu9gRdyQxZ1yXo9bC9rZsJw6KMqyygIHD4f1uJdJZEOtsza7L0jkdX7eLhlQ0EvA4jSsN86h3jBzpLIiIyBChQFRERAMZWFPCDK08FwFpLSyxFWUHgqF9fEPDy8w/P4ZkNzWxriRNPZtjQ2IUFXGvpSmSIp7Ks2N7Go6u6qSgIUFsS5p5l9T2TTR1JWcTP/PHluBa2t8ZpjaepKQ6RyriURHz8efVuHAMXTa+mqTPJNfPqOGdCxXFPIPV2lsm6tMbTZF1LVVGuC7i19oDW7XgqQyyZ5dmNzQS8DosmDcPvdQDoTmUJ+T28vLWVTyxZTiTgpTORAaChI8HskSVcNru2/09MRESGDAWqIiLyJsaYYwpS9/I45rCtr/tzXYsxueO0xlIsfaOR5s4UIb+HmuIQBUEvWdeys62bqcOLWLurgz++uoOXtrbiupba0jDjKgvY3tqN1zG8vrGDgNdhek0RD6/chbXw4uYWQj4P88eXk3Etp48uZVJVlOauFLs7Evg8hpAvN0vyjBHFTKspwucx+D3OcXdXttayqTlGVyLDqLJIn3bz/v2yeu5/ZTuum/shwM0Hos1dSbbuifekC/s9lBX4aWhPML2miHgqy+sNnW/a3/Ci3O2Wlm1pYeWOds6dWEFzV4rKaJDHP7uARDrLBbf9jbtf2MbdL2wj4HO45JThfXZ+IiIytClQFRGRfrf/zMAlET/vPfXIrXMTq6K859Saw25PZVy6khlKI35c15LMuLy2vY1fPL2Z9Y1dpLMuf3298ajz5/c4OLiE/vYXfB4Hvzf/8DgUhnwYYGNTLBdsA44xGAPprEtzVwoAY6AsEqAyGmD+hHLOHleOYwzJTJZYMkssmSGWynVXDvk8VBYGCHg9dHSnKQr5WLG9jXgqSyKTpTvlEvA51JaEGF0W4fG1jdy3fDsThhVQFgngcwyOMbjWUlcWYfHMGsoL/CTTLjvautkTS5EZ7rK9tZuyAj/FYR+TqwqZP6GcopCPgoCXWx9dx90vbiXrWiYOi7L0jSashW9fNp2CQK7b9tLPLaShPcHHlyzn24++3tPK2plIM2FYlLqyCKPLI0f9PouIiByOAlUREXnb83sdSr25samOYwj5PcwbU8a8MWVArqVzza4OEmmXsoifqqIg6axLMuOSdS1PrG2kK5kmnc0FuamMy+YtW6msHk4q4+bSZl3SGZc9sRTprMuiSRV4HIO1YG2uVdMCU6oLqSwMsKGxi90dCba1xPnpU5v46VObjvm8Qj4PIb+HoNehO52lNZ7u2XbFnFr+7b3T8XqcY97vwd2AAd6dbx11rcXrcbh/+XYeW7P7gB8RokEf0aCPf7l4Ev/n7lf40v0rD9iHMXDxtGpKIj6qi0JMqS7k3EmVx5w/ERERBaoiInLSM8YwdXjRAeuCPg/R/PMPzR35ptcsXdrAwoXTeuX4W5pjNHYmsdbi9zoUBLxEAl4ifi9+r0M8laGhI0E6awGIJzPMqish6PMcsJ9YMsNNdy/nyXVNfP7CSccVpAKH7Nq8t5XbIff3fbNqed+sQ7d0L5o0jGVfOY+2eBoL+BzDtpY4v1tWz/Ob99CZyNAWT+MY+Ok1c5hUFe2ZBdrvcdjY1EVXMkNrPEXI5yUa9OJay862BEUhHyURHyVhPwGvw862BBnXJeTz8MLmFiIBDxOGRSmN+KkoCBz3e/B2Zq2lI5GhoztNWzxNYchLXdnhW7I3NXUR9HkYXhyiK5lh2eaWns/WmIoIldEAsVT2iJOViYj0N12RRERE+tio8gijjtAlNuT3HNWY4EjAy0+vmcPujgTlxzGGuDeF/V7C/n1fIyoLg8wZVdqzXN8S5/Lbn+Wjd750wOuMybVA94aQz8PEqiiZ/AzRJZFcq3pxyEdhyIfHyXXJdozp6aLtGEhkXFpjKaqLgj1B+97Y3WDYsiXFa5n1+63LKY74GVseYdmWVra2xGjsSPZ0OS8O+2iLp1m9sx2/1+Gc8RVMrMrNVl1REGD8sCgHO1TLdns8jcWydU+curIwf1m9mynDC5lWU4TrWm55eC2PrmpgR1v3Ae/plOpCfAcF7VWFQZq7kry0tRWA6qIgrfEUibTbk8bjGIpDPtq703zgtBGcNqqUmSOKqSsL99qtpZq7krlx2+oWLiLHQIGqiIjI24g/f5uYwW5EaZgnPruQZ9Y305FIE8+PyY0lcxNNlUf9FIdz42g7E7lu1yNLw3Qmc62ErfEUnYkMw4tDBLwOjZ1JptcUURDwsnpnOx3daTY2xVjX0EnI76c7laW+JTeJ1Mp4ms5EGndvl+z9uma71uLzOEQDXvbEUoc/gQ1vHHaTY3JB4LCiINGgl90dCdY1dOJay1njytnY2MUfXtlBPJXteU1lNEBZQYCikJdE2qWjO83O9m5mjigm4veyuzNBe3ea+pbuQx6zIhqgqTMJwKkji/nwGXWURPwUBr08v6mFTflbSu1lrWXVznYyWcv7Z9cysSrKyh3tlEb8nDuxEo9j2BNLsXpnO00dSV7c0sKSF7ax5IVt+fIL8Y5Jw0hlXdY1dDK9poiKaIBFkyqZXF3Yc5xUxiWZybXG5rrOZ+lOZ/ns719jc3OMrGvZ3ZHAtfCxBWP5woUTB/+9lbvboLsV0nFIxSDVBY1rId4CbhrcLGTTUDUdZl0z0LmVQaw7laWhI3FMr4mnMj3XMoA9sRTrd3exZlcHHmPwOlBCOwUkiJgEn5o/nKhJwK5XIbYHRp0Fk9/d26cyIBSoioiISJ8oCHi5cFpVr+93XGVBr+7P5pt497b0Ln1qKQsWLDxgG8DOtgTrGzuZNbKkp/X2SPvc2Z5g/e5ONjXFWJUPrlvjacJ+D5XRAGMrC6hvidPRnaEk4mN0eQGLZ9RQGPJSFPKxuTnOjNoinnqjiYxrcV1LNOjl65dOPSDYu3Ba9XGf+6Uz9s3cnMm6rG/sYvm2Vn7z/Db+56V6AOrKItz1fG6irdsee4MJw6J0p7NYa2nsTBJPZQn6HFIZF3e/1vIzxpThOHDZrFrqW+Pc/tRGfrdsGxXRABdPr+a0UaVEg15GloYH/v7HrgvfGsH8TAaWHiawMB5wvODxgZvJBazTLgN/mPZ4mng6QzLt9oxzT2WzZN19P5YUh32MKA2TSKZIdsdIdXeR7u4gFevg5R1xQkVlLJg1jWGFwX3HzKZzQXPjmtzfcedB4M2t80NFVzLDa/VthP2enluVhf1eRpeHGVNeQMa1ZGJ7MMlOQpFC6NwF1oVIJRQefz05FhubutjTlSKTdfncvSsO6P0AYHCZYrbRbAu5zPM0V3kfp5wOkviwQBmWOt7c7cSzd3iGdQmQ3Lfh7v0S+aO5z4cCVREREZG3v4O7/zrG9Hwp3NfxF0aWhRlZdnSt2cYYaopD1BSHWDjxxPJ30fT++YLt9ThMri5kcnUhV82tO2BbIp2lozvNT5ZuZFtLnIJAblxxWcRPTUmILXviRANeHMfwl9UNfHPxNM4cV97zemstM0cU88TaRuKpDN9/fP0B+48GvQS8HgqDXpIZl2k1hZSE/fi9DoH8rNsBr+eA536vg89jMMZQEvaRyVoCPoeSsJ9M1pLIZHHyt8EyQGNnkvbuNNZa0llLZTRARTRANOgjkUpTOeFKdu/eTfXYafiLqwmEC2nodujIBmjzV/PYDg+bm2PEU1mmdj3HN2Lf4Gv/cRsbbQ0dnR2MNTs5zVlHoYkTJU6hiWOADhthnLODImL4SFNgMm967/eOhu98MkSHcXAxeHEpIH5Auozx0xodR9JbSNwpIOmJ4BovgVQbaU+QSHkdTnENnkAELxk8NovHTeJxU3jIki2bQLu/it1dLgnr4Dp+rOMja3xY48HYLFgXY7MY64KboTuVIpVKY90sIS9UF/qJJxKkUmkKAoaIF8pCHqybIZ1O05QJ4SNNxI3hKyjBHwgRKK1lRydsau4ikc7QncoSS2XpTLp0Ji1OJk5pYQG1ma2YTDedsTiV2d0UxLcTa28i4YniOn6au7PEMw4Z65DJhzEznQ20mg4et1FCJJnvWXXIz3ciUEbWeDHW4svGaC6ZxUbvWJzyscyYOJ5o0xq616ZwLXgdQ2NXmljTVioLvGwsW8SLuy33v1zPpMoAF80YwdhImjGe3WSLRxH2AA0r2frkL9mwvYm/u1P5uzuNDmcY//qemT3jv73pGGc8fyNlLa/05KupfC7biqfiZHPBp3EcCkMBTP4a5HUcCoPeA3siFNVCqAT8kdzDF4Hy8RDeN/ziZKBAVURERESOKOjzEPR5+PqlU98y7RcunPSmdcYYrjtrNNedNRqA1liK1Ts76ErmujnWt8ZJZ13au9MYDGt3dRBLZXpaJ/fO0N23FuX+1B+8PgtsJ+hzmFRVSEHAS0f5LIjBzclbc0nyQ8ZTvkJSgTKy/ihZfxUGiy/ZSjIyk12+auJZD8YXBF8Y/BGcQAQnWEh1gUOqbRc7t7xOMp3FYElkYVt3gA5TSCJQzsYOh9PTyxjdupNC00wh26g0cRxcYoQpIkVl48N4zJHfp2Kg7ogpTszhbjZWBEw5xn3FbYC4t4hQKpYLvMng9WYPSJMqqKHFV82oZBs+N8ny0qvY7h/FrsYmNsXDtKUMdaaB0ZkGPOTGZ6fwcsbuNcwzf8e7y4WVMBtg9b79jtjvGKXADOvhkyYLXZDZ6OA1LgeL2gJm+sJc4ObG5lvjYJZGYcaHoL0eXn8IMHD+LeANgD9CxfT3U+Hpu3tuv50pUBURERGRflUS8XP2+PK3TrifTNYllXVJpvf/myXjWroSGTyOIZF2aYml8Hsdgj6nZ3xyrvU3QGm+y7bf69DYkaSxM0FXMkPI58HrMaxYsZIRYyfRlcyws62baTVFlBcESGayTK8pOnDSs2Xfy41hLarNdQmOVOAfMRe/8+aZqAuAsqM4x7fqKJ/OusSSGRzHEPF7MeR6AlQYQ3s8zcqmdhItO0inEmSsQxoPaeMngZ+saylsXUup6aQsBEHHxWRTGDedf2TA8WCNA8abf+7B7/Pi9/kwjpd4xrInliUS8uPz+YlloLXbJZbOtQR6vF4qPXHSTpCYCZGJt5FJJvB07cJvchNqBbwOPo+HgNfg5Ftt8YWwyU7SFdOwvggBvx9K6ghHKggfPKbZ2ly3azcNbga/v4Cq/dLMyj8g1xNg6544Po/B53HweRwslu5UFtdCs9dl55Z11O/YQf32ekbU1mDIzbxeUxygIBxiS2uKGcmXKfQkKSssIGt8tLbuoTEb5dnWInZvXUPaetlkq3nZncCnLprJx6Yb2PocZvcq2PgkvPBfuR8nAMrGwZk3HcWnQRSoioiIiMig5/U4eD0OvTWcNTcW9MDbVjkNa1k4+3Btggc57freycgx8Hmcw47nLQr7mFlXDnVH+gHgxPqhFwB9dWdkAxxV0RoDHm/u8RaC+ZnBj6Sq9DRmzTqNpUuXsnDhwjdtPxWAS3qWvUBF/jGVfd3iH3xtJ0//aS1njCmD0mIoHbNvJ91tgIXfXAbn/stb5ltyFKiKiIiIiIgch73d4q8/ezSLJlUypuIQk72FinN/P/rX/s3c29zQu0u2iIiIiIhILzLGHDpIleOmQFVEREREREQGFQWqIiIiIiIiMqgoUBUREREREZFBRYGqiIiIiIiIDCoKVEVERERERGRQUaAqIiIiIiIig4oCVRERERERERlUFKiKiIiIiIjIoKJAVURERERERAYVBaoiIiIiIiIyqChQFRERERERkUFFgaqIiIiIiIgMKgpURUREREREZFBRoCoiIiIiIiKDigJVERERERERGVSMtXag89DDGNMEbB3ofBxBOdA80JmQAaGyH7pU9kOXyn7o/qaLSgAAB9lJREFUUtkPXSr7oUtl3z/qrLUVR5NwUAWqg50x5iVr7ZyBzof0P5X90KWyH7pU9kOXyn7oUtkPXSr7wUddf0VERERERGRQUaAqIiIiIiIig4oC1WPzs4HOgAwYlf3QpbIfulT2Q5fKfuhS2Q9dKvtBRmNURUREREREZFBRi6qIiIiIiIgMKgpUj5Ix5kJjzDpjzAZjzBcHOj/Se4wxI4wxTxpj1hhjVhtjPp1fX2qMecwYsz7/tyS/3hhjfpj/LKwwxswa2DOQE2WM8RhjXjHGPJRfHm2MeSFfxr8zxvjz6wP55Q357aMGMt9yYowxxcaYe40xrxtj1hpjzlC9HxqMMZ/JX+9XGWN+a4wJqt6fnIwxvzTGNBpjVu237pjruTHm2nz69caYawfiXOTYHKbsv5O/5q8wxvzBGFO837Yv5ct+nTHmgv3WKwYYIApUj4IxxgP8GLgImAJ80BgzZWBzJb0oA3zWWjsFmAd8Ml++XwSesNaOB57IL0PuczA+/7gR+K/+z7L0sk8Da/db/jZwm7V2HNAKXJ9ffz3Qml9/Wz6dvH39AHjUWjsJmEHuM6B6f5IzxtQAnwLmWGunAR7gSlTvT1a/Ai48aN0x1XNjTCnwNWAucDrwtb3BrQxqv+LNZf8YMM1aewrwBvAlgPz3viuBqfnX/CT/I7ZigAGkQPXonA5ssNZustamgHuAxQOcJ+kl1tpd1trl+eed5L6s1pAr41/nk/0aeE/++WLgTpvzPFBsjKnu52xLLzHG1ALvAn6RXzbAIuDefJKDy37vZ+Je4B359PI2Y4wpAs4B7gCw1qastW2o3g8VXiBkjPECYWAXqvcnJWvt34CWg1Yfaz2/AHjMWttirW0lF+wcHADJIHOosrfW/sVam8kvPg/U5p8vBu6x1iattZuBDeS+/ysGGEAKVI9ODVC/3/L2/Do5yeS7dJ0KvAAMs9buym9qAIbln+vzcHL5PvB5wM0vlwFt+/0j2798e8o+v709n17efkYDTcB/57t9/8IYE0H1/qRnrd0BfBfYRi5AbQdeRvV+KDnWeq76f3L6CPBI/rnKfhBSoCqSZ4wpAO4D/q+1tmP/bTY3PbamyD7JGGMuARqttS8PdF6k33mBWcB/WWtPBWLs6/4HqN6frPJdNheT+7FiOBBBrWNDlur50GSM+TK5oV9LBjovcngKVI/ODmDEfsu1+XVykjDG+MgFqUustffnV+/e27Uv/7cxv16fh5PHWcClxpgt5LrzLCI3brE43yUQDizfnrLPby8C9vRnhqXXbAe2W2tfyC/fSy5wVb0/+Z0HbLbWNllr08D95K4FqvdDx7HWc9X/k4gx5h+AS4Cr7L77dKrsByEFqkdnGTA+PyOgn9xg6wcHOE/SS/Jjje4A1lprv7ffpgeBvTP7XQv8cb/1H87PDjgPaN+vC5G8jVhrv2StrbXWjiJXr/9qrb0KeBK4PJ/s4LLf+5m4PJ9ev8S/DVlrG4B6Y8zE/Kp3AGtQvR8KtgHzjDHh/PV/b9mr3g8dx1rP/wycb4wpybfIn59fJ28zxpgLyQ33udRaG99v04PAlflZvkeTm1DrRRQDDCija+3RMcZcTG4smwf4pbX2lgHOkvQSY8zZwNPASvaNU/wXcuNUfw+MBLYCV1hrW/JfbH5ErqtYHLjOWvtSv2dcepUxZiHwz9baS4wxY8i1sJYCrwBXW2uTxpggcBe5ccwtwJXW2k0DlWc5McaYmeQm0fIDm4DryP2Aq3p/kjPG3Ax8gFzXv1eAG8iNO1O9P8kYY34LLATKgd3kZu99gGOs58aYj5D7bgBwi7X2v/vzPOTYHabsvwQE2Ncr4nlr7cfy6b9MbtxqhtwwsEfy6xUDDBAFqiIiIiIiIjKoqOuviIiIiIiIDCoKVEVERERERGRQUaAqIiIiIiIig4oCVRERERERERlUFKiKiIiIiIjIoOJ96yQiIiJyNIwxWXK3uvKRu8XBncBt1lr3iC8UERGRAyhQFRER6T3d1tqZAMaYSuBuoJDc/ftERETkKKnrr4iISB+w1jYCNwI3mZxRxpinjTHL848zAYwxdxpj3rP3dcaYJcaYxcaYqcaYF40xrxpjVhhjxg/UuYiIiPQ3Y60d6DyIiIicFIwxXdbagoPWtQETgU7AtdYm8kHnb621c4wxC4DPWGvfY4wpAl4FxgO3Ac9ba5cYY/yAx1rb3b9nJCIiMjDU9VdERKR/+IAfGWNmAllgAoC19iljzE+MMRXAZcB91tqMMeY54MvGmFrgfmvt+gHLuYiISD9T118REZE+YowZQy4obQQ+A+wGZgBzAP9+Se8ErgauA34JYK29G7gU6AYeNsYs6r+ci4iIDCy1qIqIiPSBfAvp7cCPrLU23613u7XWNcZcC3j2S/4r4EWgwVq7Jv/6McAma+0PjTEjgVOAv/brSYiIiAwQBaoiIiK9J2SMeZV9t6e5C/hefttPgPuMMR8GHgVie19krd1tjFkLPLDfvq4ArjHGpIEG4N/6If8iIiKDgiZTEhERGWDGmDC5+6/Osta2D3R+REREBprGqIqIiAwgY8x5wFrgPxWkioiI5KhFVURERERERAYVtaiKiIiIiIjIoKJAVURERERERAYVBaoiIiIiIiIyqChQFRERERERkUFFgaqIiIiIiIgMKgpURUREREREZFD5/92hbQoJyxkHAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1152x504 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(16, 7))\n",
    "plt.plot(data, label='Actual')\n",
    "plt.plot(predictions, label='Predicted')\n",
    "plt.legend()\n",
    "plt.xlabel('Days')\n",
    "plt.ylabel('Price')\n",
    "plt.grid()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see in the above plot, the actual value is shown in red color and the predicted value is shown in blue color. As we are making predictions for the time step greater than 1000, you can see after the time step 1000, red and blue lines interest each other, which implies that our model has correctly predicted the actual values.\n",
    "\n",
    "In the next section, we will learn about the GRU cell which acts as a simplified version of GRU cell."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "btc.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}