MNIST_Classification/mnist_functional_model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## MNIST Classification using Functional Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The TensorFlow Keras API can also be used to construct more complicated networks using the Functional Model. This may look a little bit confusing at first, because each call to the TensorFlow Keras API will create and return an instance that is itself callable. It is not clear whether it is a function or an object - but we can call it as if it is a function. This allow us to build computational graphs that are more complex than the Sequential Model allows."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rest of the things remain same. Only change can be visible in defining the model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1. Import required Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TensorFlow version  1.12.0\n",
      "TensorFlow Keras version  2.1.6-tf\n"
     ]
    }
   ],
   "source": [
    "# TensorFlow and tf.keras\n",
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "from tensorflow.keras.layers import Input, Conv2D, MaxPooling2D, Dropout, Flatten, Dense\n",
    "from tensorflow.keras.models import Model\n",
    "from tensorflow.keras import backend as K\n",
    "# Images, plots, display, and visualization\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib import style\n",
    "import seaborn as sns\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import IPython\n",
    "from PIL import Image\n",
    "\n",
    "import math\n",
    "\n",
    "print(\"TensorFlow version \", tf.__version__)\n",
    "print(\"TensorFlow Keras version \", keras.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "style.use('ggplot')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2. Load the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "mnist = keras.datasets.mnist\n",
    "\n",
    "\"\"\"Load the data\"\"\"\n",
    "(train_images, train_labels), (test_images, test_labels) = mnist.load_data() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Dataset | Images | Labels |\n",
    "| ------- | ------ | ------ |\n",
    "| <b>Train</b> | 60000 | 60000 |\n",
    "| <b>Test</b> | 10000 | 10000 |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3. Reshape our images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_images = train_images.reshape(train_images.shape[0], 28, 28, 1)\n",
    "test_images = test_images.reshape(test_images.shape[0], 28, 28, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4. Normalized our data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "max_pixel = 255\n",
    "train_images = train_images / max_pixel\n",
    "test_images = test_images / max_pixel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 5. Build the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create an input layer\n",
    "inputs = Input(shape=(train_images[0].shape))\n",
    "\n",
    "# First convolutional layer with ReLU-activation and max-pooling\n",
    "x = Conv2D(filters = 16, kernel_size = (5, 5), strides = 1, padding = 'same', \n",
    "                     activation = 'relu', name = 'layer_conv1')(inputs)\n",
    "x = MaxPooling2D(pool_size = (2, 2), strides = 2)(x)\n",
    "\n",
    "# Second convolution layer with ReLU-activation and max-pooling\n",
    "x = Conv2D(filters = 36, kernel_size = (5, 5), strides = 1, padding = 'same', \n",
    "                     activation = 'relu', name = 'layer_conv2')(x)\n",
    "x = MaxPooling2D(pool_size = (2, 2), strides = 2)(x)\n",
    "\n",
    "# Add Dropout layer\n",
    "x = Dropout(rate = 0.5)(x)\n",
    "\n",
    "# flatten the output of the conv-layer\n",
    "x = Flatten()(x)\n",
    "\n",
    "# first fully connected dense layer with ReLU-activation \n",
    "x = Dense(units = 128, activation = 'relu')(x)\n",
    "\n",
    "# Last fully connected dense layer with softmax-activation\n",
    "predictions = Dense(units = 10, activation = 'softmax')(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 6. Model Compilation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have now defined the architecture of the model with its input and output. We now have to create a Keras model and compile it with a loss-function and optimizer, so it is ready for training."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a new instance of the Keras Functional Model. We give it the inputs and outputs of the Convolutional Neural Network that we constructed above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model = Model(inputs = inputs, outputs = predictions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model.compile(optimizer = tf.train.AdamOptimizer(),\n",
    "             loss = 'sparse_categorical_crossentropy', # used in case of classification problems\n",
    "             metrics = ['accuracy']) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         (None, 28, 28, 1)         0         \n",
      "_________________________________________________________________\n",
      "layer_conv1 (Conv2D)         (None, 28, 28, 16)        416       \n",
      "_________________________________________________________________\n",
      "max_pooling2d (MaxPooling2D) (None, 14, 14, 16)        0         \n",
      "_________________________________________________________________\n",
      "layer_conv2 (Conv2D)         (None, 14, 14, 36)        14436     \n",
      "_________________________________________________________________\n",
      "max_pooling2d_1 (MaxPooling2 (None, 7, 7, 36)          0         \n",
      "_________________________________________________________________\n",
      "dropout (Dropout)            (None, 7, 7, 36)          0         \n",
      "_________________________________________________________________\n",
      "flatten (Flatten)            (None, 1764)              0         \n",
      "_________________________________________________________________\n",
      "dense (Dense)                (None, 128)               225920    \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 10)                1290      \n",
      "=================================================================\n",
      "Total params: 242,062\n",
      "Trainable params: 242,062\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "\"\"\"Look at the model summary\"\"\"\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 7. Train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 48000 samples, validate on 12000 samples\n",
      "Epoch 1/5\n",
      "48000/48000 [==============================] - 112s 2ms/step - loss: 0.2897 - acc: 0.9094 - val_loss: 0.0778 - val_acc: 0.9769\n",
      "Epoch 2/5\n",
      "48000/48000 [==============================] - 114s 2ms/step - loss: 0.0914 - acc: 0.9716 - val_loss: 0.0532 - val_acc: 0.9841\n",
      "Epoch 3/5\n",
      "48000/48000 [==============================] - 113s 2ms/step - loss: 0.0681 - acc: 0.9781 - val_loss: 0.0417 - val_acc: 0.9868\n",
      "Epoch 4/5\n",
      "48000/48000 [==============================] - 117s 2ms/step - loss: 0.0520 - acc: 0.9839 - val_loss: 0.0422 - val_acc: 0.9879\n",
      "Epoch 5/5\n",
      "48000/48000 [==============================] - 119s 2ms/step - loss: 0.0470 - acc: 0.9845 - val_loss: 0.0379 - val_acc: 0.9892\n"
     ]
    }
   ],
   "source": [
    "EPOCHS = 5\n",
    "BATCH_SIZE = 128\n",
    "history = model.fit(x = train_images, y = train_labels, epochs = EPOCHS, validation_split = 0.2, \n",
    "          batch_size = BATCH_SIZE, verbose = 1 )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's visualize the model training process"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>val_loss</th>\n",
       "      <th>val_acc</th>\n",
       "      <th>loss</th>\n",
       "      <th>acc</th>\n",
       "      <th>epoch</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.077826</td>\n",
       "      <td>0.976917</td>\n",
       "      <td>0.289672</td>\n",
       "      <td>0.909396</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.053218</td>\n",
       "      <td>0.984083</td>\n",
       "      <td>0.091409</td>\n",
       "      <td>0.971562</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.041656</td>\n",
       "      <td>0.986833</td>\n",
       "      <td>0.068106</td>\n",
       "      <td>0.978146</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.042182</td>\n",
       "      <td>0.987917</td>\n",
       "      <td>0.052022</td>\n",
       "      <td>0.983896</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.037875</td>\n",
       "      <td>0.989167</td>\n",
       "      <td>0.047025</td>\n",
       "      <td>0.984500</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   val_loss   val_acc      loss       acc  epoch\n",
       "0  0.077826  0.976917  0.289672  0.909396      0\n",
       "1  0.053218  0.984083  0.091409  0.971562      1\n",
       "2  0.041656  0.986833  0.068106  0.978146      2\n",
       "3  0.042182  0.987917  0.052022  0.983896      3\n",
       "4  0.037875  0.989167  0.047025  0.984500      4"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hist = pd.DataFrame(history.history)\n",
    "hist['epoch'] = history.epoch\n",
    "hist.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x250ef498f60>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD8CAYAAAB3u9PLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X98U+Xd//HXdRJaKLU/kpZ2heJspSrCRiEoAwer7V23\nwVh1zFuYOlcdat06UabgmDi1iENW1vuG4UYXb2VuuHtbHVO5u+It3eg9bWn65YcMWqZOR2shqVUK\nBdJzvn8EQmPBJG3TpO3n+XjwYOm5Ts77XLLz6blyrlzKMAwDIYQQw54W7gBCCCEigxQEIYQQgBQE\nIYQQZ0hBEEIIAUhBEEIIcYYUBCGEEIAUBCGEEGdIQRBCCAFIQRBCCHGGFAQhhBAAmMMdIFiHDx/u\n1X5JSUkcPXq0n9P0neQKjuQKjuQKTqTmgr5lS0tLC6id3CEIIYQApCAIIYQ4QwqCEEIIYBB+hiCE\nGH4Mw6CzsxNd11FKhew477//PidPngzZ+/eFv2yGYaBpGiNHjux1H0lBEEJEvM7OTkaMGIHZHNpL\nltlsxmQyhfQYvRVINrfbTWdnJ6NGjerVMWTISAgR8XRdD3kxGArMZjO6rvd6fykIQoiIF8phoqGm\nL301LAqC0fgmHb9/LtwxhBAiog2PglBfw7HNGzEO/T3cUYQQImINi4KgvroIzZKM/tx6DLc73HGE\nEEPchAkTLrjt3Xff5dprrx3ANIEbHgVhZAwXffs++Nc7GFUvhjuOEEJEpGHzsf3Iq2fTPmUGxtZf\nY0ybhUpODXckIUQv6L/5Bca7b/Xre6r0S9Bu+vYFt69atYq0tDRuu+02ANauXYvJZKKmpob29nbc\nbjcPPPAA1113XVDH7ezsZPny5ezevRuTycTKlSuZNWsWBw4c4L777uPUqVMYhsHPf/5zxo4dyx13\n3EFzczO6rvO9732Pr371q3057R6GTUEA0BZ+G/3h76A/vxGteKU8uSCECMj8+fNZuXKltyBs3bqV\nX/3qV9x+++1cdNFFuFwuvvKVr5Cfnx/UdeWZZ55BKcX27dtpampi4cKF/OUvf+G5557j9ttv54Yb\nbuDUqVN0dXXxv//7v6SmpvLcc54HZD788MN+P89hVRCUJRlV8A2MLZsw6naipl8T7khCiCB90m/y\noTJp0iSOHj1KS0sLTqeT+Ph4xowZwyOPPMLrr7+OUoqWlhaOHDnCmDFjAn7f2tpavvWtbwFw6aWX\nMm7cOP7xj38wbdo0ysrKaG5u5ktf+hIZGRlcccUVrFy5kpKSEvLy8rj66qv7/TyHxWcI3alr58LF\nl2Js+QXG8WPhjiOEGCTmzZvHSy+9xB//+Efmz5/P73//e5xOJ6+88gp//vOfSUpK6revvbj++uux\n2+2MHDmSW265hb/+9a9kZmaybds2Lr/8cn784x9TWlraL8fqbvgVBM2EdksRfNiO8QeZmyCECMz8\n+fN58cUXeemll5g3bx4fffQRSUlJjBgxgp07d/Lee+8F/Z5XXXUVf/jDHwA4dOgQ//rXv8jMzOSd\nd97h4osv5vbbb+e6665j//79tLS0MGrUKL72ta9x1113sWfPnv4+xeE1ZHSWuvhSVO48jO1bMWbk\noDIvD3ckIUSEu+yyy+jo6CA1NZWUlBRuuOEGvvnNb5Kbm8tnPvMZLr300qDf85vf/CbLly8nNzcX\nk8lEaWkp0dHRbN26ld/97neYzWbGjBnDd7/7Xfbu3cuPfvQjlFKMGDGCJ554ot/PURmGYfT7u4ZQ\nf62YZnQeR3/4OxAzGm1FKSpM35MSqSs0Sa7gSK7gBJvr+PHjxMTEhDCRh9lsxh2hc5UCzXa+vpIV\n0/xQI2PQFi2WuQlCCHFGQL8WNzQ0YLfb0XWd3NxcCgoKfLYbhoHdbsfhcBAdHU1RUREZGRkcPnzY\n54OP1tZWbrzxRubOncsLL7zA9u3biYuLA2DhwoVMnTq1H0/NPzVlBsjcBCFECOzfv5/i4mKfn0VH\nR/OnP/0pTIn881sQdF2nvLycFStWYLVaWb58OTabjXHjxnnbOBwOWlpaKCsro7GxkU2bNnkncqxZ\ns8b7PnfeeSdXXXWVd7+5c+cyf/78EJxW4GRughCRb5CNbANwxRVX8Oc//3nAj9uXvvI7ZNTU1OT9\nEMVsNjNz5kxqa2t92tTV1TF79myUUmRlZdHR0UFbW5tPmz179pCamkpycnKvw4bC2bkJ7K3HqNsZ\n7jhCiPPQNC1ix/YjidvtRtN6/0mA3zsEl8uF1Wr1vrZarTQ2NvZok5SU5NPG5XKRmJjo/dnOnTuZ\nNWuWz37btm2jurqajIwMbr31VmJjY3scv6qqiqqqKgBWr17tc5xgmM3mC+5rfP2buOr+iv7bciyz\nc9FGX9SrY/R3rnCSXMGRXMEJNpdhGLhcrpAXBV3XI/ZuJJBsI0aMICUlJbKX0HS73ezatYtFixZ5\nf5afn8+CBQsA2LJlC88++yxFRUU99s3LyyMvL8/7urdPTPh7qsFYuBi9ZClHN61D+8bdvTpGKHKF\ni+QKjuQKTm9zhXp5y0jtLwgsm2EYOJ3OHj/vt6eMLBaLzwGcTicWi6VHm+5BP97G4XBwySWXkJCQ\n4P1ZQkICmqahaRq5ubkcOnQooMCh4p2bsGObrJsghBiW/BaEzMxMmpubaW1txe12U1NTg81m82lj\ns9morq7GMAwOHjxITEyM3+Gi7p8xvPHGG6Snp/f1XPpMfXURJFjRN2+QdROEEMOO3yEjk8lEYWEh\nJSUl6LpOTk4O6enpVFZWAp6hn+zsbOrr6ykuLiYqKspn6Kezs5Pdu3ezePFin/fdvHkzb7/9Nkop\nkpOTe2wPh7NzE/T1qzC2/xF13Q3hjiSEEANm2M5U/iRd61fBm/VoP1qPSkrp1fFCkWsgSa7gSK7g\nSK7g9SWbzFTuA23ht0GZ0J9/OmKfOBBCiP4mBeE8vHMT9tTBLpmbIIQYHqQgXMDZdRP03/wC43hH\nuOMIIUTISUG4AFk3QQgx3EhB+ATn5ia8InMThBBDnhQEP2RughBiuJCC4Id33YT33sbY/sdwxxFC\niJCRghAA77oJf3we4+j74Y4jhBAhIQUhQDI3QQgx1ElBCJDMTRBCDHVSEIIgcxOEEEOZFIQgyNwE\nIcRQJgUhSDI3QQgxVElB6AWZmyCEGIqkIPSCzE0QQgxFAa2p3NDQgN1uR9d1cnNzKSgo8NluGAZ2\nux2Hw0F0dDRFRUVkZGRw+PBhSktLve1aW1u58cYbmTt3LseOHaO0tJQjR46QnJzMkiVLiI2N7d+z\nCyGfuQnTZoV83QQhhAg1v3cIuq5TXl7OQw89RGlpKTt37uS9997zaeNwOGhpaaGsrIzFixezadMm\nwLMow5o1a1izZg1PPvkkUVFRXHXVVQBUVFQwefJkysrKmDx5MhUVFSE4vdCSuQlCiKHEb0Foamoi\nNTWVlJQUzGYzM2fOpLa21qdNXV0ds2fPRilFVlYWHR0dPmsmA+zZs4fU1FSSk5MBqK2tZc6cOQDM\nmTOnx3sOBjI3QQgxlPgtCC6XC6vV6n1ttVpxuVw92iQlJX1im507dzJr1izv6/b2dhITEwFISEig\nvb29d2cQZjI3QQgxVAT0GUJfud1udu3axaJFi867XSmFUuq826qqqqiqqgJg9erVPoUnGGazudf7\n+nP6uw/heuAOol/5LXF3Lo2YXH0huYIjuYIjuYI3ENn8FgSLxYLT6fS+djqdWCyWHm26L/788TYO\nh4NLLrmEhIQE78/i4+Npa2sjMTGRtrY24uLiznv8vLw88vLyvK97u8h0SBfPjk9C5czlxP/8gZNT\nZqAyL4+MXH0guYIjuYIjuYLXl2xpaWkBtfM7ZJSZmUlzczOtra243W5qamqw2Ww+bWw2G9XV1RiG\nwcGDB4mJifEOB0HP4aKz++zYsQOAHTt2MH369IACRypV8A2It8jcBCHEoOW3IJhMJgoLCykpKWHJ\nkiV87nOfIz09ncrKSiorKwHIzs5mzJgxFBcX8/TTT3PHHXd49+/s7GT37t1cffXVPu9bUFDA7t27\nKS4uZs+ePT0eZR1sPHMT7pS5CUKIQUsZg+x5ycOHD/dqv4G6FexaXwJvOtB+tD6guQmReosquYIj\nuYIjuYIXEUNGIjjawsWgNJmbIIQYdKQg9DOZmyCEGKykIISAypkH4zNlboIQYlCRghACymRCu/Ue\nWTdBCDGoSEEIEXXxpahr58q6CUKIQUMKQgjJ3AQhxGAiBSGEZG6CEGIwkYIQYip7Bky52rNuwtH3\nwx1HCCEuSArCAJC5CUKIwUAKwgCQuQlCiMFACsIAkbkJQohIJwVhgMjcBCFEpJOCMIBkboIQIpJJ\nQRhgMjdBCBGppCAMMJmbIISIVFIQwqD73ISu1uZwxxFCCCCANZUBGhoasNvt6LpObm5uj9XNDMPA\nbrfjcDiIjo6mqKiIjIwMADo6Oti4cSPvvvsuSinuvvtusrKyeOGFF9i+fbt3LeWFCxcyderUfj69\nyKUtXIz+8D18+POnMO5chlIq3JGEEMOc34Kg6zrl5eWsWLECq9XK8uXLsdlsjBs3ztvG4XDQ0tJC\nWVkZjY2NbNq0iVWrVgFgt9uZMmUK999/P263m5MnT3r3mzt3LvPnzw/BaUW+s3MTTm0pR9u1E2zX\nhDuSEGKY8ztk1NTURGpqKikpKZjNZmbOnEltba1Pm7q6OmbPno1SiqysLDo6Omhra+P48ePs37+f\na6+9FgCz2czo0aNDcyaDkMqZhznjMpmbIISICH4Lgsvlwmq1el9brVZcLlePNklJST3atLa2EhcX\nx4YNG3jggQfYuHEjnZ2d3nbbtm1j6dKlbNiwgWPHjvXH+QwqymQiruhBmZsghIgIAX2G0FtdXV28\n9dZbFBYWMmHCBOx2OxUVFdx0003k5+ezYMECALZs2cKzzz5LUVFRj/eoqqqiqqoKgNWrV/sUnmCY\nzeZe7xtK5tRUYuYu4PhLvyXuiwVEXTYp3JGACO4vyRUUyRWcSM0FA5PNb0GwWCw4nU7va6fTicVi\n6dHm6NGjPdoopbBarUyYMAGAGTNmUFFRAUBCQoK3fW5uLk8++eR5j5+Xl0deXp73dffjBCMpKanX\n+4ZSUlISndfdAH/dTtt/rkL7wU9Q5pDW6YBzRWp/Sa7ASa7gRGou6Fu2tLS0gNr5HTLKzMykubmZ\n1tZW3G43NTU12Gw2nzY2m43q6moMw+DgwYPExMSQmJhIQkICVquVw4cPA7Bnzx7vh9FtbW3e/d94\n4w3S09MDPrmhRuYmCCEigd9fRU0mE4WFhZSUlKDrOjk5OaSnp1NZWQlAfn4+2dnZ1NfXU1xcTFRU\nlM/QT2FhIWVlZbjdbsaMGePdtnnzZt5++22UUiQnJ7N48eIQneLg4LNuwrRZqKSUcEcSQgwzyhhk\nX9B/9m4jWJF6K9g9l+E6gv7wPZA1Ce27Pwzr3ITB0F+RRHIFR3IFLyKGjMTAkXUThBDhJAUhwsi6\nCUKIcJGCEGFk3QQhRLhIQYhAsm6CECIcpCBEKFk3QQgx0KQgRCiZmyCEGGhSECKYz9yEo++HO44Q\nYoiTghDhtIWLQWnozz/NIJsyIoQYZKQgRDiZmyCEGChSEAYBmZsghBgIUhAGAZmbIIQYCFIQBgmZ\nmyCECDUpCIOIzE0QQoSSFIRBROYmCCFCSQrCICNzE4QQoSIFYRCSuQlCiFAIaPHehoYG7HY7uq6T\nm5tLQUGBz3bDMLDb7TgcDqKjoykqKiIjIwOAjo4ONm7cyLvvvotSirvvvpusrCyOHTtGaWkpR44c\nITk5mSVLlhAbG9v/ZzgEnZ2bYGwp98xNsF0T7khCiCHA7x2CruuUl5fz0EMPUVpays6dO3nvvfd8\n2jgcDlpaWigrK2Px4sVs2rTJu81utzNlyhTWrVvHmjVrGDt2LAAVFRVMnjyZsrIyJk+eTEVFRT+f\n2tAmcxOEEP3Nb0FoamoiNTWVlJQUzGYzM2fOpLa21qdNXV0ds2fPRilFVlYWHR0dtLW1cfz4cfbv\n38+1114LgNlsZvTo0QDU1tYyZ84cAObMmdPjPcUn85mbUCFzE4QQfed3yMjlcmG1Wr2vrVYrjY2N\nPdokJSX5tHG5XJhMJuLi4tiwYQPvvPMOGRkZ3HbbbYwcOZL29nYSExMBSEhIoL29/bzHr6qqoqqq\nCoDVq1f7HCcYZrO51/uGUp9yJSXx0dwFHH/pt8R/8XpGZF0ZGblCSHIFR3IFJ1JzwcBkC+gzhN7q\n6urirbfeorCwkAkTJmC326moqOCmm27yaaeUuuCC8nl5eeTl5Xlf93aR6UhdPLuvuYzrboC/bsf1\nHyVoP/gJytw//0mHan+FiuQKjuQKXl+ypaWlBdTO75CRxWLB6XR6XzudTiwWS4823YOebWO1WrFa\nrUyYMAGAGTNm8NZbbwEQHx9PW1sbAG1tbcTFxQUUWPjynZuwNdxxhBCDmN+CkJmZSXNzM62trbjd\nbmpqarDZbD5tbDYb1dXVGIbBwYMHiYmJITExkYSEBKxWK4cPHwZgz549jBs3zrvPjh07ANixYwfT\np0/v73MbNmRughCiP/gdXzCZTBQWFlJSUoKu6+Tk5JCenk5lZSUA+fn5ZGdnU19fT3FxMVFRURQV\nFXn3LywspKysDLfbzZgxY7zbCgoKKC0t5dVXX/U+dip6T1u4GP3he9Cffxrtuz+84BCcEEJciDIG\n2cyms3cbwYrUscH+zKVXvYixpRztrgdR02ZFTK7+JLmCI7mCE6m5IEI+QxCDh3duwq9lboIQInhS\nEIaQc3MTPpC5CUKIoElBGGK86ya89grGPw6EO44QYhCRgjAEeddNeG69rJsghAiYFIQhSOYmCCF6\nQwrCECVzE4QQwZKCMIR51k1Qsm6CECIgUhCGsLPrJrCnDuprwh1HCBHhpCAMcTI3QQgRKCkIQ5zM\nTRBCBEoKwjAgcxOEEIGQgjBMyNwEIYQ/UhCGCZmbIITwRwrCMCJzE4QQn0QKwjAjcxOEEBciBWGY\nkbkJQogLCWhF9oaGBux2O7quk5ubS0FBgc92wzCw2+04HA6io6MpKioiIyMDgHvuuYeRI0eiaRom\nk4nVq1cD8MILL7B9+3bvWsoLFy5k6tSp/Xlu4gJUzjyM/3sN/de/QLtiCipmdLgjCSEigN+CoOs6\n5eXlrFixAqvVyvLly7HZbN61kQEcDgctLS2UlZXR2NjIpk2bWLVqlXf7ypUrvRf+7ubOncv8+fP7\n6VREoM7OTdBLlmJUPIdadFe4IwkhIoDfIaOmpiZSU1NJSUnBbDYzc+ZMamtrfdrU1dUxe/ZslFJk\nZWXR0dFBW1tbyEKLvpO5CUKIj/N7h+ByubBard7XVquVxsbGHm2SkpJ82rhcLhITEwF47LHH0DSN\nf/u3fyMvL8/bbtu2bVRXV5ORkcGtt95KbGxsj+NXVVVRVVUFwOrVq32OEwyz2dzrfUMpnLn024tx\nNvwN7ddPY1nzS5T53D8H6a/gSK7gSK7gDUS2gD5D6IvHHnsMi8VCe3s7jz/+OGlpaUycOJH8/HwW\nLFgAwJYtW3j22WcpKirqsX9eXp5PEentItORunh22HP9+7dxb1jFkS12tOuuj5xcFyC5giO5ghOp\nuaBv2dLS0gJq53fIyGKx4HQ6va+dTicWi6VHm+5Bu7c5+3d8fDzTp0+nqakJgISEBDRNQ9M0cnNz\nOXToUECBRf+SuQlCiLP8FoTMzEyam5tpbW3F7XZTU1ODzWbzaWOz2aiursYwDA4ePEhMTAyJiYl0\ndnZy4sQJADo7O9m9ezfjx48H8PmM4Y033iA9Pb0/z0sEQeYmCCEggCEjk8lEYWEhJSUl6LpOTk4O\n6enpVFZWApCfn092djb19fUUFxcTFRXlHfppb2/nqaeeAqCrq4trrrmGKVOmALB582befvttlFIk\nJyezePHiUJ2j8OPs3ARjS7lnbsK0WeGOJIQIA2UMsl8JDx8+3Kv9InVsMFJyGV1d6KuWQnsb2qPr\nSR5/cUTk+rhI6a+Pk1zBkVzBi4jPEMTwIOsmCCGkIAiv7nMTTh/cF+44QogBJgVB+Di7bkL7f67C\n+MAV7jhCiAEkBUH4UCNj0G4rpuv9w+iPfg/jzYZwRxJCDBApCKIHdWU21jXlEBuHvm4l+ovPY+hd\n4Y4lhAgxKQjivMzjM9B+sBY14wsYf/oN+k8exmiX76cSYiiTgiAuSEWPRCtcgrrte/DWAc8Q0v7/\nF+5YQogQkYIg/NJm5aI9tBZiYtFLH0b/469lCEmIIUgKggiIGnuxZwjp6jkYW3+Nvu4RjA9lCEmI\noUQKggiYGjkKVbgEdet3oGk/+qP3Yvx9d7hjCSH6iRQEERSlFNrn89EeegpGxaD/5GH0P/1GhpCE\nGAKkIIheUeM+jfaDn6Cu+jzGi8+j//RHGB9+EO5YQog+kIIgek2NHIW6/T7PEFLjm54hpAN7wx1L\nCNFLUhBEn5wbQloDI0ehr12B/tILGLoe7mhCiCBJQRD9Qo27BG3FWtT0azAqNnuGkD5qD3csIUQQ\nAlpTuaGhAbvdjq7r5ObmUlBQ4LPdMAzsdjsOh4Po6GiKiorIyMgA4J577mHkyJFomobJZGL16tUA\nHDt2jNLSUo4cOUJycjJLliwhNja2n09PDCQ1MgbuuB8um4Tx61+gP/o9tG8vRWVNCnc0IUQA/N4h\n6LpOeXk5Dz30EKWlpezcuZP33nvPp43D4aClpYWysjIWL17Mpk2bfLavXLmSNWvWeIsBQEVFBZMn\nT6asrIzJkydTUVHRT6ckwkkphTb7i2jL10DUSPSnVqC//FsZQhJiEPBbEJqamkhNTSUlJQWz2czM\nmTOpra31aVNXV8fs2bNRSpGVlUVHR4fPmsnnU1tby5w5cwCYM2dOj/cUg5san4G24ico2yyMPzyH\n/h+PyhCSEBHOb0FwuVxYrVbva6vVisvl6tEmKSnpgm0ee+wxHnzwQaqqqrw/a29vJzExEYCEhATa\n2+ViMdSoUTGoby9FfeNu+Psez1NIjW+GO5YQ4gIC+gyhLx577DEsFgvt7e08/vjjpKWlMXHiRJ82\nSimUUufdv6qqyltIVq9e7VN4gmE2m3u9bygNi1wLbuH01KtoX7OCrqd+QOw3FhNT8A2UFvwzDcOi\nv/qR5ApOpOaCgcnmtyBYLBacTqf3tdPpxGKx9GjTffHn7m3O/h0fH8/06dNpampi4sSJxMfH09bW\nRmJiIm1tbcTFxZ33+Hl5eeTl5Xlf93aR6UhdPHvY5IqzYjy0FvVf/8Gx537GsYZatG/di7ro/P/d\nByxXP5FcwZFcwetLtrS0tIDa+f0VLTMzk+bmZlpbW3G73dTU1GCz2Xza2Gw2qqurMQyDgwcPEhMT\nQ2JiIp2dnZw4cQKAzs5Odu/ezfjx47377NixA4AdO3Ywffr0oE5QDD5qVAzqzgdQi+6C/Q3oj92L\n0SRDSEJECr93CCaTicLCQkpKStB1nZycHNLT06msrAQgPz+f7Oxs6uvrKS4uJioqiqKiIsDzOcFT\nTz0FQFdXF9dccw1TpkwBoKCggNLSUl599VXvY6di6FNKoXK+jJFxGfrTT6KveQh1/S2o/Ot7NYQk\nhOg/yjAMI9whgnH48OFe7Rept4LDOZdxvAP9v/4D6mtgsg2t8F5U7CcPIQ3n/uoNyRWcSM0FETJk\nJESoqJjRaHc9iFq4+NwQ0qG/hzuWEMOWFAQRVkoptGvnoT34JGgm9DXL0Sv/wCC7cRViSJCCICKC\n+vQEtB+WwmevwvitHX19CUbHR+GOJcSwIgVBRAwVE4t21zLUTd+GvfWeiWwyhCTEgJGCICKKUgot\n9yueISSlzgwhVcgQkhADQAqCiEjqkgloP1wHk6dj/PaXZ4aQjoU7lhBDmhQEEbHU6Fi0ouWof7/D\nM4T02L2cPigT2YQIFSkIIqIppdDy5qM96PnqdNcP7kKvelGGkIQIASkIYlBQl2Sh/bCU6OwZGFvK\n0Tc8IUNIQvQzKQhi0FCjLyJ++ZOorxfCnlrPRLa3GsMdS4ghQwqCGFSUUmj5BWjffwIMA/3JB9G3\nb5UhJCH6gRQEMSipzMvRHl4Hk6Zi/OYX6BtXYxyXISQh+kIKghi01OiL0O75Aerr34L/9wb6Y0sw\n3pYhJCF6SwqCGNQ8Q0jXe4aQ9C7PENKrf5IhJCF6QQqCGBJU5uWeiWxXTMH49c/Rn34S43hHuGMJ\nMahIQRBDhoqNQ/vOCtSC28DxN/THl2C8cyjcsYQYNPyumAbQ0NCA3W5H13Vyc3MpKCjw2W4YBna7\nHYfDQXR0NEVFRWRkZHi367rOsmXLsFgsLFu2DIAXXniB7du3e9dSXrhwIVOnTu2v8xLDlNI01HU3\nYGRejv7zp9BXfx914x2oL3wJpVS44wkR0fwWBF3XKS8vZ8WKFVitVpYvX47NZmPcuHHeNg6Hg5aW\nFsrKymhsbGTTpk2sWrXKu/3ll19m7Nix3vWVz5o7dy7z58/vx9MRwkNdOhHth+vQf1mK8fxGOLgX\nbv0OalRMuKMJEbH8Dhk1NTWRmppKSkoKZrOZmTNnUltb69Omrq6O2bNno5QiKyuLjo4O2traAHA6\nndTX15ObmxuaMxDiAtRFcWjf/SHqhm9i1Nd4JrL9U4aQhLgQvwXB5XJhtVq9r61WKy6Xq0ebpKSk\n87Z55plnuPnmm897u75t2zaWLl3Khg0bOHZMniEX/U9pGtqXvoa2dBWcPo3+xAPor70iTyEJcR4B\nfYbQW7t27SI+Pp6MjAz27dvnsy0/P58FCxYAsGXLFp599lmKiop6vEdVVRVVVVUArF692qfwBMNs\nNvd631CSXMHpda6k2egTJ9P+08c49aufEfXOQeLuXoYWMzq8uUJMcgUnUnPBwGTzWxAsFgtOp9P7\n2ul0YrFYerQ5evRojzavv/46dXV1OBwOTp06xYkTJygrK6O4uJiEhARv+9zcXJ588snzHj8vL4+8\nvDzv6+7HCUZSUlKv9w0lyRWcvuYy7lqG2vY7Tlb8iiMH96Pd+QBqfIb/HUOcK1QkV3AiNRf0LVta\nWlpA7fxsPhcIAAAOVklEQVQOGWVmZtLc3Exraytut5uamhpsNptPG5vNRnV1NYZhcPDgQWJiYkhM\nTGTRokVs3LiR9evXc++99zJp0iSKi4sBvJ8xALzxxhukp6cHc35C9IrSNLQvfx1t6eNwqhP9ie+j\n79gmQ0hCEMAdgslkorCwkJKSEnRdJycnh/T0dCorKwHP0E92djb19fUUFxcTFRV13qGfj9u8eTNv\nv/02SimSk5NZvHhx389GiACprEmep5DKSzE2b/A8hXRLEWqkPIUkhi9lDLJfjQ4fPtyr/SL1VlBy\nBae/cxm6jvHKf2O8+DyM+RTaXQ+gxl0S9lz9RXIFJ1JzQYQMGQkxlClNQ5t7I9r9j0PnCfRV30f/\nS6UMIYlhSQqCEIC6bBLaw6Vw6RUYz/4nRvlPMDpP+N9RiCFECoIQZ6i4RLR7H0F9dRHGG39BL7kP\n4723wx1LiAEjBUGIbpRmQpt3E9p9j8KJ4+irlsoQkhg2pCAIcR7q8s94VmQ7O4T0y3UyhCSGPCkI\nQlyAdwhp/iKM119DL7kf41/vhDuWECEjBUGIT6A0E9pXbkJb8igcP4a+6n70nVXhjiVESEhBECIA\n6orPoj38U7jkMoxnytB/uQ7jZGe4Yw0aRlcXRucJjI/aMZxH6Dr6vnwuE4FC+uV2QgwlKj4R7b5H\nMbZuwXhpC8bbjWh3PogaOz7c0QJmGAZ0ueH0aTh9Ek6dgtNn/pzq9rf7FMapUxduc+aP4d3nJLhP\ne/4+Tzu6unxyHAWwJKOuzEZNmgqXfxbVT180KHpPCoIQQVCayfNY6oSJ6JvWoq+6H/WNu2D+vwf9\nXoZheC6iH78Ynz7pc8E2elywz158u1+wT2Oc7nkxPtrVRVfnCd8LuqH3vgPMIyAqCkZ0+xMVDSNG\neP6OjUONiPK8HhHt27bb/46NjuKjuhqM2r9g/KUSNA0yL0ddOdVTINIzUJoMYAw0KQhC9IKaOAXt\n4Z+ib1qLYf8pH+yrR4+J9V6MjR4X91Pnfns+e7E/fRr6MmwSFeW56J69AEdFn7vwjhoN8YmYYy9C\nN4xuF/CzF2nfC7byuWifvcB3a2/27NNfF+mYpCSOX/UFDLcb/vF3jH0OjL31GBWbMSo2w0XxqIlT\nYNJU1MRsVFyC/zcVfSYFQYheUgkWtCWPYvzpN5x67WUM1LmLZ1S3355jR5656F7gN2affaJR57lg\n8/ELttkc0BrRCRH83TwAymyGrEmorElw/S0YH7ZhvNkAe+sx9jng9R0YAOMzUZOmoq6cChmXefYT\n/U56VYg+UCYT6qvfIOn270X0hXewUHGJqBk5MCMHQ9fh3X947hz21mNs+x3Gy7+FUTFw+We8BUJZ\nx4Q79pAhBUEIEZGUpsHFl6IuvhTm3ohx/Bj8fbenOOyrx3D8zXP3kDru3N1D1pWoqOhwRx+0pCAI\nIQYFFRMLU2eips70fCDf8t65u4fXXsGo+qNnWC3ryjMFYhqkjg1oaE14SEEQQgw6Sin4VDrqU+nw\nb1/FOHkSGveeuXtwYGwpx6Dc82jr2buHyz8jj7b6EVBBaGhowG63o+s6ubm5FBQU+Gw3DAO73Y7D\n4SA6OpqioiIyMs6tU6vrOsuWLcNisbBs2TIAjh07RmlpKUeOHCE5OZklS5YQGxvbj6cmhBguVHQ0\nTJqGmjQNAOPo+57CsK8e441qjOr/kUdbA+C3IOi6Tnl5OStWrMBqtbJ8+XJsNhvjxo3ztnE4HLS0\ntFBWVkZjYyObNm1i1apV3u0vv/wyY8eO5cSJc18OVlFRweTJkykoKKCiooKKigpuvvnmfj49IcRw\npJJSUHO+CHO+eO7R1rN3D90fbb0yG67Mlkdbz/BbHpuamkhNTSUlJQWz2czMmTOpra31aVNXV8fs\n2bNRSpGVlUVHRwdtbW0AOJ1O6uvryc3N9dmntraWOXPmADBnzpwe7ymEEP1Bmc2eNbRvuBXTD0vR\n1v4XqnAJauIUT5EoL0W//1a6Hr+PY5s3Yhzc5ykiw5DfOwSXy4XVavW+tlqtNDY29miTlJTk08bl\ncpGYmMgzzzzDzTff7HN3ANDe3k5iYiIACQkJtLe3n/f4VVVVVFV5vkxs9erVPscJhtls7vW+oSS5\ngiO5giO5ziMpCTImwFe+jqHruP9xgJOO1znleJ2OP/wK9C5UzGhGfGY6UdlXET3lakxjPhWerN0M\nRJ+F9EPlXbt2ER8fT0ZGBvv27btgO6XUBZ8EyMvLIy8vz/u6t896R+ri2ZIrOJIrOJIrAAnJkDMP\ncuaRPCoa51//F2NfPSf31XPyb6/xEXg+wL4yO6yPtvalz9LS0gJq57cgWCwWnE6n97XT6cRisfRo\n0z3o2Tavv/46dXV1OBwOTp06xYkTJygrK6O4uJj4+Hja2tpITEykra2NuLi4QM9NCCFCQht9EWra\nTNS0M4+2Nr977ms1hsGjrX4LQmZmJs3NzbS2tmKxWKipqaG4uNinjc1mY9u2bcyaNYvGxkZiYmJI\nTExk0aJFLFq0CIB9+/axdetW7742m40dO3ZQUFDAjh07mD59eghOTwghekcpBWnjUWnjz/Noa/25\nR1utY87dPVzxWdSomHBH7zW/BcFkMlFYWEhJSQm6rpOTk0N6ejqVlZUA5Ofnk52dTX19PcXFxURF\nRVFUVOT3wAUFBZSWlvLqq696HzsVQohIdcFHW/d2e7TVZPJ819KVUz3t0i8ZVI+2KmOQrVJx+PDh\nXu0XUWOW3Uiu4Eiu4Eiu4PQ2l++jrfXwz394NngfbZ3quYu4KH7As0E/foYghBDik/l8a+sNt3q+\ntXXfmW9t3VsPf3sNQynPt7aeKQ6R+K2tkZVGCCGGABWXiPpcDnzuzLe2/vPQuc8etv03xssveL61\n9YrPnikQU1HW5HDHloIghBChpDQNPj0B9ekJMO/fPd/aun+3pzjsq8eo/z/Pt7Z+Kv3c3UOYHm2V\ngiCEEANIxcTCxx9tPfu1Gq+9jFH1oufR1ssmnXl6yfNo60CQgiCEEGHi82hrfoHn0daDe8/dPXR7\ntPVU8QpI+3RI80hBEEKICKGio2HyNNTkM4+2Hmk5862tDrSklJAfXwqCEEJEKJWcivrCl+ALX8Kc\nlAQhflR38MyYEEIIEVJSEIQQQgBSEIQQQpwhBUEIIQQgBUEIIcQZUhCEEEIAUhCEEEKcIQVBCCEE\nMAjXQxBCCBEaw+YOYdmyZeGOcF6SKziSKziSKziRmgsGJtuwKQhCCCE+mRQEIYQQAJgeeeSRR8Id\nYqBkZGSEO8J5Sa7gSK7gSK7gRGouCH02+VBZCCEEIENGQgghzhhy6yE0NDRgt9vRdZ3c3FwKCgp8\nthuGgd1ux+FwEB0dTVFR0YDcIvrLtW/fPn784x8zZswYAK6++moWLFgQ0kwbNmygvr6e+Ph41q5d\n22N7uPrKX65w9BXA0aNHWb9+PR988AFKKfLy8vjyl7/s0yYcfRZIrnD02alTp1i5ciVut5uuri5m\nzJjBjTfe6NMmHP0VSK5w/RsD0HWdZcuWYbFYejxZFPL+MoaQrq4u4zvf+Y7R0tJinD592li6dKnx\n7rvv+rTZtWuXUVJSYui6bhw4cMBYvnx5ROTau3ev8cQTT4Q8S3f79u0zDh06ZNx3333n3R6Ovgok\nVzj6yjAMw+VyGYcOHTIMwzCOHz9uFBcXR8S/r0ByhaPPdF03Tpw4YRiGYZw+fdpYvny5ceDAAZ82\n4eivQHKF69+YYRjG1q1bjXXr1p33+KHuryE1ZNTU1ERqaiopKSmYzWZmzpxJbW2tT5u6ujpmz56N\nUoqsrCw6Ojpoa2sLe65wmDhxIrGxsRfcHo6+CiRXuCQmJnp/Gxs1ahRjx47F5XL5tAlHnwWSKxyU\nUowcORKArq4uurq6PGsIdxOO/gokV7g4nU7q6+vJzc097/ZQ99eQGjJyuVxYrVbva6vVSmNjY482\nSUlJPm1cLheJiYlhzQVw4MABli5disVi4ZZbbiE9PT1kmQIRjr4KVLj7qrW1lbfeeotLL73U5+fh\n7rML5YLw9Jmu6zz44IO0tLRw3XXXMWHCBJ/t4eovf7kgPP31zDPPcPPNN3PixInzbg91fw2pgjCY\nXXLJJfzsZz9j5MiR1NfXs2bNGsrKysIdKyKFu686OztZu3Ytt912GzExMQN2XH8+KVe4+kzTNNas\nWUNHRwdPPfUU//znPxk/fnzIj9vXXOHor127dhEfH09GRgb79u0L6bEuZEgNGVksFpxOp/e10+nE\nYrH0aHO020LV52sTjlwxMTHe29ipU6fS1dXFhx9+GNJc/oSjrwIRzr5yu92sXbuWz3/+81x99dU9\ntoerz/zlCve/r9GjR3PllVfS0NDg8/Nw/xu7UK5w9NeBAweoq6vjnnvuYd26dezdu7dHEQp1fw2p\ngpCZmUlzczOtra243W5qamqw2Ww+bWw2G9XV1RiGwcGDB4mJiQn57WkguT744AOMM1NCmpqa0HWd\niy66KKS5/AlHXwUiXH1lGAYbN25k7NixzJs377xtwtFngeQKR599+OGHdHR0AJ4ne3bv3s3YsWN9\n2oSjvwLJFY7+WrRoERs3bmT9+vXce++9TJo0ieLiYp82oe6vITVkZDKZKCwspKSkBF3XycnJIT09\nncrKSgDy8/PJzs6mvr6e4uJioqKiKCoqiohcf/vb36isrMRkMhEVFcW9994b8g+61q1bx5tvvslH\nH33EXXfdxY033ojb7fZmCkdfBZIrHH0Fnt/gqqurGT9+PN///vcBWLhwofc3tnD1WSC5wtFnbW1t\nrF+/Hl3XMQyDz33uc0ybNi3s/38MJFe4/o2dz0D2l8xUFkIIAQyxISMhhBC9JwVBCCEEIAVBCCHE\nGVIQhBBCAFIQhBBCnCEFQQghBCAFQQghxBlSEIQQQgDw/wFD1BX6moxD7QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x250ef498978>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.lineplot(data = hist['val_loss'], label = 'val_loss')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can conclude from the above 'line plot' that the validation loss keep on decreasing after every epoch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x250ef82b630>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD8CAYAAAB3u9PLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtgVNW59/HvmpkEGISQyRAicUJCAA2i8RIFUaJICq0C\nph603hGqQCKH1toU9RW1tlRa5eIlgtWUKGi1VtFCyzENHKUlpzSCIPckiEpCIJchQAK5TPZ6/xgc\nTLkkE5LsSeb5/OWw15757ZWYZ/baa6+ttNYaIYQQQc9idgAhhBCBQQqCEEIIQAqCEEKIE6QgCCGE\nAKQgCCGEOEEKghBCCEAKghBCiBOkIAghhACkIAghhDhBCoIQQggAbGYH8Nf+/ftbtZ/T6aSioqKN\n05w7yeUfyeUfyeWfQM0F55atf//+LWonZwhCCCEAKQhCCCFOkIIghBAC6ITXEP6T1pra2loMw0Ap\ndcZ2Bw8epK6urgOTtUxXyqW1xmKx0L1797P+LIQQganTF4Ta2lpCQkKw2c5+KDabDavV2kGpWq6r\n5fJ4PNTW1tKjR492SCWEaE+dfsjIMIxmi4HoODabDcMwzI4hhGiFTl8QZGgi8MjPRIjOqdMXBCGE\n6Kp0Qz165xaMD96ksbK83T9PxlqEECJAaKMR9u1F79iC3rkZinZCQz1YrXguHw5xF7br50tB6GCD\nBw+msLDQ7BhCiACgtYbyUvTOL7wFYNdWqDnq3Rg9AHX991EJiTDkYrpdEMPRdr6LWgpCkPJ4PHIx\nXggT6CNV6F1fwM4t6J1boLLMu8HhRF12NSRchrroUlRYeIdn61J/EYx3XkPv23v6bUp5q7GflCsO\nyx0PnnH7b37zG/r378/9998PwPz587FareTl5XH48GE8Hg+/+MUvGDduXLOfVVNTw5QpU06733vv\nvcerr74KQEJCAi+99BLl5eU8+uijfP311wA8++yzREVFMXnyZNauXQvAkiVLqKmp4ZFHHmHSpEkM\nHTqU/Px8brnlFgYOHMiLL75IfX094eHhvPzyy/Tt25eamhqefPJJNm/ejFKKhx9+mKNHj7Jjxw6e\neeYZAN566y0KCgr45S9/6XefChFMdF0tFG5H79yC3rEFik/8jbL3hIsuRY271XsW0K+/6RMyulRB\nMMPEiRN56qmnfAVh5cqVvPXWW/z4xz+mV69euN1uJkyYwNixY5v9YXfr1o2srKxT9isoKOCFF17g\nL3/5Cw6Hg0OHDgEwZ84cRowYQVZWFo2NjdTU1HD48OGzfkZDQwOrV68GoKqqipUrV6KU4u233+aV\nV17hqaeeYtGiRfTq1Ys1a9b42oWEhPDiiy8yZ84cQkJCePfdd/ntb397jr0nRNejGxvhq0L0zs3e\nM4A9u6HRAzYbDBqK+uG9qITLYMBAlCWw7kHqUgXhbN/kbTYbHo+nzT9z2LBhVFRUcODAASorKwkL\nCyMyMpKnn36aDRs2oJTiwIEDlJeXExkZedb30lozb968U/Zbv34948ePx+FwABAe7j2VXL9+PS+8\n8AIAVquV3r17N1sQJk6c6Pvv0tJS0tLSKCsro76+npiYGAD+8Y9/8Pvf/97Xrk+fPgBce+215Obm\nMnjwYDweDwkJCX72lhBdj9YaSvedvA5QsA2OHwOlICYe9b1bvGcAgxJQod3MjntWXaogmGX8+PH8\n9a9/paysjIkTJ/LBBx9QWVnJ6tWrCQkJYfjw4S1aBqK1+32X1WptcmNYbW1tk+12u93333PmzGHa\ntGmMHTuWvLw8FixYcNb3vvPOO3nppZcYNGgQt99+u1+5hOhK9KFK77f/b68DHHZ7N/SNQl2VjBqa\nCBdegjqvt7lB/SQFoQ1MnDiRjIwM3G4377//PitXrsTpdBISEsL69espLi5u0fscPXr0tPtde+21\n/PjHP2batGm+IaPw8HCuu+463nzzTR588EHfkFHfvn2pqKjA7XbTs2dPcnNzGT169Gk/78iRI0RF\nRQHeaxTfSk5O5g9/+ANPP/004B0y6tOnD1dccQX79+9n69at5ObmnkOPCdG56GM1ULD1xFnAFijd\n593QKwx10aWQkIhKSEQ5+5kb9BxJQWgDF154ITU1NURFRdGvXz9uvfVWJk+ezJgxY7j00ksZNGhQ\ni97nTPtdeOGFzJo1i0mTJmGxWBg2bBiLFi3imWee4Re/+AXvvPMOFouFZ599lqSkJB5++GHGjx9P\nVFTUWT/7kUceYfr06YSFhXHttdeyb5/3l/wnP/kJTzzxBDfeeCMWi4Wf/exn3HTTTQBMmDCB7du3\n+4aRhOiKdEMDfLn75HWArwrBMCC0Gwy5GHVdivc6QPQAlKXr3N+rdGum3pjoP5+YduzYsSbDIGfS\nXtcQzlVny3Xffffx4IMPMmrUqDPu29KfSWsE6hOtJJd/Ai2XNgwo/gr7N0VUb8yDgu1QXwcWC8QN\n8X77T0iEgReibCGmZOyIJ6bJGYJokcOHD3PzzTczdOjQsxYDIToLXXGw6XWA6iNUA5zvQl33vRM3\nhA1D2XuaHbXDSEEwwc6dO5k1axbgXQhOa023bt1YtWqVycnOLCwsjH/+859mxxCi1XT1EdjlvQag\nd26B8gPeDX0cqEuuhITLiLj2Bg4Zwbs4Y6cvCJ1sxAvw3lj297//HQjcIaNz0Rl/JqLr0fV1ULTj\nxLpAW2Dfl6A19LB7ZwClTPSeBURd4LtHyOpwQgANZXW0Tl8QLBaLLMMQQDweD5YudJFNdB7aaISv\n96B3fHtD2C7wNIDVBvEXoSbe5S0AsYNRAfhQqkDQ6f+Kdu/endraWurq6s56J3C3bt0C8lGVXSnX\ndx+hKUR701rDwf0nhoA2w+6tcKzGu9EVh7pxvLcADB6K6ia/ky3RooKwefNmli5dimEYjBkzhtTU\n1Cbbq6urWbx4MQcPHiQkJIS0tDTfXa+rVq1i7dq1KKVwuVykp6cTGhrKV199xWuvvUZ9fT1Wq5UH\nHnigxdMzv0sp1aLHNQbarIZvSS4hWk4fPtT0QvChE7+jEZGoK6/13g9w0aWoXmHmBu2kmi0IhmGQ\nlZXFE088QUREBI899hhJSUlccMEFvjYrVqwgNjaWjIwMSkpKyMrK4sknn8TtdrN69WoWLlxIaGgo\nCxYsIC8vjxtuuIHly5czadIkLr/8cjZt2sTy5ct9N0IJIQSArj0GBdtPXggu8S7kSM9e3hvCht6O\nuijRe4ewPKnvnDVbEIqKinw3XAGMHDmS/Pz8JgWhuLjYd9YQHR1NeXk5VVVVgLegfHsW8O2qmuD9\nZn/8+HHAO2/9238XQgQv7fHA3oKTN4TtLYDGRggJ9Q79jLjBe0OYK65L3RAWKJotCG63m4iICN/r\niIiIUx7wMmDAADZs2EBCQgJFRUWUl5fjdrsZOHAgEyZMIC0tjdDQUBITE0lMTARg8uTJzJ07l2XL\nlmEYBr/+9a/b+NCEEIFOaw0lX6N3nVgaumA71B0HZYHYQSeXho6/CBUSanbcLq9NLiqnpqaSnZ1N\nRkYGMTExxMXFYbFYqK6uJj8/n8zMTOx2OwsWLGDdunUkJyeTk5PD5MmTGTFiBHl5eSxZsoQ5c+ac\n8t65ubm+dXPmzZuH0+lsVUabzdbqfduT5PKP5PJPoOXSWtN4oIS6T1YT8vkG6r/4DKPKuzCcNTqG\n0NE/IDTxKkKHXY7FhIXhAq2/vqsjsjVbEBwOB5WVlb7XlZWVvmWYv2W320lPTwe8P/CZM2cSGRnJ\nli1biIyMpHdv7w92+PDhFBQUkJyczKeffsqUKVMAuOaaa3wPf/lPKSkppKSk+F639kJnoF4klVz+\nkVz+MTuXbmiAb/agi3aii3bCnp1w9MQS7b37nFgS4jJUwqXg6Es9UA9QWw+1HZ/b7P46m4BYuiI+\nPp7S0lLKyspwOBzk5eX57rL9Vk1NDd26dcNms7FmzRoSEhKw2+04nU4KCwupq6sjNDSUrVu3Eh8f\nD3gLzY4dO7j44ovZtm2bb9VNIUTnpY8egT3eP/56zy7vonCeBu/GyPNRw66AQQk4kkZyqEcvuRAc\nYJotCFarlalTpzJ37lwMw2D06NG4XC5ycnIAGDt2LCUlJWRmZgLgcrmYMWMG4H2g/IgRI5g9ezZW\nq5XY2Fjft/3p06f7prKGhIQwffr09jpGIUQ70FrDgRJ00Q7Yswu9ZyccKPFutNpgQDzqxptR8Qkw\n6CJU75MTR2xOJypAv4kHs06/2mlLBeqpoOTyj+TyT1vm0g318FXRiW//J4Z/qo96N57XC+ITUPEJ\nqEEJ3mJwlqeDBUN/tbWAGDISQgQnfeQQFHm/+euinfD1Hu+zgQGiolGJw72zfwYN9b6W4Z9OTwqC\nEML7PIDSYvSeHfDtBeBvVwO1hXjX//neLaj4i7xnAr0616MhRctIQRAiCOm6Oviq4OTsny93nVwH\nqFeY94/+9T/wDv/ExKNCzHkojOhYUhCECAK6qtL3zV/v2eVdCrqx0bvxfJd3HaBBJ8b/+54vwz9B\nSgqCEF2MNhqh5Bv0np0c3vcljds3Q2WZd2NoKMQO8d4BPCjB+0jInr3MDSwChhQEITo5XXsM9hae\nHP7ZuxuOHwOgPtwJA4egxkzwFgDXQJQ8O0ScgfxmCNHJaHe59w//t9M/930F2gClIHoA6upk7/BP\nfALOiy5ustKAEGcjBUGIAKYbG6H4K9+yD7po58lnAHTrDnFDUDff5r35a+CFpzwQXq4FCH9IQRAi\ngOhjNfDl7pNz//cWQF2td2O40zvs8+3NXxfEyqMgRZuSgiCESbTWUHHQO+zz7eyfkq+9D4JXFnDF\nokaO8Q3/qIi+ZkcWXZwUBCE6iPZ4YN+XJ7/9F+2Cw96ln+neAwZehLr8mhOzf4agutvNDSyCjhQE\nIdqJrqmGL3ednP3zVQHU13s3RkSiLrzk5Nz/6BiURYZ/hLmkIAjRBrTW6LL9J2f/FO2E0n3ejRaL\nd7rnqHEnrwGER5z9DYUwgRQEIc6BbqhH53xIxSd/8z35ix49vYu+Db/eWwBiB6O6dTc3qBAtIAVB\niFbQWsPn/8J47w9QcZCQK0fSkHCZtwCc75IHwItOSQqCEH7S+7/BeOc12LkF+sdg+dmvCB81JmDX\n0ReipaQgCNFCuqYavfKP6P/9K3TvgbpjGuqGH8i9AKLLkIIgRDO00Yj+59/RK5ZDzVFU8jjULffI\nMwFElyMFQYiz0IU7MN75PXzzJQweiuWOaaiYgWbHEqJdSEEQ4jS0uwL9fjb63+u8S0ZMy0AlXSdr\nA4kuTQqCEN+hG+rRH69Ar/4zGAZq/I9Q3/8vmTYqgoIUBCH4zjTSP2V5HyZzxUgst01BOfuZHU2I\nDiMFQQQ9XfINxrtNp5GqhESzYwnR4aQgiKB1yjTSO6d5Hywv00hFkGpRQdi8eTNLly7FMAzGjBlD\nampqk+3V1dUsXryYgwcPEhISQlpaGjExMQCsWrWKtWvXopTC5XKRnp5OaGgoAKtXr+bjjz/GYrFw\nxRVXcM8997Tx4QlxKm00ov/xd/SHy6CmBpU8VqaRCkELCoJhGGRlZfHEE08QERHBY489RlJSEhdc\ncIGvzYoVK4iNjSUjI4OSkhKysrJ48skncbvdrF69moULFxIaGsqCBQvIy8vjhhtuYNu2bXz22Wc8\n99xzhISEcPjw4XY9UCEAdMF27zTSfXthyMVYfvSgTCMV4oRmC0JRURFRUVH06+e9uDZy5Ejy8/Ob\nFITi4mLfWUN0dDTl5eVUVVUB3oJSX1+P1Wqlvr6e8PBwAHJycrjlllsICQkBICwsrG2PTIjv0O5y\n9J+z0fn/AIdMIxXidJotCG63m4iIk0v1RkREUFhY2KTNgAED2LBhAwkJCRQVFVFeXo7b7WbgwIFM\nmDCBtLQ0QkNDSUxMJDHRe7GutLSUXbt28c477xASEsK9997LoEGDTvn83NxccnNzAZg3bx5Op7N1\nB2qztXrf9iS5/ONvLl1fR81Hb1Pz/jLQBj1vn0rPW+9p82mkXaW/Oork8l9HZGuTi8qpqalkZ2eT\nkZFBTEwMcXFxWCwWqquryc/PJzMzE7vdzoIFC1i3bh3JyckYhkF1dTVz585lz549LFy4kJdffvmU\nb2wpKSmkpKT4Xrd2ATGn0xmQi49JLv+0NNeZppHWOvtRe7QajlabkqujSS7/BGouOLds/fv3b1G7\nZguCw+GgsrLS97qyshKHw9Gkjd1uJz09HfD+jzhz5kwiIyPZsmULkZGR9O7tvVg3fPhwCgoKSE5O\nxuFwcPXVV6OUYtCgQVgsFo4ePeprK0RrNZlGGj1AppEK0ULNLtoeHx9PaWkpZWVleDwe8vLySEpK\natKmpqYGj8cDwJo1a0hISMBut+N0OiksLKSurg6tNVu3biU6OhqAq666iu3btwOwf/9+PB4PvXr1\nauvjE0FE11RjvPMaxjOz4Os9qDunYZmzSIqBEC3U7BmC1Wpl6tSpzJ07F8MwGD16NC6Xi5ycHADG\njh1LSUkJmZmZALhcLmbMmAHA4MGDGTFiBLNnz8ZqtRIbG+sb/rnxxht55ZVXeOSRR7DZbDz00ENy\ngU+0yinTSK8fh5p4t0wjFcJPSmutzQ7hj/3797dqv0AdG5Rc/vnPXKdMI71jGsoVZ3quQCG5/BOo\nuSBAriEIEYhOnUb6C1TStXKWKcQ5kIIgOhVdV4ex6l3vaqRao8bfcWI10m5mRxOi05OCIDoF7zTS\n/6Pi/TfQZaWyGqkQ7UAKggh4uuRr70Ptd32BLWYgyDRSIdqFFAQRsHRNNfovb6M/+Rt0t6PunIbj\nv+6h8lCV2dGE6JKkIIiAc7ZppMoqv7JCtBf5v0sElECZRipEMJKCIAKCTCMVwnxSEISpdH0dOudD\nmUYqRACQgiBM8e00UuNPf/CuRnrlSCyTZBqpEGaSgiA63HenkRI9AMsjv0ZddKnZsYQIelIQRIc5\nZRrpXdNRyd+Xh9oLESCkIIh2d9pppLfcjTpPViMVIpBIQRDtquk00mFY7nhQppEKEaCkIIh2IdNI\nheh8pCCINuWdRrrixDRSUBPuQI2TaaRCdAZSEESbOO000tumoiIizY4mhGghKQjinMk0UiG6BikI\notV0zVH0R2+jP10t00iF6AKkIAi/aaMRvS4H/dHyE9NIv4+65S6ZRipEJycFQfjllGmkdz6IukCm\nkQrRFUhBEC0i00iF6PqkIIizkmmkQgSPFhWEzZs3s3TpUgzDYMyYMaSmpjbZXl1dzeLFizl48CAh\nISGkpaURExMDwKpVq1i7di1KKVwuF+np6YSGhvr2XblyJcuWLeP111+nd28Zgw4UWmv0pjyZRipE\nELE018AwDLKysnj88cdZuHAh69evp7i4uEmbFStWEBsby/PPP8/MmTPJzs4GwO12s3r1aubNm8f8\n+fMxDIO8vDzffhUVFXzxxRc4nc62PSpxTnRlGVVPzcJYPA+698DyyK+xznhUioEQXVyzBaGoqIio\nqCj69euHzWZj5MiR5OfnN2lTXFzMsGHDAIiOjqa8vJyqKu+D0A3DoL6+nsbGRurr6wkPD/ft98Yb\nb3D33XfLOHQA0UYjxqu/o6FoJ+qu6VjmLJJ7CoQIEs0WBLfbTUREhO91REQEbre7SZsBAwawYcMG\nwFtAysvLcbvdOBwOJkyYQFpaGtOmTcNut5OYmAhAfn4+DoeD2NjYNjwcca507l9gbwG90n6BZfTN\nck+BEEGkTS4qp6amkp2dTUZGBjExMcTFxWGxWKiuriY/P5/MzEzsdjsLFixg3bp1DB8+nBUrVvDE\nE080+965ubnk5uYCMG/evFYPL9lstoAcmgqkXJ6Sb6j86C26XXUd593wA3o0Npod6RSB1F/fJbn8\nI7n81xHZmi0IDoeDyspK3+vKykocDkeTNna7nfT0dMB7MXLmzJlERkayZcsWIiMjfReLhw8fTkFB\nAbGxsZSVlZGRkeF7z9mzZ/Pss8/Sp0+fJu+dkpJCSkqK73VFRUWrDtTpdLZ63/YUKLm0YWAsegZs\nITTc/mMaGxsDItd/CpT++k+Syz+Sy3/nkq1///4tatdsQYiPj6e0tJSysjIcDgd5eXnMmjWrSZua\nmhq6deuGzWZjzZo1JCQkYLfbcTqdFBYWUldXR2hoKFu3biU+Pp6YmBhef/113/4PPfQQzz77rMwy\nMpH+5G9QtAN1/09QfSKa30EI0eU0WxCsVitTp05l7ty5GIbB6NGjcblc5OTkADB27FhKSkrIzMwE\nwOVyMWPGDAAGDx7MiBEjmD17NlarldjY2Cbf9kVg0OUH0B+8CcOuQI280ew4QgiTKK21NjuEP/bv\n39+q/QL1VNDsXFprjAVz4KtCLE+/jIroGxC5zkRy+Udy+SdQc0HHDBk1O8tIdG36Hx/Dri9Qk6b4\nioEQIjhJQQhi2l2Ofm8pXHQpKnmc2XGEECaTghCktNYYyzLBMLDcN1NuDhRCSEEIVvr/1sK2Tahb\nJ6P6RpkdRwgRAKQgBCFdVYl+93UYNBQ1+iaz4wghAoQUhCCjtcZYvhgaGrBM/m+URX4FhBBe8tcg\nyOj8f8CWf6NuuRsVFW12HCFEAJGCEET0kSr0H1+FuCGo7000O44QIsBIQQgi+o+/h9rjWCbPQllk\nFVMhRFNSEIKE3pSH/uyfqPF3oKJjzI4jhAhAUhCCgK4+gvHWEogZiBp3q9lxhBABSgpCENDvvg41\nR7Hc/xOUrU0egSGE6IKkIHRxeks++l+foH5wG8oVZ3YcIUQAk4LQhelj1RjLMyF6AOrm28yOI4QI\ncFIQujD93lI4XIXl/lkoW4jZcYQQAU4KQheld3yO/uffUeN+iIodbHYcIUQnIAWhC9K1xzDeeBmi\nolET7zQ7jhCik5CC0AXp99+EQxXeG9BCQs2OI4ToJKQgdDF691b0J39DjZmAGpRgdhwhRCciBaEL\n0XW1GG+8BH2jUKn3mh1HCNHJSEHoQvSHy6H8gHdZ627dzI4jhOhkpCB0EbpoJ3rNStQNN6EuvMTs\nOEKITkgKQheg6+sw3ngRHH1R/3Wf2XGEEJ1Uixa22bx5M0uXLsUwDMaMGUNqamqT7dXV1SxevJiD\nBw8SEhJCWloaMTHeFTVXrVrF2rVrUUrhcrlIT08nNDSUZcuWsXHjRmw2G/369SM9PZ2ePXu2/REG\nAb3yHThQguXhX6K6282OI4TopJo9QzAMg6ysLB5//HEWLlzI+vXrKS4ubtJmxYoVxMbG8vzzzzNz\n5kyys7MBcLvdrF69mnnz5jF//nwMwyAvLw+ASy+9lPnz5/P8889z/vnns2LFirY/uiCg9xaiP16B\nuu57qKGXmx1HCNGJNVsQioqKiIqKol+/fthsNkaOHEl+fn6TNsXFxQwbNgyA6OhoysvLqaqqArwF\npb6+nsbGRurr6wkPDwcgMTERq9X7kJYhQ4bgdrvb9MCCgW5o8A4VhfVB3TbF7DhCiE6u2YLgdruJ\niIjwvY6IiDjlj/eAAQPYsGED4C0g5eXluN1uHA4HEyZMIC0tjWnTpmG320lMTDzlM9auXctll112\nrscSdPTf3oOSr7Hc8xDKfp7ZcYQQnVybLI6fmppKdnY2GRkZxMTEEBcXh8Viobq6mvz8fDIzM7Hb\n7SxYsIB169aRnJzs2/eDDz7AarUyatSo0753bm4uubm5AMybNw+n09mqjDabrdX7tqfW5mrYW4B7\n9Xt0v34cYWN+EDC52pvk8o/k8k+g5oKOydZsQXA4HFRWVvpeV1ZW4nA4mrSx2+2kp6cDoLVm5syZ\nREZGsmXLFiIjI+nduzcAw4cPp6CgwFcQPvnkEzZu3MiTTz6JUuq0n5+SkkJKSorvdUVFhZ+H6OV0\nOlu9b3tqTS7t8WAsegZ69qL+h/e1y3F1pf7qCJLLP5LLf+eSrX///i1q1+yQUXx8PKWlpZSVleHx\neMjLyyMpKalJm5qaGjweDwBr1qwhISEBu92O0+mksLCQuro6tNZs3bqV6OhowDtz6aOPPmL27Nl0\nk5uo/KI//gC++RLL3TNQPXuZHUcI0UU0e4ZgtVqZOnUqc+fOxTAMRo8ejcvlIicnB4CxY8dSUlJC\nZmYmAC6XixkzZgAwePBgRowYwezZs7FarcTGxvq+7WdlZeHxePjVr37laztt2rR2OciuRJd8g171\nDirpOtQVI82OI4ToQpTWWpsdwh/79+9v1X6BeiroTy7d2Ijx29lQXorll5mo3n0CIldHklz+kVz+\nCdRcECBDRiJw6Ny/wN4C1B3T2rUYCCGCkxSETkIfKEF/9BZcNhx1dXLzOwghhJ+kIHQC2jC8y1qH\nhGC5O+2MM7KEEOJcSEHoBPT//g2KdqB+9ACqj6P5HYQQohWkIAQ4XX4A/cEbMOxK1DU3mh1HCNGF\nSUEIYFpr71CRxYLl3nQZKhJCtCspCAFMr/sYdm9F3TYF5ehrdhwhRBcnBSFA6cpy9J+XQkIiatQ4\ns+MIIYKAFIQApLXGWPYyGAaWex+SoSIhRIeQghCAdN5a2P456tbJqL5RZscRQgQJKQgBRldVov/0\nOgwaihp9k9lxhBBBRApCANFaYyxfDA0NWO6fhbLIj0cI0XHkL04A0f9eB1v+jUq9G9WvZYtRCSFE\nW5GCECD0kUPod34PcUNQKRPNjiOECEJSEAKEfvv3UHv8xFCR1ew4QoggJAUhAOiNeeiN61Hj70D1\njzE7jhAiSElBMJlx5DDGW4shJh417laz4wghgpgUBJMdzVoIx6q9Q0W2Zp9oKoQQ7UYKgon0ln9T\nuy4H9YPbUK44s+MIIYKcFAST6GPVGMtfwRYzEHXzbWbHEUIIKQhm0X/6Axypovd//z+ULcTsOEII\nIQXBDHr75+j1uahxPyRkUILZcYQQApCC0OF07TGMN1+GqAtQE+40O44QQvhIQehg+v034FCFd1ZR\nSKjZcYQQwqdF8xw3b97M0qVLMQyDMWPGkJqa2mR7dXU1ixcv5uDBg4SEhJCWlkZMjPcGq1WrVrF2\n7VqUUrhcLtLT0wkNDaW6upqFCxdSXl5O3759efjhhznvvPPa/ggDiN71BfqT1aiUW1DxF5kdRwgh\nmmj2DMFXBOezAAAO10lEQVQwDLKysnj88cdZuHAh69evp7i4uEmbFStWEBsby/PPP8/MmTPJzs4G\nwO12s3r1aubNm8f8+fMxDIO8vDwAPvzwQy655BJefPFFLrnkEj788MO2P7oAoutqvUNFfaNQqfeY\nHUcIIU7RbEEoKioiKiqKfv36YbPZGDlyJPn5+U3aFBcXM2zYMACio6MpLy+nqqoK8BaU+vp6Ghsb\nqa+vJzw8HID8/Hyuv/56AK6//vpT3rOr0SuWQfkBLJNnobp1MzuOEEKcotkhI7fbTUREhO91REQE\nhYWFTdoMGDCADRs2kJCQQFFREeXl5bjdbgYOHMiECRNIS0sjNDSUxMREEhMTATh8+LCvOPTp04fD\nhw+f9vNzc3PJzc0FYN68eTidztYdqM3W6n3PVf3OLzi0dhU9vn8rva+9IWBynY3k8o/k8o/k8l9H\nZGuTtRJSU1PJzs4mIyODmJgY4uLisFgsVFdXk5+fT2ZmJna7nQULFrBu3TqSk5Ob7K+UOuNzg1NS\nUkhJSfG9rqioaFVGp9PZ6n3Pha6vw3jhV+DoS93Nt5+SwaxczZFc/pFc/pFc/juXbP37t+z5Ks0W\nBIfDQWVlpe91ZWUlDoejSRu73U56ejrgferXzJkziYyMZMuWLURGRtK7d28Ahg8fTkFBAcnJyYSF\nhXHo0CHCw8M5dOiQr01Xo//yRzhYguXhX6K6282OI4QQZ9TsNYT4+HhKS0spKyvD4/GQl5dHUlJS\nkzY1NTV4PB4A1qxZQ0JCAna7HafTSWFhIXV1dWit2bp1K9HR0QAkJSXx6aefAvDpp59y1VVXtfWx\nmU7vLUTnfIgaNRY19HKz4wghxFk1e4ZgtVqZOnUqc+fOxTAMRo8ejcvlIicnB4CxY8dSUlJCZmYm\nAC6XixkzZgAwePBgRowYwezZs7FarcTGxvqGf1JTU1m4cCFr1671TTvtSnRDA0b2CxAWjpo0xew4\nQgjRLKW11maH8Mf+/ftbtV9Hjw0aH72FXvUulv+eg7r0zGc/gTpmKbn8I7n8I7n81xHXEORO5Xag\nv/kSvfrPqBGjz1oMhBAikEhBaGPa4/EOFfXshbrjAbPjCCFEi0lBaGP6f96HfXux3J2G6tnL7DhC\nCNFiUhDakC75Br3qXVTSdagrrjE7jhBC+EUKQhvRjY3eoaIedtRd082OI4QQfpOC0EZ07kfwVSHq\nzmmoXmFmxxFCCL9JQWgD+kAx+sO34LIRqKtGmR1HCCFaRQrCOdJGI8YbL0FoKJa7Z5xxTSYhhAh0\nUhDOkf7fv0HRTtSPHkD1cTS/gxBCBCgpCOdAl5WiP3gThl2JuuZGs+MIIcQ5kYLQStowvE9As1iw\n3JsuQ0VCiE5PCkIr6XUfw+6tqNumohx9zY4jhBDnTApCK+jKcvSfsyEhETVqrNlxhBCiTUhB8JPW\nGmPZy4DGcu9DMlQkhOgypCD4Seetge2fo269D9U3yuw4QgjRZqQg+EFXVaLfzYLBQ1E33GR2HCGE\naFNSEFpIa42xfDF4GrBMnoWySNcJIboW+avWQvrf62DLv1Gpd6P6tezpQ0II0ZlIQWgBfeQQ+o+/\nh7ghqJSJZscRQoh2IQWhBYy3X4W641jun4WyWM2OI4QQ7UIKQjP0xvWwMQ814U5U/xiz4wghRLuR\ngnAW+ugRjLeWQEw8auwPzY4jhBDtSgrCWeh3X4Nj1VimzELZbGbHEUKIdtWiv3KbN29m6dKlGIbB\nmDFjSE1NbbK9urqaxYsXc/DgQUJCQkhLSyMmJob9+/ezcOFCX7uysjJuv/12br75Zr766itee+01\n6uvrsVqtPPDAAwwaNKhtj+4c6C3/Rm/4FDXhDtQFcWbHEUKIdtdsQTAMg6ysLJ544gkiIiJ47LHH\nSEpK4oILLvC1WbFiBbGxsWRkZFBSUkJWVhZPPvkk/fv357nnnvO9z/Tp07n66qsBWL58OZMmTeLy\nyy9n06ZNLF++nKeffrp9jtJP+lg1xrJXIHoA6qbbzI4jhBAdotkho6KiIqKioujXrx82m42RI0eS\nn5/fpE1xcTHDhg0DIDo6mvLycqqqqpq02bp1K1FRUfTt610ZVCnF8ePHATh27Bjh4eFtckBtQf8p\nC45WYZnyE5QtxOw4QgjRIZo9Q3C73URERPheR0REUFhY2KTNgAED2LBhAwkJCRQVFVFeXo7b7aZP\nnz6+NuvXr+faa6/1vZ48eTJz585l2bJlGIbBr3/969N+fm5uLrm5uQDMmzcPp9Pp3xGeYLPZWrRv\n3ef/omr9Guy33kuvK0e06rPaI1dHk1z+kVz+kVz+64hsbXKlNDU1lezsbDIyMoiJiSEuLg7Ld5Z2\n8Hg8bNy4kbvuusv3bzk5OUyePJkRI0aQl5fHkiVLmDNnzinvnZKSQkpKiu91RUVFqzI6nc5m99XH\nj2G8/BuIuoDalFuoa+VntXUuM0gu/0gu/0gu/51Ltv79W7a6QrMFweFwUFlZ6XtdWVmJw9H02cF2\nu5309HTAu+bPzJkziYyM9G3//PPPiYuLa3LG8OmnnzJlyhQArrnmGl599dUWBW5P+v1sOFSJZfZv\nUSGhZscRQogO1ew1hPj4eEpLSykrK8Pj8ZCXl0dSUlKTNjU1NXg8HgDWrFlDQkICdrvdt/0/h4vA\nW2h27NgBwLZt24iKMncpab1zC/rT/0GlTETFX2RqFiGEMEOzZwhWq5WpU6cyd+5cDMNg9OjRuFwu\ncnJyABg7diwlJSVkZmYC4HK5mDFjhm//2tpavvjiC6ZNm9bkfadPn+6byhoSEsL06dPb8rj8outq\nvc9Hjjwfdcs9puUQQggzKa21NjuEP/bv39+q/c42/ma88xp6zUosGb9BDRl2LvHaNJeZJJd/JJd/\nJJf/OuIaQtDfqayLdqDXrkKNvqnDi4EQQgSSoC4Iur4OI/slcPRF3TrZ7DhCCGGq4C4If3kbDpZg\nuW8mqnsPs+MIIYSpgrYg6L0F6JyPUKPGooZeZnYcIYQwXVAWBN3QgLH0BQgLR02aYnYcIYQICMFZ\nEP76LpTuw3LfQyh7T7PjCCFEQAi6gqC/2YNe/WfUNaNRlyQ1v4MQQgSJoCoI2uPBWPoi9ApD/egB\ns+MIIURACa6C8D/vQ/FeLHenoXr2MjuOEEIElKApCJ6v96BXvYu6ahTq8vZf1loIITqboCgIurGR\nwy//BnrYUXdOa34HIYQIQsFREP7+IZ6inai7pqN6hZkdRwghAlJQFAT6OOh+402opOvMTiKEEAGr\nTZ6YFugsI0YTNv62gF3FUAghAkFwnCEIIYRolhQEIYQQgBQEIYQQJ0hBEEIIAUhBEEIIcYIUBCGE\nEIAUBCGEECdIQRBCCAGA0lprs0MIIYQwX9CcITz66KNmRzgtyeUfyeUfyeWfQM0FHZMtaAqCEEKI\ns5OCIIQQAgDr008//bTZITrKwIEDzY5wWpLLP5LLP5LLP4GaC9o/m1xUFkIIAciQkRBCiBO63PMQ\nNm/ezNKlSzEMgzFjxpCamtpku9aapUuX8vnnn9OtWzfS09M75BSxuVzbt2/nd7/7HZGRkQAMHz6c\nSZMmtWumV155hU2bNhEWFsb8+fNP2W5WXzWXy4y+AqioqCAzM5OqqiqUUqSkpHDTTTc1aWNGn7Uk\nlxl9Vl9fz1NPPYXH46GxsZERI0Zw++23N2ljRn+1JJdZv2MAhmHw6KOP4nA4TplZ1O79pbuQxsZG\nPXPmTH3gwAHd0NCgf/7zn+t9+/Y1abNx40Y9d+5cbRiG3r17t37ssccCIte2bdv0s88+2+5Zvmv7\n9u16z549+mc/+9lpt5vRVy3JZUZfaa212+3We/bs0VprfezYMT1r1qyA+P1qSS4z+swwDH38+HGt\ntdYNDQ36scce07t3727Sxoz+akkus37HtNZ65cqVetGiRaf9/Pbury41ZFRUVERUVBT9+vXDZrMx\ncuRI8vPzm7T57LPPSE5ORinFkCFDqKmp4dChQ6bnMsPQoUM577zzzrjdjL5qSS6zhIeH+76N9ejR\ng+joaNxud5M2ZvRZS3KZQSlF9+7dAWhsbKSxsRGlVJM2ZvRXS3KZpbKykk2bNjFmzJjTbm/v/upS\nQ0Zut5uIiAjf64iICAoLC09p43Q6m7Rxu92Eh4ebmgtg9+7d/PznP8fhcHDvvfficrnaLVNLmNFX\nLWV2X5WVlbF3714GDRrU5N/N7rMz5QJz+swwDGbPns2BAwcYN24cgwcPbrLdrP5qLheY01/Z2dnc\nc889HD9+/LTb27u/ulRB6Mzi4uJYvHgx3bt3Z9OmTTz33HO8+OKLZscKSGb3VW1tLfPnz+f+++/H\nbrd32Oc252y5zOozi8XCc889R01NDc8//zzffPMNMTEx7f6555rLjP7auHEjYWFhDBw4kO3bt7fr\nZ51JlxoycjgcVFZW+l5XVlbicDhOaVNRUXHWNmbkstvtvtPYK664gsbGRo4cOdKuuZpjRl+1hJl9\n5fF4mD9/PqNGjWL48OGnbDerz5rLZfbvV8+ePbn44ovZvHlzk383+3fsTLnM6K/du3fz2Wef8dBD\nD7Fo0SK2bdt2ShFq7/7qUgUhPj6e0tJSysrK8Hg85OXlkZSU1KRNUlIS69atQ2tNQUEBdru93U9P\nW5KrqqoKfeKWkKKiIgzDoFevXu2aqzlm9FVLmNVXWmuWLFlCdHQ048ePP20bM/qsJbnM6LMjR45Q\nU1MDeGf2fPHFF0RHRzdpY0Z/tSSXGf111113sWTJEjIzM/npT3/KsGHDmDVrVpM27d1fXWrIyGq1\nMnXqVObOnYthGIwePRqXy0VOTg4AY8eO5fLLL2fTpk3MmjWL0NBQ0tPTAyLXv/71L3JycrBarYSG\nhvLTn/603S90LVq0iB07dnD06FFmzJjB7bffjsfj8WUyo69aksuMvgLvN7h169YRExNDRkYGAHfe\neafvG5tZfdaSXGb02aFDh8jMzMQwDLTWXHPNNVx55ZWm///Yklxm/Y6dTkf2l9ypLIQQAuhiQ0ZC\nCCFaTwqCEEIIQAqCEEKIE6QgCCGEAKQgCCGEOEEKghBCCEAKghBCiBOkIAghhADg/wO1A06Eb8Gw\neQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x250ef842748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.lineplot(data = hist['val_acc'], label = 'val_accuracy')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can conclude from the above 'line plot' that the validation accuracy keep on increasing after every epoch. That's the good point."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 8. Test the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000/10000 [==============================] - 9s 949us/step\n",
      "Test Accuracy:  0.9896\n",
      "Test Loss:  0.03142303820503876\n"
     ]
    }
   ],
   "source": [
    "test_loss, test_acc = model.evaluate(test_images, test_labels)\n",
    "print(\"Test Accuracy: \", test_acc)\n",
    "print(\"Test Loss: \", test_loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test Accuracy is near to 99%"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 9. Make Prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True: 7 and Predicted: 7\n",
      "True: 2 and Predicted: 2\n",
      "True: 1 and Predicted: 1\n",
      "True: 0 and Predicted: 0\n",
      "True: 4 and Predicted: 4\n",
      "True: 1 and Predicted: 1\n",
      "True: 4 and Predicted: 4\n",
      "True: 9 and Predicted: 9\n",
      "True: 5 and Predicted: 5\n",
      "True: 9 and Predicted: 9\n"
     ]
    }
   ],
   "source": [
    "# Look at top ten samples of testing dataset\n",
    "for i in range(10):\n",
    "    preds = model.predict(test_images)[i]\n",
    "    true = test_labels[i]\n",
    "    print(\"True: {} and Predicted: {}\".format(true, np.argmax(preds)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 10. Mis-Classified Images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can plot the examples of some miss-classified images from the test dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def plot_mis_class_imgs():\n",
    "    test_preds = np.argmax(model.predict(test_images), axis = 1)\n",
    "    incorrect = (test_preds != test_labels)\n",
    "    # get images wrong classified\n",
    "    images = test_images[incorrect]\n",
    "    # true labels of wrong classified images\n",
    "    test_cls = test_labels[incorrect]\n",
    "    # predicted labels of wrong classified images\n",
    "    test_preds = np.argmax(model.predict(images), axis = 1)\n",
    "    print(test_cls)\n",
    "    print(test_preds)\n",
    "    print('{} percentage images are miss-classified.'.format((len(test_preds)/100)))\n",
    "    \n",
    "    # plot the images\n",
    "    plt.figure(figsize = (15, 2))\n",
    "    for i in range(5):\n",
    "        plt.subplot(1, 5, i+1)\n",
    "        plt.xticks([])\n",
    "        plt.yticks([])\n",
    "        plt.imshow(images[i].reshape(28,28), cmap = plt.cm.binary)\n",
    "        plt.xlabel(\"True: {} and Pred: {}\".format(test_cls[i], test_preds[i]))\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[6 5 6 8 5 7 4 8 6 7 7 9 9 5 8 8 5 1 8 3 9 5 7 8 9 5 1 5 3 6 4 6 1 1 9 2 9\n",
      " 5 6 5 8 5 9 9 8 1 8 6 9 6 5 8 7 8 1 9 9 2 8 9 7 3 4 5 8 9 1 6 8 8 3 8 9 4\n",
      " 8 8 8 5 3 5 5 5 9 8 9 1 7 0 8 0 5 1 8 0 8 7 7 0 9 4 6 9 5 4]\n",
      "[0 3 0 2 3 2 9 9 5 2 2 4 5 7 0 2 3 5 7 7 5 3 2 3 4 6 7 3 7 0 9 1 3 3 1 0 7\n",
      " 3 1 3 0 7 5 7 3 2 9 0 1 4 0 9 2 3 3 4 3 7 2 7 3 2 7 3 7 1 2 0 9 7 5 0 4 9\n",
      " 6 4 7 3 8 9 3 9 0 9 7 7 1 7 7 8 3 6 3 6 5 2 2 1 7 7 3 7 6 9]\n",
      "1.04 percentage images are miss-classified.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzcAAACPCAYAAADHoCVfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAG/VJREFUeJzt3Xl8TXf+x/FPIkOsDUJRIWMba4yd1t6F2loiGFWx1VJF\nO1TttKqWqbFUa6mk8lBKETOig6Ed66OpvUxRlMTSscROkETy+8PD/eXzxb25SW5yc+7r+dd5O+ee\n8018c3K+Oedzvl4pKSkpAgAAAAA5nHd2NwAAAAAAMgODGwAAAACWwOAGAAAAgCUwuAEAAABgCQxu\nAAAAAFgCgxsAAAAAlsDgBgAAAIAlMLgBAAAAYAkMbgAAAABYAoMbAAAAAJbA4AYAAACAJTC4AQAA\nAGAJDG4AAAAAWAKDGwAAAACWwOAGAAAAgCUwuAEAAABgCQxuAAAAAFgCgxsAAAAAlsDgBgAAAIAl\nMLgBAAAAYAkMbgAAAABYgk92N8BVDh8+rHLLli1VjouLsy3v2bNHratbt67rGgYAAADAJbhzAwAA\nAMASGNwAAAAAsAQGNwAAAAAswTI1N3379lV56dKlKiclJalcqVIl23KJEiVc1zAAAJDjeHl5qdyp\nUyeVU1JSVK5WrZrKkydPdk3DYHlbtmxROTExUeWoqCiV58+fn+5jBQYGquzn56fykCFDVO7Tp0+6\nj5VVuHMDAAAAwBIY3AAAAACwBK8U875qDlW6dGmVf//9d5VTP4YmIrJx40bbsnlLDgDg2Ny5c1Ue\nOnRoNrUEyHze3vrvv+Zjaublk7l+1apVKpuPtQGPrFixQuU33nhDZWcu1c1tCxUqpPKoUaNUfuGF\nF1Ru0KCBynny5Enzsd0Fd24AAAAAWAKDGwAAAACWwOAGAAAAgCXk2FdBv/322ypfvHhR5T/96U8q\nb9iwQWXqbJDZrl+/rnLHjh1V3rZtm93PFy9eXOWxY8eqnPp1jDExMWqd+ay3uf7nn3+2u70jjp4t\n79Chg225bNmyTu0b7uvOnTsqm89qnz59WmVqbmAlCxYssLt+3LhxKsfFxak8depUlam5wdPs27dP\n5YCAAJXPnDmT7n2b5+3Ro0ene185BXduAAAAAFgCgxsAAAAAlsDgBgAAAIAl5Nh5booVK6bylStX\nVDZrDGrUqOHyNsGzXL58WeXQ0FCVN23alKnHK1++vG357t27drc1ayXMeqDMrrn517/+ZVtu1aqV\nU/uG+/rvf/+rclBQkMq7d+9WuW7dui5vE+AuBg4cqPKXX36pcu3atVXes2ePy9sEa7h9+7bK5vVE\nSEjIUz+bO3duldevX6/ySy+9lMHWuT/u3AAAAACwBAY3AAAAACyBwQ0AAAAAS8gx89yEh4erbNYQ\ndO3aVeUqVaqked+///67yj/88IPd7Vu2bKlyqVKl0nwsWMfevXtVzuwaG9PJkydty87WzGRUxYoV\nVR4/frzKderUycrmIIsMGzZMZbN20dfXNyubA7g1szaxSZMm2dQS5HQFChRQ2ZwHxx7zGtUTamxM\n3LkBAAAAYAkMbgAAAABYAoMbAAAAAJaQY2pubt68qfKDBw9UbtCggco+PvpL27Bhg8rTp0+3LZ86\ndUqtO3funN22lC5dWuX8+fOr7O/vr/LQoUNVNueC+OMf/2j3eHBPVatWVdmsQ5k8eXJWNidDihcv\nrvLYsWNVfuedd7KyOcgmmzdvVtk8z5rzh2Wm3377TWWzrtKs6/rPf/6j8q5du5w6Xs2aNW3L7du3\nd+qzgIjI2rVrVTZrITt27JiVzYGFpK6xFRH5/PPPVbY3RWX37t1d0qachDs3AAAAACyBwQ0AAAAA\nS2BwAwAAAMASvFLsPbjnRsx5Nszns8+fP6+y+U7wkJAQle/fv5+JrXOOWauxfv16lQMDA7OwNXCV\nNWvWqGw+M7tt2zan9pf6R9XZeW6GDBmi8uzZs536PDzD8OHDVT5w4IDKjuYAM6WeQ+z111+3u61Z\nV2meo5977jmV4+LiVD5+/LhTbStWrJhtuWzZsmrd7t27ndoXPNPAgQNVXrRokcrJyclZ2RxYyLx5\n81Q2a7dNrVu3ti2btWB58uTJvIblENy5AQAAAGAJDG4AAAAAWAKDGwAAAACW4Lbz3ERERKgcExNj\nd/tPP/1UZfOZQ/P57caNG9uWR4wYodaZz3Y7a+XKlSovX75c5SNHjqj82WefqTxz5swMHR/uITg4\nWOXU9QciztfcZIT58xQfH6+y+aw4PIPZJw8dOqTy4sWLVd67d6/KZcqUUdmcL6lPnz62ZbOmxiz3\nPHHihN22hoaGqmzOwTNlyhS7nzddvnzZtly/fn2nPgvPkLqPiIhMnTpVZfM6w6ynBdLL2evA1DU5\nnlhjY+LODQAAAABLYHADAAAAwBIY3AAAAACwBLetubl48aLK5vPVplmzZtld/+yzz6q8ZMkS23K5\ncuWca5wDderUUblt27Yqt2jRQmXzfeYNGza0LZvz8yDnMGsKsrOu5caNGypHRUWpvHHjRpVTvzMf\n1tWjRw+Vt27dqvKAAQNUPnPmjMrLli1T2ay5yZ8/v2151apVap3582Ge801NmzZV+ezZsyqvXr1a\n5dOnT6uckJCgcuo+Hh4ebvfYsK7Y2FiVU89/9PXXX6t15vxg+fLlUzkr6yiB1FLXS9arV0+tK1y4\nsMre3ta/r2H9rxAAAACAR2BwAwAAAMASGNwAAAAAsAS3rbnJKPPZ7xUrVqic2XU29lSsWFFls/7H\nfNY8OjratkzNjXWY8xtlp0uXLqn83nvvqUzNjTX99NNPKpvz1tSqVUvladOmqWzOvVCkSBG7x1uz\nZo2zTUyzChUqqLx7926VBw8erLJZP1GyZEnbcuo6C3gWc46j1H3c7P9eXl4qjxkzRuXKlStncuvg\nqQ4cOKByjRo1VD5//rzKo0aNsi1/8MEHat1f/vIXldu0aWP32IGBgSqnnhcyp+DODQAAAABLYHAD\nAAAAwBK8Usz3cbqJGTNmqJz6ltuTmI8VDBo0SOVJkyZlSrsyg/m6SPPV0IUKFbItX79+PUvaBNeb\nO3euyu+++65Tnw8KCrItDxw4UK0LDg5W+ZVXXlH54MGDKpuPV5jMtr7zzjtpbifcV58+fVRO/Up8\nkccfOzMfV3RnJ0+eVLlSpUp2t+/du7dtOSwszCVtgvuJjIxU2Tx3VqlSxbZ89OjRp64Tca9HjWFt\nO3fuVNm8Bvjll19cduzp06erPHLkSJcdK7Nw5wYAAACAJTC4AQAAAGAJDG4AAAAAWIJlXgVtvurZ\nrGNxJ+YrTOEZQkNDVTZf9RgXF6dyp06dVG7fvr1t2d/f3+6xNm/erLL5anRHNTewho8++khl83XI\nTZo0UXnIkCEub1NmMesozefCzZq2Tz75ROVcuXK5pF3IXmadjPk6crOfmOfCzp0725bNOoZ//OMf\nKn/88ccqjxs3zrnGAmlkvo7ZfI1/fHy8bXnZsmVqndnH161bp7JZ223ue/z48SqXKVNG5W7duj2t\n2dmGOzcAAAAALIHBDQAAAABLYHADAAAAwBIsU3NTuXLl7G5Cmpm1FfAMzzzzjMpfffWVy47lqCYH\nnmHixIkqm89ee3vrv2/5+Ljvr4QxY8aobNaVffDBByq3atVKZV9fX9c0DNkqNjZW5bFjx6q8du1a\nlZs1a6ZyTEyMyt27d7ct37lzR62rWrWqymYtQmBgoMo9evR4cqOBDDLPZ6mzo9pJc966qKgolV97\n7TWVExISVJ4zZ47K1NwAAAAAgIswuAEAAABgCQxuAAAAAFiC+z5gbSGJiYkqm/MtmIKDg13ZHMCh\nyMhIlc1ndGENN2/eVPnChQsqlyhRIsvaYs6tMH/+fJWXLl2qcsmSJVXu2bOnyuXKlcvE1sFdmf/v\nO3fuVNmc4+vvf/+7yuacHanrFVPPHSIiUqVKFZU7duyo8pQpU1TOly+fyubcZUBWMGvHzFqxsLAw\np/YXFBSU4Ta5GnduAAAAAFgCgxsAAAAAlsDgBgAAAIAleKWkpKRkdyOe5PDhwyrXr19f5fv376s8\nY8YMlfv27aty4cKFM7F1zvn5559VrlWrlsoFCxZU+ccff7Qtm+/VB9LDnN/EzI4kJydnZnOQRZz9\nf3/55ZdV/uabb1QuUqRIutty6NAhlVetWqWyeQ5v06aNyvXq1VO5adOmKjdu3DjdbUPOcfnyZZXN\nmhpzHputW7e6uklPZbZt48aNKteuXTsrmwMPcf78eZX79eun8qZNm5zaX0hIiMpff/21yn/4wx+c\n2l9W4M4NAAAAAEtgcAMAAADAEhjcAAAAALAEt53npkaNGiqbc78sX75c5ZEjR6q8ePFilc15OgYO\nHGhb9vHJ2LchKSlJ5ZMnT6rctWtXu5/v3LmzytTZWINZzmY+b22qW7euysWKFcu0tmS05gY5U4UK\nFVQ26xXMeW42b96scrdu3VT+4osvVH7//fdVNs999o41dOhQlfft26dyqVKlVM5IvQ+sY+3atSqb\n5zJz7pnsZM7NdOTIEZWpuUFm2bNnj225S5cual1MTIxT+zI/v3LlynS3K7tw5wYAAACAJTC4AQAA\nAGAJDG4AAAAAWILb1tyYhg8frrL5nO2aNWtUPn78uMrm890//fSTbdnf39+ptrRs2VLlb7/9VuVl\ny5apXLRoUZVnz56tcosWLZw6PnKGBQsWqGzWfZkiIiJU7tGjR5qPZb7X3nzW21mDBg3K0OfhHk6c\nOKFynz59VC5QoIDK0dHRKm/ZskXlSpUqOXX81PMfDBs2TK178cUXVa5evbpT+4ZnMn9fm3nhwoUq\nBwQEqNypUyfXNExEIiMjVTZrhc3rFmfO8cjZZs6cqXK5cuVUbtKkicpmvz537pzK5vXFp59+als2\n54E0+52vr6/Kb775psrz5s2TnI47NwAAAAAsgcENAAAAAEtgcAMAAADAErxSzMk4cqjDhw+rPGPG\nDJXNupjExESXtaVEiRIqT5o0SeX+/fu77NhwH++++67Kn332md3tzXqGo0ePpvlYbdq0UXnTpk0q\nmz/mjua5mTNnjsqO6oWQM5i1iOZz32fOnFG5Q4cOKl+8eNGp43344Ye25bffftupzwJpYdawmnPc\nxcbGqjx69GiVx4wZk+5jf/zxxypPnz5d5Tt37qg8btw4lT/66KN0Hxs5S6NGjVROXfctIvLss8+q\nXKhQIZWvXr2q8pUrV556LPP3fbVq1VSeNm2ayu3atXvqvnIq7twAAAAAsAQGNwAAAAAsgcENAAAA\nAEuwTM2NI0eOHFF56tSptmVzXhpHSpcurbJZQ2O+275KlSpO7R/WYM7r4ajmpnjx4iqPHTtW5e3b\nt9uWzXmdHHFUc9OvXz+VFy1a5NT+AcAdmLWKr776qsp79+5V2Zl57sz5QI4dO6ZyXFycyubcJq6c\nYwfuLfU1p4iuRxRxPDeNI3ny5LEtm3XevXr1Utms77Ei7twAAAAAsAQGNwAAAAAsgcENAAAAAEvw\nmJobIKs5O8+NI6l/VJ19HrdMmTIqh4SEqGw+o5svXz7nGgcAbmjHjh0qmzWw9mpuzPqd+vXrq2zO\nmWPW3zpTzwPPsnr1apVHjhypckxMjN3Pv/baayqnruEJCgrKWOMsgDs3AAAAACyBwQ0AAAAAS+Cx\nNMBFzEcazNeAmredExIS7O7P3mNpuXPnVjkwMFDlyMhIlXk9OQAAsCLu3AAAAACwBAY3AAAAACyB\nwQ0AAAAAS6DmBsgmY8aMUXn69Ol2t0/9ozphwgS1rkKFCir36NEjg60DAADIebhzAwAAAMASGNwA\nAAAAsAQGNwAAAAAsgZobAAAAAJbAnRsAAAAAlsDgBgAAAIAlMLgBAAAAYAkMbgAAAABYAoMbAAAA\nAJbA4AYAAACAJTC4AQAAAGAJDG4AAAAAWAKDGwAAAACWwOAGAAAAgCUwuAEAAABgCbkmTZo0Kbsb\nkROMHz9evL29JTAw0KOOjZyLPouciH6LnIY+i5zG6n3Wx97KN99807ackJAgPj4+4u398GZP//79\npUmTJi5r2NP89ttvEhERIadPnxZfX18JDg6W1q1bZ3k7UluxYoX885//FB8fH8mVK5cEBARIz549\npWLFitnSnnXr1klUVJQkJCRIo0aNpF+/fuLjY/e/2jLcrc+m7huPzJo1S/z9/bO0HSZ36rM7duyQ\n1atXy/Xr1yV37txSq1Yt6dOnj/j6+mZ5W7KLu/VbEc61acG59iF36LMJCQkSHh4ue/fulQcPHkjl\nypXlrbfekiJFimRpO0zu1Gd/+OEH2bRpk1y4cEHy5s0rTZs2lW7dutn+36zO3fos1weOpff6wO5Z\neOnSpbblwYMHy4ABAyQoKOip2z948EBy5crlZNPT7saNGzJ16lTp1auXNGzYUBISEuTatWsuO54z\nGjduLIMHD5akpCRZtmyZzJw5UxYsWPDYdq7+Hu3fv1+ioqJk4sSJ4ufnJzNmzJDVq1dLt27dXHZM\nd+JufVbk//uGu3GXPlulShX58MMPxc/PT+7evSsLFy6UlStXSmhoqMuO6W7crd9yrnWMc6179dn1\n69fL6dOnZebMmZI3b16ZP3++LFmyRP7617+67Jhp5S59NjExUXr37i0VKlSQGzduyLRp06RAgQLS\noUMHlx3TnbhbnxXh+sCR9F4fZOhPTCtWrJD//e9/4uXlJfv375c+ffrIoUOHpESJEtKlSxcRETl0\n6JAsXLhQPv/8cxERuXLlioSHh8uxY8fE19dX2rdvn+a/BkZFRUnt2rWlcePGDxvv4yP58uV74rbJ\nyckya9YsOXbsmCQmJkpgYKD069dPSpcuLSIic+fOlQIFCsiFCxfk2LFjEhAQIMOGDZPixYuLiMjB\ngwflq6++kuvXr0vz5s3T/D3x8fGRZs2ayXfffSd37tyR6Oho2b59u5QtW1Z27Nghr776qnTp0kW+\n//57iYqKkhs3bkjFihWlf//+ttF6eo8tIrJ161Z56aWXbF9ncHCwzJ8/32N+4TqS1X3WGZ7aZ82/\nUnl5ecmFCxec2ofVca59XHb3W8619mV1n7106ZLUrFlTnnnmGRERef755+Wbb7554rae2mdbtWpl\nWy5atKg0btxYfv31V6f2YWVcHzwuu/tseq8PMnwvcvfu3dK4cWNZsmSJPP/883a3TU5OlmnTpkn5\n8uVl4cKFMn78eImKipLDhw+LiMiRI0ekb9++T/38iRMnJF++fDJ27Fjp16+fzJgxQ65cufLU7evU\nqSNz586VRYsWSUBAgMybN0+t37Vrl3Tt2lXCw8PF399fVqxYISIP/2o5c+ZMeeONNyQsLEyKFi0q\nJ06cSNP3IzExUbZu3SrFihWT/Pnzi4jIr7/+KqVLl5awsDB5/fXXJTo6WtatWycjR46UxYsXS4UK\nFWTu3LlpOvalS5ekV69ecvXq1Sce/9y5c1K2bFlbLlu2rFy9elXi4+PT1H5PkJV99tHxevfuLcOH\nD5fNmzfb3dYT+6zIw+9jaGiohIaGyt69e6Vt27Zparsn4VyrZXe/5VzrWFb22RdffFGOHTsm165d\nk3v37snOnTvlz3/+81O398Q+azp69KgEBASkaVtPwfWB5g59Nj3XBxke3FSuXFnq1q0r3t7ekjt3\nbrvbHj9+XO7evSudOnUSHx8fKVGihLRo0UJ27dolIiJVq1aVsLCwp37+ypUrsm3bNunbt6988cUX\nUqRIEds30OTt7S3NmzeXvHnzSu7cuSUkJEROnTol9+7ds23ToEEDKV++vPj4+EiTJk0kNjZWRET2\n7dsngYGBUr9+ffHx8ZF27dpJoUKF7H5tO3fulF69esmgQYPk7Nmz8v7779vW+fv7yyuvvGL7Hm3e\nvFk6duwopUqVkly5cklwcLCcPHlSrl696vDYxYsXlyVLljz1OeJ79+6pv7A+Wr57967d9nuSrOyz\nL7zwgsyePVvCwsLkrbfekm+//VZ+/PHHJ27rqX320fcxIiJC5s+fL+3bt8/2Z47dEefah9yl33Ku\ndSwr+2ypUqXEz89PBgwYIL169ZILFy5IcHDwE7f11D6b2pYtWyQ2NlbatWvncFtPwvXBQ+7UZ9Nz\nfZDhyseiRYumedu4uDiJi4uTXr162f4tOTlZqlatmqbP586dW6pXry7lypUTEZHOnTtL//795d69\ne48VFyUnJ8vy5cslOjpabt26JV5eXiIicuvWLdu2fn5+at+POsm1a9fU1+Xt7e3w67T33KT52bi4\nOAkLC5MlS5bY/s3Ly0uuXLmSrmOn5uvrq365PvorYt68edO8D6vLyj6b+q9ilStXltatW0t0dLQ0\natTosW09tc+axw0KCpI5c+bI1KlT07UPq+Jc+5C79FvOtY5lZZ/98ssvJTk5WcLDwyVPnjyydu1a\nmTZtmkyePPmxbT21zz4SHR0tK1eulAkTJkiBAgWc/ryVcX3wkLv12UfHTev1QYYHN4++wY/4+vpK\nQkKCLV+/fl01rGTJkjJr1qx0HatMmTLqeF5eXo8d/5Ft27bJgQMHZMKECVKsWDG5deuW9OvXT1JS\nUhwep3DhwnLw4EFbTk5OtvtIhiNmG4sWLSpdu3Z94i3Ps2fPZujYpUuXltjYWGnQoIGIiMTExEiR\nIkWe+ry8J8rKPvukYz+tD3pqnzU9ePBALl68mO7PWxXnWsc417qXrOyzMTEx0rNnT9vFeuvWrWX1\n6tVy584d2+M0j3hqnxV5+CKMxYsXy5gxY3gk7Qm4PkhbO1Nzx+uDTH//X2BgoOzfv19u374t165d\nkw0bNtjWVapUSXx8fGyvzkxOTpYzZ87IqVOn0rTvFi1aSHR0tMTGxkpSUpKsWbNGqlat+sRXwt29\ne1d8fHykYMGCcv/+fduzh2lRu3ZtiYmJkT179khSUpJ89913cvPmzTR/3pGXX35ZIiMj5dy5cyIi\ntiKtzDh2s2bN5Pvvv5fz58/L7du3JTIy0ukCLk/jyj67e/duuXPnjqSkpMiJEydkw4YNUq9evSdu\n66l9dvv27RIXFyciD5+/XblypVSvXj3T2m5VnGsd41zrXlzZZ8uXLy9bt26V+Ph4SUpKkn//+9/i\n7+//2MBGxHP77KFDh2TevHkyYsQI211Z2Mf1gWPueH2Q6S/kb968uRw+fFgGDx4sxYsXl2bNmtk6\nQ65cuWT06NESEREh69atk6SkJHnuuedsb5f55Zdf5G9/+5u6tZVazZo1pWvXrvLJJ59IQkKCVKlS\nRYYMGfLEbVu0aCGHDh2SAQMGSMGCBSUkJES2bNmSpq/Bz89P3nvvPYmIiJB58+ZJ8+bNM/X93o0a\nNZL79+/LrFmzJC4uTvLlyyc1a9aUhg0bOjz2xYsXZcSIETJnzpwnPqNYp04dadu2rUycOFESExOl\nYcOG0rlz50xruxW5ss/u3LlT5s+fL0lJSVK0aFEJDg5+6rv0PbXPnjlzRpYtWybx8fGSP39+qV27\ntnTv3j3T2m5VnGsd41zrXlzZZ0NDQyU8PFyGDh0qSUlJUrZsWRkxYsQTt/XUPrt69WqJj4+XKVOm\n2P6tWrVqMmrUqExrv9VwfeCYO14feKWk5Z4WAAAAALg5z5iWFgAAAIDlMbgBAAAAYAkMbgAAAABY\nAoMbAAAAAJbA4AYAAACAJTC4AQAAAGAJDG4AAAAAWAKDGwAAAACWwOAGAAAAgCX8HyYHVcEP5rwu\nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x250efb949b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_mis_class_imgs()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 11. Plotting Convolution Layer output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         (None, 28, 28, 1)         0         \n",
      "_________________________________________________________________\n",
      "layer_conv1 (Conv2D)         (None, 28, 28, 16)        416       \n",
      "_________________________________________________________________\n",
      "max_pooling2d (MaxPooling2D) (None, 14, 14, 16)        0         \n",
      "_________________________________________________________________\n",
      "layer_conv2 (Conv2D)         (None, 14, 14, 36)        14436     \n",
      "_________________________________________________________________\n",
      "max_pooling2d_1 (MaxPooling2 (None, 7, 7, 36)          0         \n",
      "_________________________________________________________________\n",
      "dropout (Dropout)            (None, 7, 7, 36)          0         \n",
      "_________________________________________________________________\n",
      "flatten (Flatten)            (None, 1764)              0         \n",
      "_________________________________________________________________\n",
      "dense (Dense)                (None, 128)               225920    \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 10)                1290      \n",
      "=================================================================\n",
      "Total params: 242,062\n",
      "Trainable params: 242,062\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We ccan count the indices from the above output summary, to get the layers we want."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.engine.input_layer.InputLayer at 0x250edf4b3c8>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The input layer at index 0\n",
    "layer_input = model.layers[0]\n",
    "layer_input"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.layers.convolutional.Conv2D at 0x250edf4b400>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# First convolutional layer at index 1\n",
    "layer_conv1 = model.layers[1]\n",
    "layer_conv1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.layers.convolutional.Conv2D at 0x250e34bf3c8>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Second convolutional layer at index 3\n",
    "layer_conv2 = model.layers[3]\n",
    "layer_conv2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Output of the first convolution layer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take an example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEIBJREFUeJzt3X9M1PUfB/DnHTYQieOOmE4mW3BOYsOcg1mmoXE2U3NE\nxUapWWvOYXPTlTP+0DbdxOXtWAtmyzJza4NMqVz5x2FA6pzX0NxkljotbdgJd9wwoQI+3z+any+n\n3OcDx33uc/h6Pv76HC8+n3vtM558fn/eFkVRFBCROFazGyAiczD8REIx/ERCMfxEQjH8REIx/ERC\nMfxEQjH8REIx/ERCMfxEQk0az8znzp3D/v37MTQ0hNLSUpSVlcWqLyIyWNRb/qGhIXzyySeorq6G\nx+PByZMncePGjVj2RkQGijr8ly9fxrRp0zB16lRMmjQJ8+fPh8/ni2VvRGSgqHf7A4EAMjMz1c+Z\nmZm4dOnSfb/n9Xrh9XoBADU1NdF+HRHF2LiO+UfD5XLB5XIZ/TVENEZR7/Y7HA50d3ern7u7u+Fw\nOGLSFBEZL+rw5+XlobOzE36/HwMDAzh16hSKiopi2RsRGcgynjf5tLe348CBAxgaGsLixYtRXl4e\ny96IyEDjCj8RTVy8w49IKIafSCiGn0gohp9IKIafSCiGn0gohp9IKIafSCiGn0gohp9IKIafSCiG\nn0gohp9IKIafSCiGn0gohp9IKIafSCiGn0gohp9IKIafSCiGn0gohp9IKIafSCiGn0gohp9IKIaf\nSCiGn0gohp9IKIafSKhJ45l5w4YNSElJgdVqRVJSEmpqamLVFxEZbFzhB4Dt27cjPT09Fr0QURxx\nt59IKIuiKEq0M2/YsAGpqamwWq1YsmQJXC5XLHsjIgONK/yBQAAOhwOhUAg7d+7E66+/joKCgrDf\n8Xq98Hq9AMBzAkQJZFzhH66xsREpKSlYuXJlLBZHRAaL+pi/v78ffX196vT58+eRk5MTs8aIyFhR\nn+0PhULYs2cPAGBwcBALFizAnDlzYtYYERkrZrv9RDSx8FIfkVAMP5FQDD+RUAw/kVAMP5FQDD+R\nUON+qk+KQ4cORax9/PHHmvNOnz5ds56SkqJZf/XVV9Xpxx9/HD///HNYfdq0aRHndTqdmssmubjl\nJxKK4ScSiuEnEorhJxKK4ScSiuEnEorhJxKKj/SO0qOPPhqxdu3atbj14fP5UFxcHPYzrbcn3/ta\nNSN9+umneOONN+L2fXpmzJihTu/atQvvvvuu+nnLli2a8xYVFRnWV6Lglp9IKIafSCiGn0gohp9I\nKIafSCiGn0gohp9IKD7PP0r79u2LWLv3+fp76V1r7+jo0KyfPXtWnXY4HFi1alVYvaWlJeK8p0+f\n1ly23kArv//+u2Z9uL/++kv3+4Z76KGHNOuPPPKIZr2zs1OzPryXLVu24Msvv1Q/D78HYCS8zk9E\nDyyGn0gohp9IKIafSCiGn0gohp9IKIafSCjd5/nr6+vR3t4Om80Gt9sNALh9+zY8Hg9u3bqFrKws\nbNq0CWlpaXFpmO4XDAYj1obfIzASvevZPp9v1H0UFxeP6feTk5M167NmzdKs5+fna9YDgYA6fe97\nEOrq6jTnraqq0qw/CHS3/IsWLUJ1dXXYz5qamlBYWIgPPvgAhYWFaGpqMqxBIjKGbvgLCgru26r7\nfD6UlJQAAEpKSsb0356IEkNUx/yhUAh2ux0AkJGRgVAoFNOmiMh4476332KxwGKxRKx7vV54vV4A\nQE1NzXi/jkbw8MMPR6zpHdNPmTJFs37v+wL1ljWW39f6uwH0xzBsbm7WrA8MDKjTjz32WNgeqt4z\nDRJEFX6bzYZgMAi73Y5gMKj5AkmXywWXyxV1g6Svt7c3Yu1BPuFXWlqqWecJP21R7fYXFRWhtbUV\nANDa2jqm//ZElBh0t/y1tbXo6OhAb28v1q9fj4qKCpSVlcHj8eD48ePqpT4imlj43n5KWF999ZVm\n/eWXX9asFxYWqtNffPEFXnnlFfXzDz/8oDmvw+EYRYcTG+/wIxKK4ScSiuEnEorhJxKK4ScSiuEn\nEoqX+sg0fr9fsz78Ul008x86dEidfuaZZ3D8+HH184svvjiKDh9s3PITCcXwEwnF8BMJxfATCcXw\nEwnF8BMJxfATCcUhusk0em/T0buOn5GRoVkf/iaglJQU3TcDScMtP5FQDD+RUAw/kVAMP5FQDD+R\nUAw/kVAMP5FQfJ6fDHXixImINb0Rd/755x/N+t2BYyJ5+umnNevScctPJBTDTyQUw08kFMNPJBTD\nTyQUw08kFMNPJJTu8/z19fVob2+HzWaD2+0GADQ2NqK5uRnp6ekAgMrKSsydO9fYTmlC+u677yLW\n9K7ju1wuzfqTTz4ZVU/0H93wL1q0CEuXLr3vxQvLly/HypUrDWuMiIylu9tfUFCAtLS0ePRCRHEU\n9Wu8jh07hra2NuTm5mLNmjX8B0E0wYzq3n6/34/du3erx/w9PT3q8X5DQwOCwSCqqqpGnNfr9cLr\n9QIAampqYtU3TRA3btyIWLt586bmvHf/xiJxOp2adauV57O1RLXlH/7ixNLSUuzevTvi77pcLt0T\nN/Tgqq+vj1jbtWuX5rx6fzdaJxMBhl9PVGsnGAyq02fOnMGMGTNi1hARxYfulr+2thYdHR3o7e3F\n+vXrUVFRgQsXLuDatWuwWCzIysrCunXr4tErEcUQn+encenr61Onk5OT8ffff4fVn3rqqYjzdnR0\naC77+PHjmvX58+ePokOKhAdFREIx/ERCMfxEQjH8REIx/ERCMfxEQnGIbhqX999/X51+8803sW/f\nvrD62bNnI8773HPPaS6bl/KMxS0/kVAMP5FQDD+RUAw/kVAMP5FQDD+RUAw/kVB8pJc0HT16VLP+\nwgsvqNOnT5/GE088EVafMmVKxHm///57zWXz1dzG4pafSCiGn0gohp9IKIafSCiGn0gohp9IKIaf\nSCg+zy9cd3e3Zn3jxo2a9YGBAXVaUZSwzwCwbNmyiPPyOr65uOUnEorhJxKK4ScSiuEnEorhJxKK\n4ScSiuEnEkr3On9XVxfq6urQ09MDi8UCl8uFZcuW4fbt2/B4PLh16xaysrKwadMmpKWlxaNnGoPB\nwUHN+tKlSzXrV69e1aw7nU51Ojk5OewzAOzYsUOnQzKLbviTkpKwevVq5Obmoq+vD1u3bsXs2bPR\n0tKCwsJClJWVoampCU1NTVi1alU8eiaiGNDd7bfb7cjNzQUATJ48GdnZ2QgEAvD5fCgpKQEAlJSU\nwOfzGdspEcXUmI75/X4/rl69CqfTiVAoBLvdDgDIyMhAKBQypEEiMsao7+3v7++H2+3G2rVrkZqa\nGlazWCywWCwjzuf1euH1egEANTU142iVomG1av9/P3DggGb9zp07mvXk5GR1Oi8vD4cPHw6rZ2dn\n63RIZhlV+AcGBuB2u7Fw4ULMmzcPAGCz2RAMBmG32xEMBpGenj7ivC6XCy6XK3Yd05gMDQ1p1l97\n7TXN+k8//aRZH36C7/DhwygvLw+rHzt2LOK8eXl5mssmY+nu9iuKgr179yI7OxsrVqxQf15UVITW\n1lYAQGtrK4qLi43rkohiTvfV3RcvXsS2bduQk5Oj7tpXVlZi5syZ8Hg86Orq4qW+BPbrr79q1mfN\nmjWu5X/zzTfq9MKFC/Hjjz+G1Z9//vlxLZ+Mo7vbn5+fj8bGxhFr27Zti3lDRBQfvMOPSCiGn0go\nhp9IKIafSCiGn0gohp9IKA7R/QD47bffItbuPnwVzbwAsGfPHs365s2bNeuRbvsm83HLTyQUw08k\nFMNPJBTDTyQUw08kFMNPJBTDTyQUh+h+AHz00UcRa3rX8fXo3SfA6/gTF7f8REIx/ERCMfxEQjH8\nREIx/ERCMfxEQjH8RELxOv8EMPxd+HPmzMG5c+fC6h9++GG8W6IHALf8REIx/ERCMfxEQjH8REIx\n/ERCMfxEQjH8RELpXufv6upCXV0denp6YLFY4HK5sGzZMjQ2NqK5uRnp6ekAgMrKSsydO9fwhiU6\nceKEOu10OsM+A0Bvb2/Uy3Y6nZr1tLS0qJdNiU03/ElJSVi9ejVyc3PR19eHrVu3Yvbs2QCA5cuX\nY+XKlYY3SUSxpxt+u90Ou90OAJg8eTKys7MRCAQMb4yIjDWm4br8fj+2b98Ot9uNo0ePoqWlBamp\nqcjNzcWaNWtG3EX0er3wer0AgJqamth1LkhnZ6c6nZmZie7u7rD6H3/8EfWyk5OTNet6hwWTJ0+O\n+rvJXKMOf39/P7Zv347y8nLMmzcPPT096vF+Q0MDgsEgqqqqDG1Wql27dqnTa9euxWeffRZWr66u\njnrZeuH+9ttvNev5+flRfzeZa1Rn+wcGBuB2u7Fw4ULMmzcPAJCRkQGr1Qqr1YrS0lJcuXLF0EaJ\nKLZ0w68oCvbu3Yvs7GysWLFC/XkwGFSnz5w5gxkzZhjTIREZQveE3y+//IK2tjbk5OTgnXfeAfDf\nZb2TJ0/i2rVrsFgsyMrKwrp16wxvlsZuzpw5mvXm5mbNusPhiGU7lEB0w5+fn4/Gxsb7fs5r+kQT\nG+/wIxKK4ScSiuEnEorhJxKK4ScSiuEnEmpM9/YT0YODW34ioRh+IqEYfiKhGH4ioRh+IqEYfiKh\nGH4ioRh+IqFMDf/WrVvN/HpNidpbovYFsLdomdUbt/xEQjH8REIlvffee++Z2UBubq6ZX68pUXtL\n1L4A9hYtM3rjgz1EQnG3n0go3bf3GuHcuXPYv38/hoaGUFpairKyMjPaGNGGDRuQkpICq9WKpKQk\nU4cYq6+vR3t7O2w2G9xuNwDg9u3b8Hg8uHXrFrKysrBp0yZTRtIdqbdEGbk50sjSZq+7hBvxWomz\nwcFB5a233lJu3ryp/Pvvv8rbb7+tXL9+Pd5tRFRVVaWEQiGz21AURVEuXLigXLlyRdm8ebP6s4MH\nDypHjhxRFEVRjhw5ohw8eDBhemtoaFC+/vprU/oZLhAIKFeuXFEURVHu3LmjbNy4Ubl+/brp6y5S\nX2att7jv9l++fBnTpk3D1KlTMWnSJMyfPx8+ny/ebUwIBQUF922ZfD4fSkpKAAAlJSWmrbuReksU\ndrtdPYE2fGRps9ddpL7MEvfd/kAggMzMTPVzZmYmLl26FO82NO3YsQNWqxVLliyBy+Uyu50woVBI\nHTI9IyMDoVDI5I7CHTt2DG1tbZojN8eT3+/H1atX4XQ6E2rdDe/r4sWLpqw3U475E9mOHTvgcDgQ\nCoWwc+dOTJ8+HQUFBWa3NSKLxQKLxWJ2G6pnn30WL730EoD/Rm7+/PPPTR25ub+/H263G2vXrkVq\nampYzcx1d29fZq23uO/2OxyOsPHlu7u7E2o8uLu92Gw2FBcX4/LlyyZ3FM5ms6mDpAaDQfUkUSJI\npJGbRxpZOhHWXSKNeB338Ofl5aGzsxN+vx8DAwM4deoUioqK4t3GiPr7+9HX16dOnz9/Hjk5OSZ3\nFa6oqAitra0AgNbWVhQXF5vc0f8lysjNSoSRpc1ed5H6Mmu9mXKTT3t7Ow4cOIChoSEsXrwY5eXl\n8W5hRH/++Sf27NkDABgcHMSCBQtM7a22thYdHR3o7e2FzWZDRUUFiouL4fF40NXVZeqlvpF6u3Dh\nwn0jN989xo6nixcvYtu2bcjJyVF37SsrKzFz5kxT112kvkYa8Toe6413+BEJxTv8iIRi+ImEYviJ\nhGL4iYRi+ImEYviJhGL4iYRi+ImE+h98gZznGX4unQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x250efd888d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "rescaled_data = test_images[0].reshape(28, 28)\n",
    "plt.imshow(rescaled_data, cmap=plt.cm.binary)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Take an example of test image at index 0 and,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAERVJREFUeJzt3W9MU2cbBvCrxUUkhFKwkYzIFDWRJppNQRIjoqEyo8Yg\nGhLmdG7J1OiyzG3ZHHO6BD/gIqtZInMum3EuS2RR2Kc5U5eB0WR064yJiFEim0t0/GlpcMImcN4P\nb97ztkCfg6X/4L5+n87pzWnvnHntnJ7n9DwmTdM0EJE45ng3QETxwfATCcXwEwnF8BMJxfATCcXw\nEwnF8BMJxfATCcXwEwnF8BMJNW0iG1+7dg2nTp3C8PAwSkpKUFZWFqm+iCjKwj7yDw8P44svvkBV\nVRWcTieuXLmCP//8M5K9EVEUhR3+O3fuICsrC7NmzcK0adOwfPlyuN3uSPZGRFEU9mm/1+tFZmam\nvp6ZmYnbt2+P+juXywWXywUAqKmpCffjiCjCJvSdfzwcDgccDke0P4aInlDYp/0ZGRno6enR13t6\nepCRkRGRpogo+sIO/7x583D//n10dnZicHAQV69eRX5+fiR7I6IoMk3kST4ejwenT5/G8PAwVq9e\njfLy8kj2RkRRNKHwE9HkxTv8iIRi+ImEYviJhGL4iYRi+ImEYviJhGL4iYRi+ImEYviJhGL4iYRi\n+ImEYviJhGL4iYRi+ImEYviJhGL4iYRi+ImEYviJhGL4iYRi+ImEYviJhIr6jD0UX7/++quy3tDQ\noKyfO3dOWb9165a+3NLSgmXLlgXVVQ+HNplMyvdeunSpsp6Xl6esv/fee/rynDlz0NHRMe5tJeCR\nn0gohp9IKIafSCiGn0gohp9IKIafSCiGn0ioCY3z7927F8nJyTCbzUhKSkJNTU2k+ppSTp48qay3\ntbUp65cvX9aXT58+jZdeemncn200zm801m40ifPI7Ueu79q1K+S2mzZtUr53aWmpsv6kOLYfbMI3\n+Rw6dAhpaWmR6IWIYoin/URCTfjIX11dDbPZjDVr1sDhcESiJyKKAZNm9KVOwev1IiMjA36/H4cP\nH8bLL78Mu90e9DculwsulwsAxF4T6OrqUtYHBgaU9b6+Pn157ty5uHv37rg/+9GjR+P+24nKy8vD\nzZs3g16z2Wwh/z49PV35fhaLJSJ90dgmFP5A9fX1SE5OxsaNGyPxdlOKlAt+P//8MwoLC4PqO3fu\nDLltrC/4UbCwv/MPDAygv79fX75+/TpycnIi1hgRRVfY3/n9fj+OHj0KABgaGsKKFSvw7LPPRqwx\nIoquiJ32U2hGp9ZG9ZSUFH25ubkZK1euDKqrxq+LioqU771w4UJlfebMmcp6eXm5sk6Ji0N9REIx\n/ERCMfxEQjH8REIx/ERCMfxEQvHR3TFgNBzW2NiorAcO5SUnJ48a2nO73eE3R2LxyE8kFMNPJBTD\nTyQUw08kFMNPJBTDTyQUw08kFMf5Y+DEiRPKusfjUdZ///13ffnff/8NWgeAP/74I+S2fMAKhcIj\nP5FQDD+RUAw/kVAMP5FQDD+RUAw/kVAMP5FQHOePAdWUVQDw6quvKusHDhzQlwcHB9Hd3R1UH7ke\niOP8FAqP/ERCMfxEQjH8REIx/ERCMfxEQjH8REIx/ERCGY7z19XVwePxwGKxoLa2FgDw8OFDOJ1O\ndHV1wWazYd++fUhNTY16s1PV8PCwsj5yFvWR662trePeNtIC5xCYPn06/vnnn6B64PTilFgMj/yr\nVq1CVVVV0GuNjY1YtGgRPvnkEyxatMhw0gkiSjyG4bfb7aOO6m63G8XFxQCA4uJizhhDNAmF9Z3f\n7/fDarUCANLT0+H3+yPaFBFF34Tv7TeZTDCZTCHrLpcLLpcLAFBTUzPRj5uSXnnlFWV97dq1+nJe\nXt6oM625c+eG3DY5OXlizRmYPn26vmw2m4PWKbGFFX6LxQKfzwer1Qqfz4e0tLSQf+twOOBwOMJu\nUIIvv/xSWf/ggw/0ZbfbjYKCgqD6mTNnQm47clLPSOMFv8krrNP+/Px8NDU1AQCamppG/WMkosRn\neOQ/duwYWltb0dfXh927d6OiogJlZWVwOp348ccf9aE+IppcTFq0B4IJXV1dyvqyZcuU9cDn9Le0\ntIz6e9U1F6P/vKptx7P9pk2b9OUjR47g3XffDapv3bo15Lbl5eXK96bo4h1+REIx/ERCMfxEQjH8\nREIx/ERCMfxEQvHR3RFgNJS3cuVKZX3klNsjLV26VF9OSUkJWgfUd/GtWLFC+d5GPv/8c2U9cHrx\nR48ejZpu/Pz58yG3NRpmNPrBmNHdi7y7UI1HfiKhGH4ioRh+IqEYfiKhGH4ioRh+IqEYfiKh+JPe\nCLh8+bKy/r+HnYayefNmZf3bb7994p5iJXB6cIvFMup5jl9//XXIbY2e+tzc3Kys2+12ZT1wv82Z\nMwcdHR36erSfcDQZ8MhPJBTDTyQUw08kFMNPJBTDTyQUw08kFMNPJBTH+SlhnTx5Ulk3etZA4HMS\nLl68iNLSUn39+++/V2478pkJUxGP/ERCMfxEQjH8REIx/ERCMfxEQjH8REIx/ERCGY7z19XVwePx\nwGKxoLa2FgBQX1+PS5cuIS0tDQBQWVmJJUuWRL9bogCBzxIYS+B8CfX19aioqNDXvV6vctu6ujpl\nfSpML244aceqVauwdu1aHD9+POj19evXY+PGjVFrjIiiy/C03263IzU1NRa9EFEMhT1d14ULF9Dc\n3Izc3Fxs376d/4MgmmTGdW9/Z2cnjhw5on/n7+3t1b/vnz17Fj6fD3v27BlzW5fLBZfLBQCoqamJ\nVN9EePz4sbJ+69YtfXnevHlob2/X14eGhpTb5uTkKOtWq3UcHSa2sI786enp+nJJSQmOHDkS8m8d\nDgccDkc4H0OkNPJhoSMFXuDjBb/Rwhrq8/l8+nJLSwtmz54dsYaIKDYMj/zHjh1Da2sr+vr6sHv3\nblRUVODGjRvo6OiAyWSCzWbDzp07Y9ErEUUQf89PU1bgc/+fe+45/Pbbb/r6W2+9pdw28FkAY6mq\nqlLW33jjjXF0GF+8w49IKIafSCiGn0gohp9IKIafSCiGn0goDvWRSE/yc+CxBN46PBaj24cTAY/8\nREIx/ERCMfxEQjH8REIx/ERCMfxEQjH8REKF/Qw/osls5syZynpRUZGy3tbWFsl24oJHfiKhGH4i\noRh+IqEYfiKhGH4ioRh+IqEYfiKhOM5PIt28eVNZb2xsVNbtdnsk24kLHvmJhGL4iYRi+ImEYviJ\nhGL4iYRi+ImEYviJhDIc5+/u7sbx48fR29sLk8kEh8OBdevW4eHDh3A6nejq6oLNZsO+ffuQmpoa\ni54nHafTqazbbDZl/cUXX4xkO2IETrOdlZWFBw8e6Ovvv/++ctu///5bWW9qappYcwnAMPxJSUnY\ntm0bcnNz0d/fj/3792Px4sX46aefsGjRIpSVlaGxsRGNjY38R0o0iRie9lutVuTm5gIAZsyYgezs\nbHi9XrjdbhQXFwMAiouL4Xa7o9spEUXUE33n7+zsxN27dzF//nz4/X5YrVYAQHp6Ovx+f1QaJKLo\nGPe9/QMDA6itrcWOHTuQkpISVDOZTDCZTGNu53K54HK5AAA1NTUTaHXyeuGFF5T1adP4E4toyMrK\n0pefeuqpoPWPPvpIue2BAweU9WeeeWZizSWAcf2rGxwcRG1tLYqKilBYWAgAsFgs8Pl8sFqt8Pl8\nSEtLG3Nbh8MBh8MRuY4noW+++UZZ5wW/6Ai8wDfygt8777yj3PbixYvK+i+//KKsL1y4cBwdxpfh\nab+maThx4gSys7OxYcMG/fX8/Hz9imdTUxMKCgqi1yURRZzhFN1tbW04ePAgcnJy9FP7yspKLFiw\nAE6nE93d3eKH+s6fP6+sb9myRVnftWuXsv7pp58+cU+JoqurK2StoaFhQu9ttL3H49GXf/jhBzz/\n/PP6utHZVnV1tbK+adOmcXSY2AxP+xcuXIj6+voxawcPHox4Q0QUG7zDj0gohp9IKIafSCiGn0go\nhp9IKIafSCjeVxoDBrdS4LPPPlPWz507py9fvHgRpaWlQfXy8vKwP9toqunMzExlPfAR1y0tLVi2\nbNm4Pz/ULeHj2RYA8vLylPWtW7fqy5mZmUHrVVVVym2NpvCeCnjkJxKK4ScSiuEnEorhJxKK4ScS\niuEnEorhJxKK4/wRoBpnB4ALFy4o60bTQQeyWq3YvHlz0Guq37V3dnYq389oqmmjsfjAZxHYbLZR\nzyZQjZdP9DfxRk/LGfm4uY8//nhCnzfV8MhPJBTDTyQUw08kFMNPJBTDTyQUw08kFMNPJJThc/uJ\naGrikZ9IKIafSCiGn0gohp9IKIafSCiGn0gohp9IKMPf83d3d+P48ePo7e2FyWSCw+HAunXrUF9f\nj0uXLiEtLQ0AUFlZiSVLlkS9YSKKDMPwJyUlYdu2bcjNzUV/fz/279+PxYsXAwDWr1+PjRs3Rr1J\nIoo8w/BbrVZYrVYAwIwZM5CdnQ2v1xv1xogoup7oMV6dnZ24e/cu5s+fj7a2Nly4cAHNzc3Izc3F\n9u3bkZqaOmobl8sFl8sFAKipqYlM10Q0YeO+t39gYACHDh1CeXk5CgsL0dvbq3/fP3v2LHw+H/bs\n2RPVZokocsZ1tX9wcBC1tbUoKipCYWEhACA9PR1msxlmsxklJSVob2+PaqNEFFmG4dc0DSdOnEB2\ndjY2bNigv+7z+fTllpYWzJ49OzodElFUGJ72t7W14eDBg8jJydEf41xZWYkrV66go6MDJpMJNpsN\nO3fu1C8MElHi4+/5iYTiHX5EQjH8REIx/ERCMfxEQjH8REIx/ERCMfxEQjH8REIx/ERCMfxEQjH8\nREIx/ERCMfxEQjH8REIx/ERCxTX8+/fvj+fHKyVqb4naF8DewhWv3njkJxKK4ScSKunDDz/8MJ4N\n5ObmxvPjlRK1t0TtC2Bv4YpHb3yGH5FQPO0nEuqJpuuKlGvXruHUqVMYHh5GSUkJysrK4tHGmPbu\n3Yvk5GSYzWYkJSXFdYqxuro6eDweWCwW1NbWAgAePnwIp9OJrq4u2Gw27Nu3b8xp0uLRW6LM3Bxq\nZul477uEm/Fai7GhoSHttdde0x48eKA9fvxYe/vtt7V79+7Fuo2Q9uzZo/n9/ni3oWmapt24cUNr\nb2/X3nzzTf21M2fOaA0NDZqmaVpDQ4N25syZhOnt7Nmz2nfffReXfgJ5vV6tvb1d0zRNe/Tokfb6\n669r9+7di/u+C9VXvPZbzE/779y5g6ysLMyaNQvTpk3D8uXL4Xa7Y93GpGC320cdmdxuN4qLiwEA\nxcXFcdt3Y/WWKKxWq34BLXBm6Xjvu1B9xUvMT/u9Xi8yMzP19czMTNy+fTvWbShVV1fDbDZjzZo1\ncDgc8W4niN/v12dGSk9Ph9/vj3NHwcYzc3MsBc4snUj7LpwZryMtLt/5E1l1dTUyMjLg9/tx+PBh\nPP3007Db7fFua0wmk0mfQi0RlJaWYsuWLQD+O3PzV199FdeZmwcGBlBbW4sdO3YgJSUlqBbPfTey\nr3jtt5if9mdkZKCnp0df7+npQUZGRqzbCOl/vVgsFhQUFODOnTtx7iiYxWLRJ0n1+Xz6RaJEkEgz\nN481s3Qi7LtEmvE65uGfN28e7t+/j87OTgwODuLq1avIz8+PdRtjGhgYQH9/v758/fp15OTkxLmr\nYPn5+WhqagIANDU1oaCgIM4d/V+izNyshZhZOt77LlRf8dpvcbnJx+Px4PTp0xgeHsbq1atRXl4e\n6xbG9Ndff+Ho0aMAgKGhIaxYsSKuvR07dgytra3o6+uDxWJBRUUFCgoK4HQ60d3dHdehvrF6u3Hj\nRkLM3BxqZukFCxbEdd8l2ozXvMOPSCje4UckFMNPJBTDTyQUw08kFMNPJBTDTyQUw08kFMNPJNR/\nAI5G/0v/qB9uAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x250efc00978>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "rescaled_data = test_images[15].reshape(28, 28)\n",
    "plt.imshow(rescaled_data, cmap=plt.cm.binary)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Take another example of test image at index 15."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we are ready to see the output of first convolution layer on above two examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "output_conv1 = K.function(inputs=[layer_input.input],\n",
    "                          outputs=[layer_conv1.output])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After this we can call the below function with the input image. Note that the image is wrapped in two lists because the function expects an array of that dimensionality. Likewise, the function returns an array with one more dimensionality than we want so we just take the first element."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1, 28, 28, 16)"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layer_output1 = output_conv1([[test_images[0]]])[0]\n",
    "layer_output1.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can plot the output of all the 16 channels of the convolutional layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def plot_conv_output(values):\n",
    "    num_filters = values.shape[3]\n",
    "    num_grids = math.ceil(math.sqrt(num_filters))\n",
    "    fig, axes = plt.subplots(num_grids, num_grids)\n",
    "    for i, ax in enumerate(axes.flat):\n",
    "        if i < num_filters:\n",
    "            # get the output image of the ith filter\n",
    "            img = values[0, :, :, i]\n",
    "            \n",
    "            ax.imshow(img, interpolation='nearest', cmap='binary')\n",
    "        # Remove ticks from the plot.\n",
    "        ax.set_xticks([])\n",
    "        ax.set_yticks([])\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUoAAADuCAYAAABf005JAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xlw1dX9//FnIBCWSAKBsIQlLLILiIispQLWvVqt1brV\nqW2tHTpTdVpn2pnW1o79trYzdkZLpzN1H9uhWOtS3BcEK6CCKEiIkLAKISuQjay/Pz6/9/ncyL35\n3JC7JDevxz+B5Obmk3Nvzud9znmf90lrbW1tRUREIuqV7AsQEenq1FGKiARQRykiEkAdpYhIAHWU\nIiIB1FGKiARQRykiEkAdpYhIAHWUIiIB1FGKiARQRykiEkAdpYhIAHWUIiIB1FGKiARQRykiEkAd\npYhIAHWUIiIB0uP55E1NTQAUFBRQWVkJQFpaGgBDhw4FIDc3l8zMTAD69u0bz8vp9srLy6mqqgKg\ntrYWgMzMTHr37g14bQnQp08fAPd5iayxsRGAsrIyjh8/Dvht279/f8aMGeP+DWrTWGpubu427ZmW\n6KMgTp06BcDHH38MQElJies87c3YntbWVuySW1paAMjLy2PSpElRP0d3s2vXLgBeeOEF3nrrLQCO\nHTsGeDejZcuWATB69GgAZs6c6b63X79+gPemjCQ93btf5uTkMGzYMMC/kWVkZMTs9+iKiouLAXj7\n7bc5fPgwgLsZzZ49m/HjxwMwcOBAAAYMGAD4QQBAQ0NDxOfv1csbtGVlZbkbmT1XT1dQUEBNTQ3g\nt9OECRMAr706yvoDe65Y0tBbRCRAzIbedXV1QHBEZxHKBRdc0OmfaVHVjh07GDx4MOBHVanEhoTb\ntm1zUc/OnTsBuPTSS91Q0T737rvvAvD555+7O/aRI0cAb1huEbzdge01u+CCC7j++usBWLRoEQDT\np0+P42+WfEePHgVg9+7dbhho76Fx48a5aYzS0lLAjzYrKyvd6MhGOA0NDW76yNrYnnPKlClkZ2cD\niig3btwIwLp16zj77LMBfwqkoKAAgMGDBzNo0CAAN8qprq5271nrR+rr6wHYsmWLGx3Ycy5ZsiRm\n719FlCIiAWIWUe7btw/woppvfOMbQPwnvm3OZ8qUKe4OnooWLFgAwKpVq9xdc/bs2QDMmjXLPc6i\nI7vLFhQUuAi0vLwc8OaI7U5tXysrKwNg7NixfPvb3wZ6TtRjc7d5eXmcddZZAFx44YUA5OfnR/y+\nuro69uzZA/iR+cmTJxkyZAjgR57WxmPGjHGjnp6otbXVRZL2fluwYIGbQ7d5xS1btgBeBF9UVATg\n2rS0tNQtCl955ZWA387r169nzZo1AFxzzTWANzKYPHky4M/Dn6mYdZR2wWvXruXee+8FcBd58cUX\nu9XDq6++GohtJ5qXl+ferKlsyZIlLFmyJOLXR4wY0eb/7f2hh7I3cE5OTo/pII11jgsXLnRD6Wja\nrX///pxzzjmBj9u7dy8AEydOPPOL7MassyspKXHZLXZzam5udp+zDjO0nWxKyF6XzMxMt9hjU0R2\ns9q1a5cLGk6cOAF4fUys+gUNvUVEAsQsorRJ7JkzZ7Jt2zYANm3aBMAXX3zhJsO/+c1vAv6E+cKF\nC91dxRYeRowY4YaHdveZN28e4IfVXxaPlIBUZ4thb7zxBuDdge+44w7An9ZIdRbBZGZmtkn56awE\nZ911WTbdkJOT40aRNg1UU1NDdXU14LeXRfOtra1uCin0dVm5cmWb57evbdq0yT2/TVXl5eXFLDdb\nvYuISICYRZRTp04FvIlX6/UtYmltbeXgwYOAP1l76NAhwIs2bV7M5iIOHDjAyJEjAT81oKKiAmib\ngiFn5p133gHgxRdfBLxFCNNTIkljoxno/IQ/+KOin//854C/6NZT5yjDLWCFtrmxeUXrC2pqatxo\n0l4Xm08O9bvf/Q7wolSbv5w7dy4A06ZN6+zlO4ooRUQCxCyitDvB+PHj3bavULb6dOuttwJ+gmlF\nRYW7Y1hEWVtbe9rWu2hWGCXY9u3bXRqFrcjafPCjjz6atOtKBdu3b+fBBx8E4L333gPg3HPPTeYl\ndRv2Hoz0/y+zTI3XX38d8NY1zj//fMDvi2KZWRPXohihbLHlyw2Qk5Nz2mNbWlq0OBNjNg3y2muv\nuVzMvLw8wN8lFW5oI8EsL/CVV17hzTffBPw2ve2225J1WSnt/vvvB/xOcdSoUW7ovXjx4pj/PPVG\nIiIBEhZRRsOG4+Xl5W7BxrLypXP2798PeLsbbM+x7YO1lCA5M9a277zzjhshWUK0xNarr74KwObN\nmwHcNN/y5ctZuHAhEJ9dZYooRUQCdKmI0rZB7tu3z9VDVETZObZAZulYZWVljBo1CvBTusKxGotK\nxYrMqja9//77gFfnwLaY2p55ia3HHnsM8GscWB3a/Px8t006HrpURxm6gNNdKh93dbY76oMPPgC8\nhTIrUXXFFVdE/D51kO1ramri008/BbxyYeDtV/7617+ezMtKaa+99por+G0dpN3sp02bFjY/M1Y0\n9BYRCdClIkqbCM/KyorJLomerqCgwE1+b926FfCKmqZ6Md5EOHDgAC+99BLgH2uycuVKbrjhhmRe\nVkq799573UjH+grbfWMRZrwoohQRCZD0sK21tfW0oruKJjvHKjW9/fbbrjKQVWtKS0vj5ptvTtq1\ndXeWXP7666/z7LPPAn7dxLvvvjtp15XK7rrrLsCrbWk7nayamKUH2ZEd8aKIUkQkQNJDt1Q+wiFZ\nNmzYAHhb6r4cnT/wwAPJuKRuz+olbt++HYA1a9a4Woo33XQTAHPmzEnOxaUom1d/6KGHAMjOzuYr\nX/kK4B1bAm2PZo6npHWUdrJgenp6jzt+IF5sh8hnn30GeGfoWBHTq666Cghf4kqCWR6qpQJ98skn\nrFixAvBLfUls3X777YBfg+CKK65oc8JiImnoLSISIGkRZVZWFuAV7LQSbFa8MyMjI/B8cPFZu738\n8suAXxx52LBhLnF/+fLlybm4bs5GPuvXrwdwCzjDhw93JQMl9u699153sqvt0lu+fDlLly4Fwlcd\niydFlCIiARIWUdpk+JcXb0LrU1pay7BhwxRRdoAlPFuxWJurzM/PZ9WqVRG/L9JrIp6WlhY332vH\nZljb3nnnnVx22WVJu7ZUt3r1atcHWErQ6NGjXQ3VREtYR2nD6/b2cNtErUSvrKyMjz76CPDPOC4o\nKAC8trZipuGog2xfZWWlyyCwc4as8r4t5Ehs2WJZY2OjG3JXVlYCXv2BeOdLRqKht4hIgIRElA0N\nDZSUlHg/8P/n9dkpi9I5VVVV7hRFa9sZM2YAKsjbWaWlpXz44YdtPrdo0SIArr322mRcUsqqq6sD\nYO3atQDU19e7z1mKWzJT2xRRiogESEhEWVlZ6U5NC00elc4bPXq0S5mweeDLL78cwJ1KJ2emrq7O\nHZthbaqUoPh4++23AX9Bd/Hixe6MeTuo7bzzzkvOxaGIUkQkUFqr5YjE0Y4dO/j1r38N+IcD2Z7N\nv/71r66yjUhXUlhYyI4dOwB/JHTRRRcl85JSSmtrKzt37gRwaxiWwbFo0SJXIcgyDZIpIUPv9PR0\nt4ndzmKxDnPdunX84Ac/SMRlCFBdXa393lEaMmQIc+fOBXQ0RjyUlpZSUVEB+GfMh6Zd2ZlEXYGG\n3iIiARIy9BYR6c4UUYqIBFBHKSISQB2liEgAdZQiIgHUUYqIBFBHKSISQB2liEgAdZQiIgHUUYqI\nBFBHKSISQB2liEgAdZQiIgHUUYqIBFBHKSISQB2liEgAdZQiIgHUUYqIBEjImTnhNDU1eReQnrRL\n6Jaqq6sBOHnyJOCdK2LnuQwcOLDNx4yMjCRcYfdi7VlUVMSJEycA3JlC48ePJysrK2nXlkqam5vp\n3bt3si/jjMWsl7JDw6I9hCmWHWR1dbXrHNLS0mL2vF1NcXGxO7Vu3759gNeOI0eOBHB/1EOGDAGg\nV69erp3tzO9w7Gutra0MGDCgzXOcddZZKX0zO3DgAADvv/8+u3fvBqC+vt593U5fPPvsswHIz88H\nICcnx7VLe+85+3vIy8tz79GepK6uDvD6h+5809HQW0QkQMxChWQc52nnohUVFZGbmwvAiBEjEn4d\n8WZnHhcXF7Nt2zbAj4Tmz59P//79AT8ytGizqqrK3dGPHz8OeG1m7WZDIYuMBg4c6KLTGTNmADB4\n8OD4/WJdgEWKzc3Nrv02b94MwJtvvklVVRXgD9HHjx8PwLBhw1yEZBFlr1693N+BtbFNMY0dO5ZL\nLrkEgKVLlwJeVJrq7MjZ8vJyNxXUFc7p7ihFlCIiAbr15JNFVTt27GDatGlAakaUffr0AbzIxSK8\nKVOmAHD55Ze7uS9bjLBD5SsqKlwkZJHRqVOnXCRp32d3/aamJkaNGgXAhAkT4vtLdRHWtueccw7n\nnHMOAHfccQcAhYWFbN++HfDmMAG++OILwI8wwWtT8CLzXr282KO4uBjAzSmnp6e716e5uRmARYsW\nuZFQd17oaI/9zrt373ZtaSOgnJwcF4FPnDgRgOzs7CRcZbBu3VH+97//BeCjjz5i7NixSb6a+LE/\n5oyMDObMmQPAvHnzgLbDmEGDBrX5aAsPQY4ePeo+2hu2J7POburUqUydOhWA66+/PqrvtSmOTz/9\nFICPP/4Y8LIUbJHo0KFDgNeZWkeZqux3rq+vd4tlH330EQAFBQVUVlYC/nvWAp3x48e7trEFtV69\nerl/WzBg00bXXXede63iQUNvEZEA3TqifPzxxwFv6HPzzTcn92LiyBYLhg4d6obesZwQtwiqf//+\n7o4tZ8YWeJYsWdLmI8Brr70G+MP3fv36ubZPVcOHDwe8drF/T58+HfCG44cPHwbgyJEjABw7dgyA\nLVu2uOg8tI1sccwWOG3xMTMzM2xEaUN/i1jPVGq/SiIiMdAtI0qbPLc5j1mzZrndFKnIfrdx48bF\nNA2rrKwMgDVr1gAwevRot0gksWcRj82rTZgwIaU3SICfXpaWlubmHC1l7eKLL6ampgbAfbSvVVRU\nuEVG24XW0tLiIk9bLLMFuO9+97un/ex9+/a5hSNFlCIicdYtI8oXX3wRwG23W7x4cdQrvN1ZrPZu\n26rjI4880ub/f/jDH2Ly/D3Vvn373NyZZSWEpv0UFRUB/iquRTupLDRitkg6dB480rbO+vp6Nw9v\n0WZtba1rQ0sHvOiii0773vLycgAaGxvdHGZndamO0nYzBA1H/vKXvwAwefJkAJYvX86wYcPie3Ep\nYvfu3S6t6r333gNg9uzZgDcUko6z9Kp9+/a5m9mX8yKPHj3qci+tvVW0JLLQxUqbbjp27JhbnGkv\nz9dShwYPHhyzNtbQW0QkQJeKKKOZ2F69ejWFhYWAP4FrSdgSmS2AFRcXs2nTJsCPaO65556kXVcq\nsAWIAQMGMH/+/LCPKSkpcQtlPWHIHUs29D5w4IBLDwq3MWLHjh1tHp+Xlxeza1BEKSISoEtElFa1\nJZrk28cee8xFkBdeeCHQM6qwdJYtMmzdutWlWCxYsACACy64IGnX1Z1ZepXtox83btxpj7Ho5sSJ\nE65SUXesnpMM1i9YO5eVlbnqTV8WWhXLok1b7I2FLtFRRtNB/t///R8ABw8e5MYbbwRg4cKFQOoW\nFIgFG3JbQYKNGze6r33ve99LyjWlgtraWnbt2gX4U0a28yTUBx98AHg5quogO8byKG2HzvDhw5k7\nd27Yxx46dMjVRLCi07GkobeISICkR5QtLS1RRZTPP/884BWqXbZsGaAhTDQ+++wzADZs2AB46UHL\nly8HiLjwIMH27t3r9iWHq1xlFYJsOJiXl5fy+7pjqb6+3k1pWMS+aNGiiI+vqalxO3/i0c565URE\nAiQtooz2MLIHH3wQ8OcpbrrpJndMgbSvqKjIJZW/+eabgJea8stf/jKZl9Wt7dmzB/AiSttpcv75\n50d83JgxYwCNfqLV2NgIeItfthBmNSrDRYq257tXr15xPbZEEaWISICkRZRBkaStKNrcpG37Wrp0\naVIOMutObJvXzp07XQ1Eq/t33333uShHomfHN9i8ZENDQ9j6h3v37m3zf2vrVK8SFCtWKchS2ICw\nFa1sDtiizlGjRsW1jZO+mBPJAw88APjhtm1+t83wEpntXHrzzTddWpDlnP7whz9M2nV1R7YYY4ti\ndrTB1KlTmTVr1mmPtz9we5/qph4dS2OzoyHq6upcgd9QtsBz8OBBAFfjId6nhWroLSISIGERZUd2\n3/zmN79xk+EWCVlJfd2hI7PhyNatWwGvOpCVsbrvvvuSdVndmg219+/fD/jv33DR5OHDhxk9ejTg\n7xbTkLt9tnfbpossIs/KygpbbNf2c1sxa0vyj3c7K6IUEQmQsIgymkjS7i6rV692EaTVSLR9snI6\nmzezM6Rff/11wLv73nXXXUD4CXFp36lTp1wkaSMiOzwslKW01NTUuIjSUoekfbZN0WpIWj9hRzx8\nmW1PtIg9UYfhdalX809/+hPgFTm1jtIqRWsIE9mBAwcA2LZtGwDr168HIDc3l1WrViXturq748eP\nu3xf+wOeNGkS4HWcdna35fLNmjVLJdQ6oLGxkdLSUsCf2gg628aKYiS66LGG3iIiAbpERGm7bh59\n9FHAu2tbRBnpTA3x1NfXu3Oit2zZAuDu0r/61a8YNWpU0q6tu8vIyHD7h610l53BcuLECZcyZO0/\nbdo0jXw6oL6+3o2GbNHMpi4iscXcRO+bV0QpIhKgS0SUa9euBfyk00WLFsXs9LRUV1tb64ry2t32\n+9//PqCUoM7KysoKu3gD3sKjLdjYQpmi946pqalxO8YsKg89w9vqS1qU3tLS4g4VS3QNWkWUIiIB\nukREaVvuLCVg/vz5mpuMUr9+/Vz0rTqTiTNkyBAuvfTSNp9TSlDHNDY2usjQsgts3re8vNxtH7WI\nfezYsS5NK9GS/spWVFS4MkqWZb9s2bKE5Ud1d71792bo0KGAf/aNFRCR+LIFBZvyUEfZMRkZGa5o\niJ1kaWk/9fX1rl3z8/MBmD59etLK1WnoLSISIK3V4lsREQlLEaWISAB1lCIiAdRRiogEUEcpIhJA\nHaWISAB1lCIiAdRRiogEUEcpIhJAHaWISIAOb05N1qb0zkp0oU8RSR3qPUREAqijFBEJoI5SRCSA\nOkoRkQDqKEVEAqijFBEJoI5SRCRAzA75aG5uBrwjKO0w84MHDwLeuTh2WJid72JHe2ZlZdG/f39A\nuY6RWNuePHnSnSNSWVkJQF5eXtKuKxXYWS179+6loqIC8M++yczMZPz48YB/lou1v5w5O5ba2rQ7\niFlHWVtbC0BRURE7d+4E4O233wZgx44dVFdXA5CdnQ34f+D5+fkMGzYMwHWY4djZvhkZGQwePBjA\nHUw0fvx497lEn/ebCNae5eXl7N+/H4CqqioALr74YqZNm5a0a+vu7H1ZX1/PjBkzABg0aBCAO1ca\ncCcChgYB1snaez8c63Rzc3OZNGkSQNIOyEo2u7nb2d2DBg1y/UF7LFAIx16X5ubmuHa8CuFERAJ0\n+HCxSFsY7WlCI7qtW7cC8OSTT/LJJ58AUFxcDMDhw4cB72xfu+vaEbXNzc3u+Ww4bs87aNAgd7yt\nRVJf+9rXWLlyJeCdtxxOdx7Wf/rppwCsX7/enX+8e/duwIuqLRK3NrJpjf79+7u2sjaur6930yA2\nBLLvnzlzZtx/F+m56uvrAX+EdN5558XsuZuamlyfEToSiJXu23uIiCRIzOYow80Nzp07t83HUDZf\nsW3bNg4cOAD4UV9DQ4OLnCyCLSsrA6CwsJBdu3YB8NlnnwEwYsQIFw1Fiii7s6lTpwLeXbOwsBCA\n0tJSAN544w3XbjZXNmvWLMCbfxs+fDgAOTk5gDfPafPD27dvB/w2mzlzJpMnTwZg2bJl8f2lpMex\nUU08CutUVla6uUwbRcVSzDrKjrLFl+XLl3f4e+0PfO3atYC3MJSKizjGhhIzZ850w2VbSJgxYwY1\nNTWAv9BgHWZmZqbrUI8fPw54w58jR44A8N577wFw6NAhwOuQlyxZAsCiRYva/OxUF7pQZkPDkpIS\n9/4Md7OXjrGOMh527drl3sc33nhjzJ9fQ28RkQAx6+ItcgFcuk+8zJ49G/BTjcrLyxk9enRcf2ZX\n0KdPH7eANX/+fKDtIpVFmRYx1tbWcuLECcBbNANvAcciSUu5+uKLLwBvyHLTTTe5n9UTfPDBBwAU\nFBTw7rvvtvlcXV0djzzyCAAjR44E/NSW9PR0F91bbmVzczM7duxo8/jf//73ANxyyy2n/exTp065\n57P84lRmo5r2Rn825dbRfNW1a9e6vwVFlCIiSRCziDIzMxPw7pJNTU3ek4fMSVhEE8tIZdy4cYAX\nCVmyaXetwB6trKysiF+zNJ8JEyZEfMzJkyfdpLct6thr16dPHyZOnBirS+0Wzj77bMCbMz/33HMB\nP3XtxIkTlJeXA95GCvCj9crKShdRWtJ6VVWVe0/++Mc/BuDqq68+7Wd+9NFHgBfRWxJ6T2Dv3fbS\n0Gx3VGZmpus/LEG/oaHhtEjzX//6FwBPPfUUr7zySsyv2SiiFBEJELOI0iK60G1EodGdzYfZPJpF\nlqFRpz2+rq6uzV0EcHNtlu4CfpSakZGR8pFkrBQWFrrULIssLVqaO3eu277XU9g8d+hWutCIx95X\nNr9m79vS0lI3crLIsqioiMsvvxwIPw9nKW6WlTB06NAe197Q/vxje6k94b7v3//+NwDnn38+F1xw\nQecvLoKEpQfZROuX92OGdnCWuX/q1CmXZW/DwnD7wO1N29zc7DpiCa+kpASAPXv2uIU3+0O3ts7O\nzm53aN8T2fvW0tmMvS8Bl54VtLPJhtz2nD1p2B1ra9asAfw2/c9//hPXn6eht4hIgKQlnIdjw5XQ\n3TUW7bQXMfbu3VtD7wD79u0DvGGftbNFlrbzxxY2pGMGDBgQ+JhNmza5Ibftxe/O9QeS7eGHHwZg\n3rx5AEyfPj2uP0+vlIhIgC4RUdqijEWFTU1NbpEnmrlHRZORWUKzfayvr3dzwdZuFlHGe6NAqrEF\nRlt4DLfYEDo3bJGnbYdM5W238fLkk08CsGHDBsDfLBFvSe8oW1paXO5UaK6lTZZb6bVI3yuRtba2\nUlBQAPgrrg0NDW4RZ8WKFYA/fJHo1dfXu06wvSkLy8kcPny4q5beU3Y9xcMTTzwB4Moq2g6oeNPQ\nW0QkQNIiSlukqa6udnloFlHW1dXpbJIY2Llzp1vEscoq/fr1czt3LMKRjquoqGiT0xuJ7ezp27dv\nj9v1FGtr1qxxOb/vv/9+Qn+2IkoRkQBJiygtiqysrHRpEqGpQO0dwKS5yfZZIeSjR4+6IyMs0T87\nO9tFQrm5ucm5wG7M9nqfPHnSpfmEY8WlbZQ0ceJEbYo4Q7Y77+GHH+aee+4B4lOctz2KKEVEAiQ8\norSo0fbOZmZmulVAq8IyYMAAdwCWdJwdkbFnzx63l/jo0aOAt83OIsnQbXjSPtsXb3PnU6ZMCfs4\ne3/b1jqr0NReRSdp3/333w94c+qrVq1KyjUkrKO0N5Dti7W9262tra6jjKb4robdke3Zswfwcybr\n6upce9nOhfT0dFcKTIJZ+x08eBAgsO2efvppwC/2ogWzM/f3v/8d8M6FAnjmmWeSdi0aeouIBEhY\nRGmT2pYuYYs1AwcOdHft9vZ1d/D48R7F2vTzzz8H/CTntLQ0N1S0YfbChQuj2pssHhsB2S6a9hZk\n/vnPf7oSdlZJKD8/P74XmKIOHjzIs88+C8Cll14KJDc6V0QpIhIgIRFlS0uLuzPb4oJtWywuLnbV\ngqwWYrhtSYooI7NI0pLL7ezvcePGuTQKO7xKCzjRa2hocPO9Y8aMifg4a++0tDTGjh0LeJG7nLkH\nHniATZs2AbBu3bokX02COsqGhgY3uW2LODaEqampoaqqCvArP5eWlrpVRe2LbV9paak7j9qG4KGL\nOrbCrT/cjmtubm43V9JYBsf8+fPdjSlcoWkJZjf7DRs2cO211yb3YkJo6C0iEiAhEWW/fv1OW0Cw\n4XZokV7bj3z48GFXCkwRZfuqqqpcRGMfLXrfv38/M2bMAFTS60xEGxVa2bSWlha9Xztp48aNgDca\n+tGPfpTkq/EpohQRCZCQiDLakveWcD548GBVD4pSXl4ekydPBvyI3BYUWltb3bxluLPWpXOKi4sB\nPx0rOztbc+udZLvK8vPz3TnrXYEiShGRAF0yvAjd560ti+0bMGAACxYsAPwUqr179wJeFoFFkEqv\nio3nn38e8LbTWZbGsmXLAK/qtioEdY6Nii655JIkX0lbCekoQwvxRruooA4yerbLyY4ksJzVSZMm\nufNwpONKSkpcgdhXX30VgFdeeQXw8lJtocx2jMyZMycJV5labBeO3Xy6Cg29RUQCpLVqTCYi0i5F\nlCIiAdRRiogEUEcpIhJAHaWISAB1lCIiAdRRiogEUEcpIhJAHaWISAB1lCIiAbpkUYyeqrvub4+2\njJ5Id6V3uIhIAHWUIiIB1FGKiARQRykiEkAdpYhIAHWUIiIBYpYedOLECcA7RW3nzp2Af3ZLnz59\n3Pnds2bNAmDMmDGAd4pgtOcn91SxSL/prqlHyVZdXQ34x23oFMvYOXXqFF988QXgn1ppH4cMGdKl\nTrKM2ateWFgIwEsvvcTTTz8NwIEDB9zX8/LyAO+sEYCMjAzAO/TKiqy3dzCTnbkzYcIEZs+eDcB5\n550HeGfDjBgxAoj+TJ6eRp1teJWVlYB3RHI4mZmZMftZdnSwdbZZWVkxe+7uKCMjgwEDBgD+OU8W\ncNXU1Libk31sz6lTp6iqqgL8fmTw4MHk5ua2+dyZ0tBbRCRAzCJKuzuOGTOGu+66C4CJEycCXjT5\n6aefAvDxxx8DuOF5cXExJSUlgH83Af8OYB/tLlxeXu6G7Raa9+/fn8bGRkARpXTMkSNHgMgRZSzl\n5OS0+X9TU1OPH8oPHz48Js/T2NhIaWkp4B95m5OT40ainX19FVGKiASI2SmMx48fBzo37xJurtLu\n+J999plpbKo/AAAIHUlEQVT7OTZHaecpp8pe40hzgHV1dQCUlZVRX18P4OZk+/btm7DFsEjX153b\n/9ixYwBs3ryZK6+8MslXI52xbds2AP7xj38A3oj2iiuuAPw1kjMV86F3Z4SbcB05ciTgTdYCVFVV\nuSF9T1FRUQF4N40PPvgAwH38/PPPXdtMnjwZ8G8g5557rhtyDBo0CPDa2P5tw55EDDu7KpvsLyws\n5NJLLwVg+fLlABQVFbkb0rRp0wD41re+lYSrlGgcPHgQgIKCAsCb2rjoooti8tzdNxQQEUmQLj+T\n3NDQAMD+/fsBWLZsWdjH2ZA0mlSC7sYiv7KyMpeuYtH3kSNHXIqLDT1CnXXWWYA/rdG3b1/3uenT\npwN+ytbMmTP52c9+1ub7GxoaXOpGKqazHD16FICTJ0+6RYC//e1vADQ3N7s8P2u/73znOwDMmTPH\nfa8tJObm5rrXYsKECQCMHTsWgMceeyzsz48mNS5VlZaWuvSggQMHnvZ1m3IKnVpqbm4Gwi/abtq0\nCYBPPvkE8No+RjOLiihFRIJ02YjSknOfeuopwJ9/C+fw4cMpPc9mKQ5Tp051KSbnn38+ALfddpuL\nYiz6sei7qKjIpUw0NTUB3sKLRTn2vFOmTAFg9uzZ7vH2XMXFxcyZMwdIzYjS5iBvueUWVq5cCfij\nmNLSUoqKigBcepvNf6WlpblI3DZWHD161EWZX/nKVwC48cYbAS+BOlzUZAtkPSmtzaLCIUOGnPZ7\nh65FWKRuj+/bt69bswjHXhtbAB4+fHjMNgwoohQRCdAlI8rNmzfz0EMPAf5Wx5/85CcRH5/IFJlk\nsKgjIyOD0aNHn/Z1m9+yx9n+5ObmZje/aHO4Q4cObTf6tud49913ARgwYEBKt62ZOHFiu9kUFmVa\n27a0tLgNErW1tYCX6Dxv3jzAn/dtz7Fjx9yqe0/SXvRsf++5ubkdmrctLCx0I6nFixcD3hbn7Ozs\nTlypr0t1lDaEef31190b8vbbb4/4eNvRM3DgwB45GW6+vCBgizWAe6NEm+to+aqWXzh//vyUntaI\nlk1TWHEX8DtD60QnTZrUocXEnthJRqujf8/PPfec6w8sJWjy5Mmu4+0sDb1FRAJ0yYiypKTEhc82\nwR6O3cmHDRsW/4vrASoqKti4cSPgJ/rn5+f3+P3IQWwBxxbJgljkE6t9zj3Zjh07AG/Rd+7cuYC/\nYcBqQsSCIkoRkQBdIlSwO7JVFqqpqeGaa66J+PjDhw8D/ta77rzXON460jbbtm1zc8M2/5aKCfyx\nFjon3B4bMUUbeUowq31bXl7ODTfcAPiphLGan4Qu0FFWV1e7jPr169cDXsgcKW+ytrbWrZrFsqhq\nKoq2kywrKwO8vbIzZ84E/Er0PWHF+0xZzl+0i13RdqgSzHahPffccwAsXLiQr371q4A3XRRrCsVE\nRAIkPaLcuXMn69atA/wdJVb4N5yqqqo2KRpyuo6mVuzZswfwcgPHjRsHhN97K77a2lq3Tzka5eXl\npxXulTNnpdRsx9mFF17odpjFgyJKEZEASYsobVl/3bp1vPrqqwBcdtllACxatOi0x1slkV69emmB\nIUC0EaUt3FhEOWbMGFfgtCcn8LfH9h13JJqEnrWXO57eeustAF577TUAli5dCsC8efPimiaoiFJE\nJEDCI0qbU7AoZuPGjZxzzjkArF69OuL32cqstn1F1tE0qXfeeQfw94GPHTtWmQQBrIp2tCurQcfh\nSsc8/vjjgP9eX7hwIdB+dbFYSFhHacUWPvzwQwCef/5597Vf/OIXEb/Pyn5ZTlQsc6NSRUeHyTZs\nsbxV2wU1atSo2F5YCrH99FaWLdrHawojdv74xz+6cndWxs7KDcZ7oUxDbxGRAAmLKK0Q7JYtWwA/\nmlm4cKELn0PZPm4rYaVoJ7KORC0FBQVs2LAB8I8rsD2xSgk6nUWG9rG9hcTm5mY3crKpovYKzUp0\n/ve//wHwzDPPuAWbFStWACTsoEFFlCIiARISUdbV1bF161YAl1xud+if/vSnYb/HJs2twkqfPn3i\nfZndUkcXcI4cOeL2Gs+YMQPo/JnHqcyK80ZzDEZZWRknT54EvNqU0jkWnf/2t78FvGNJbr75ZsA7\nihkSdzxJQjrKiooKVxDWhtLnnXce4J9BHerYsWNuclarsLE1ZswYN8Q+++yzAS2QRdLY2OgK9kYj\nMzMzZhW1BZ544gkAXn75ZcDrHG3xJtE3dw29RUQCJCSirK6udmG0LetbOB2OciVjz87OAX8RJxVP\nVYylsrIyF21HU0VJi2Gx9cILLwD+AtqKFSvcKCjRpRUVUYqIBEhIRDl69GjuvvtuILpCsEeOHHGL\nOCrK277a2loOHToE+AWQbQGid+/eLk3FFs+ys7NdVC/t69u3L5s3bwb8nTULFixI5iX1KJb2ZvOS\n48aNS9qahXohEZEACYkoo527sYiopaVFW7+i1NjY6JLzd+3aBfiV4gsKClzVGkunuPbaa1XJJkpZ\nWVm8//77ADz88MMA3HrrrQDceeedbuXVKgmVlpbqwLAY2bhxIxUVFYAfxc+bNy9pFfeTXrgXvGK8\n4O/GGT58uDrKKLW2trpyadZ+gwYNArzcU9ubbEc7DB48WB1llNLT093RGLbA+Oc//xnwzm267rrr\nAD/Nym5G0nnp6elcddVVAEybNg3wCpEk672robeISIC0VpvlFxGRsBRRiogEUEcpIhJAHaWISAB1\nlCIiAdRRiogEUEcpIhJAHaWISAB1lCIiAdRRiogEUEcpIhLg/wG24Y7Y18YZ0gAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x250f136bc50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_conv_output(layer_output1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "The above images shows the first convolutional layer output of the test image of 7."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1, 28, 28, 16)"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layer_output2 = output_conv1([[test_images[15]]])[0]\n",
    "layer_output2.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUoAAADuCAYAAABf005JAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXl0lfWZxz8Q9jUQgbDv+y6LUlFRQRRx6YLWztgZx/bo\nLJ52as/MdOxyxunUaeuxHVvbaee01dYzOjoqAhUVEKqAiOwgoCxhJwSSsGa/yfzxzvd5AwTem3Bv\nlnufzz83hJubN+/93d/v++zNqqqqqnAcx3EuSfOGvgDHcZzGjm+UjuM4EfhG6TiOE4FvlI7jOBH4\nRuk4jhOBb5SO4zgR+EbpOI4TgW+UjuM4EfhG6TiOE4FvlI7jOBH4Ruk4jhNBi4a+gOrk5+cDcOTI\nEY4fPw5ASUkJALFYDIDWrVvTrl07AHr16gVAnz59aNWqVX1fbqNF9+zYsWN2H0+ePAlAUVERzZo1\nA7B71rNnTwCysrLo3r07AC1btqzXa27sVFZWAuF9zM3N5dChQwCcOnXKntO+fXsAunbtCkCPHj0A\n6N+/Py1aNKqPW0pQUVEBkPR764rScRwngno74qRy8vLyACgrKwOgb9++tG7dGggUTfVH52LU7Ons\n2bNAoGx0b/fs2QPA9u3bAVi5ciXr1q0DMGXZqlUrRo0aBcD48eMBuOGGGwCYNWtWWque0tJSAE6c\nOAEEFk5ubi4Q3u9PP/0UgMOHD5ui7NKlCwCDBg1izJgxQLCuqz86daewsBCANWvWsHr1agA6dOgA\nhGq+f//+DBgwAIARI0bY9xKFK0rHcZwImqVCP8qqqirzu6U6mzdvBmDVqlUALF++nE8++QSAgoIC\nIPT1tm3b1nxmd999NwBf+9rXGDp0aL1ec1Nhy5YtACxbtgyAffv2sWvXLgCGDx8OhKqzTZs25tu9\n8cYbAZg6dWq9Xm8qIGtIjwcOHAAC/7rW+kcffQQE1mjnzp0BTD2OGzcOgClTpjB69GgAmjdPvP5z\nRek4jhNBUhxSxcXFQKBo6oN0UZMQZAQAvPzyywDs3bvX/DTf+973gFA9Dh48uAGusOki3+6iRYuA\nYB3L33XHHXcAMHHiRAAyMzNt3SVDwaQLTz/9NADvvPMOEGYVDB482Pzq/fr1A2DChAnmA54+fToQ\nKv1kk5SNcv/+/UBopihoUJ3Tp08DsGDBAnOU67GiosLMmoEDBwLBwtSj/i8dU1imTZsGhCk9p0+f\n5m/+5m8A+MY3vnHFry/HuQ65Nm3aXPFrNhW0Js+dOwcEpve//Mu/ADBz5swGu65UJT8/n6VLlwLw\npz/9CYDevXsD8OUvf9nMa631IUOG0K1bNwDKy8vr9Vr9KHQcx4kgKYpS5uFzzz0HBKpQJ7OQg/a/\n//u/mTx5sj0PgkRyBRyUVJ5OyuZySFn/7d/+LQA/+9nPuPnmm+v0WlKPurdt27a1VJd0RH+7AgTb\nt29nxowZDXhFqYVSA1XUkJWVZQpSATFZMlOnTuXqq68GsAKT6tS3NemK0nEcJ4KkKEo5t+XzeeKJ\nJyzAoFPilltuAQKHucq9pB67du1qCaXuKK8ZObO3bt3K//zP/wA1p6eoxEsJ/hCe0OmsHmtCSlJp\nVsuWLWPnzp0AjB07tsGuK1WQBSNFCXDvvfcC2H1WfKNjx441KsmGIikbpaKtMltycnJs8V3I3Llz\nrcJBTvRNmzaZs7Z6rSwEN9AJyczM5IUXXjjve1/84heBIEL7xz/+EQhNmoEDB9KnT5/zvucEyPWz\nfPlyIMjPfeaZZwD4r//6r4ue/+GHHwJYbl+fPn3sgHcupqYI9Z133gmEFWfaO7KysiybQ3vBqVOn\nLALuprfjOE4jIymKUopFQZrmzZvbqXu554vqJ8+GDRsAeOutt4Cgnla1yumoiFR1owqGjz76yJT4\nz3/+cyA8bRctWmSqR26Nq6++2lKMrrrqKiBU66WlpVZ3n44o2LB27VogsITWr18PwOuvvw7AZz/7\nWXu+8oXlYiooKGDQoEFAGHTzIOTlUX+Cw4cPA2Gdfdu2bc1UV+em06dP29dyhcgKkNJMFq4oHcdx\nIkhqrbd2/6KiooT5btavX8/Ro0eBsN5z2LBhadOP8pVXXgFg48aNAJw5c8bqvr/1rW8BMG/ePHu+\namiVsrV9+3ZTkhkZGUBYGCClCWEQLV2qnkpLS5k/fz4A//RP/wQEvrEvfelLAPzoRz+KfI1YLGb3\nUhUm+gxcaDU58MYbb1g3IK1nKcTRo0dbkFef84qKCuvepPWp7k4jRoywogCp+UTiitJxHCeChPso\nq6qqrBRRu351NalON0o0bdmypfkl9L3LMWnSJPt6xYoVQHDyjxw5Ekhtn1BlZaWdlkrInzRpEj/7\n2c8u+TO6H/KdDRo0yHyaiipKjZ89e9a+l2yfT2OjrKzMaot1bx988EHuv//+uF8jIyPDUlouTG05\nc+aMZ2z8P1KFr776KmfOnAGwR/WivfHGGy19TWWLzZs3t/iEfJpKJ9qyZYv5O1WPf+uttybsmhNu\nepeUlJicVmrKsGHDbBPcunUrEH5IS0pK7I/V98aNG8ejjz4KxNfYYcWKFbYwtWH6oqyZ0tJSPvjg\nAyDMrdQmOmTIkAa7robm3LlzLFy4EAgP+Lvvvjutg1uJRm4gNcAoKiqyg1+uiezsbCDcACEcq9Gi\nRYtLNkJ+4403zhslA4GIuP322xNy7W56O47jRJBwRVlUVGSpPK+99hoQNOFUmypl4sv827t3r5k8\nUqIbNmww812Bh5/+9KdA0GqpJpTSIZUkia7TyLmYY8eOAfDUU08Bgen95JNPAslxiDdmiouLzS2k\n4EFd7kFRUREQJklfLi0u3Vi5ciUQutgUuIFwP5CrSIFGCAM2R44cMcWp51X/fKvxsvaR3NxcM8Pn\nzp170evWBleUjuM4ESRcURYXF1vdppyrhYWF1gD1+uuvv+hnlEKhMsecnBzrT6d+dTt27ABg9uzZ\n/PrXv77k79dp0qlTJwBr9JkOHDx4EAj9PLUt83rsscesfO/xxx8H4POf/3wCrzD1kZJUMroKJpo3\nb25D3JwQ+S3jDcJqj1DZs8Y/VFeKChb98pe/tEDysGHDAHjggQfqdJ2NdmaOAjvabJU/OH/+fPuj\n33jjjUv+vNq4TZkyJZmXmXJ8+9vfBrD68Xfeecfut1N79KE9evSouYfUFq96BodTN/bt2wdA69at\nrcGv+P73v2+ukFgsBsB3v/tdAJslFS9uejuO40TQaIc4ywmu4I1y/bp27cpvf/tbIAwMaX5MdRQE\ncmrHd77zHQDeffddAB555BH72qk9Cly0b9+exYsXA/DXf/3XQBiAdOqO8i5VEVWd7t272xRNmeAy\n9V1ROo7jJJh6U5TqsKIqHJ0EUekTSv5VMvTtt99uDnMlrxcUFFyUBlQ99SCVUOBL96Cqqiqh1Ujq\n2DJnzhwgUJYKSKjpcqpyYTeg6snmdU2XUiFEfn6+dXxSIGL16tV85jOfqfP1pgJax3XtL6nCkuoF\nJkpWP3TokPkmlfKltEPtP/HiitJxHCeCelOUStfJyckBwnSAeBNylUielZVlSaQ6RU6ePJk2ieWK\n5is5ul+/fnY6qhTxcv6XyspK89Oo/6JU/sqVK210hDrL33vvvSmvJCFQNuo9qUj1sGHDahXxLyoq\nMjWq7kGKym7bts26PN12220Aaa8mDx8+bOpP4yESUUarbvSVlZUW25CvWL1Xa0u9B3PUbEEBgp49\ne1rxuz7ALVu2tA1UH1httJ06dTpvpnX1/0t18vPzef/99wF48803Aejbt6998FQXr5SIDRs2mJNb\nZuXOnTutTZ3yLuXC6NWrF9dddx0QfojT5cNcUFBgObsff/wxEKSUqOGxPshqxnD69Gk7YPRYUFBg\n5rVcRnJllJWV8bnPfQ7Aqp/SnbNnz1pjaTXiretGuXnzZp599lkAFixYAMDXv/51y9++VEVfvLjp\n7TiOE0HCFWVJSYnVbtfU6UPqMREVH1KSR48evSjZNBXp0KGDqUYpxNWrV5s5oREav/vd74DAhFQ9\nt5ROr169TEkq4fmee+4Bglnh8TSYlYO8rnWzjZGysjJzRVRX4RdaK9u2bQNCFwiEky4zMjIs6CNX\nkNR+ly5dPHH/Alq0aGHuIu0Zuq8lJSVmfcrSPHjw4HnWEoTt2RYsWGBBsm9+85sAfPnLX7Yg0ZWm\nC7qidBzHiSDhJYwFBQX87//+LxAGC1QWlyjUd06nS9u2bdMuwVz3+Pjx49ZzT2pQ9bBr16619BQp\n0bo6syFUTi1aNNo6hTpz6NAhK5OVgpk8eTKzZs2q8fl79+41P6SCii1btkwbf3ldUGqOkr9jsZgp\nQwUnde/PnTtnvnQFcg8fPmyxC1kzGlWSlZVlo281376kpMTeoysdFZPwjTIvL48XX3wRgOeffx4I\ncv3+/d//HQiaWsSDHOSKQEpWt2vXzjoeq0mvL87kIvMmlZshnz592hqv6MOakZHB9OnTG/KyUobq\nzXIkakpKSsyFpI1MWQInTpywWd/V5zap4YuQOBg2bJhtitpYEzlHy01vx3GcCBKuKM+ePWtyetGi\nRQAsXLjQpLIc2joZsrOzzZTTyVFcXGzPl9SWahwwYIApyXQI4NQ3Mq+rp16loqldExfWC5eUlHjj\n3QQiy0Qm+Llz56xhr+69PudZWVmWi6qc6y5dutj6lPleX2vTFaXjOE4ESelHqSCLWrjn5OSY/0en\nhE6VWCxmWfPyXfTq1cu+1v/pBGnVqlXazJpONqkcnHGaBpdKNausrDSfY2Og8VyJ4zhOI6XRdjh3\nHMdpLLiidBzHicA3SsdxnAh8o3Qcx4nAN0rHcZwIfKN0HMeJwDdKx3GcCHyjdBzHicA3SsdxnAh8\no3Qcx4nAN0rHcZwIfKN0HMeJwDdKx3GcCHyjdBzHicA3SsdxnAiaZMdWdYbTuAJv118zsVjMhrRp\nIuapU6dseJYao2r2cZcuXWy8huZTp9Ls7iuhvLwcgCNHjgCwf/9+mwaq+9iuXTub552VlQUEo0uc\n+KmsrARoVE17wRWl4zhOJElRlDWNmUwUp0+ftlOn+pzgdFU+586dszneUo8HDx4EYNeuXezatQvA\nZiQ3b97cZntrSNuECRMA6NevX1qP/tWYEt2rvLw8W2sam6E1l5uba6NVCwsLgWCkhhSlBmK1b98e\nCAbhabCe7r9UZ7qgfUGPNanGxqYkReO8KsdxnEZEUhSlfGBt27ZN+GsXFBTYKZ+uKhKwYfKrVq1i\n27ZtAOYz0/0vLS01P+7o0aMBuP7665k9ezYQ+iGdgD179gCwcuVKAPbt22dWkRRiu3bt7PmtW7cG\nwsF3zZo1M3+vVGO3bt2AQK1rYF66KUmhcbX6/Dal9eeK0nEcJ4KkKMpEKElFGTdv3gyEKqmwsJAu\nXboAmD8oneajyf+o+7Jo0SI2btwIwJ133gnA9OnTARg8eDCDBw8GPPoaD7169QKCkcgQZAjIapFC\n7NevHwBDhw5lyJAhQKg2ozh16hQQruV4fy5VkC9Xj/KNXwr53NesWQPAhg0bzC8sv7xU6YABAxg1\nahQAY8aMAYL1L9V/pTR4elBZWZktTJGfn8/q1auBcHHJKd6uXTubQ61FnE5zvjUz/fjx40CwYIYP\nHw7Aww8/DIQLxakdSo164IEHAOjbty+7d+8G4LOf/SwQbpR1Id3T2HJzc4Fw41u5ciX3338/cLE7\n4sUXX+QHP/gBEG6YHTt2ZODAgQAWGLvpppuAYFPU/2lzTNQmCW56O47jRNJgivLdd98F4Ne//jWP\nP/44EKhLgOLiYnte3759AcjOzgagT58+pi7TEf3tCtJkZ2czYsQIwJVkolBwZu7cuRw7dgyAHj16\nNOQlpQSyDhU0W7VqFXv37gXgjjvuAODmm28GAteGXEm693369GHo0KFAuB/I9G7RokVSU4tcUTqO\n40TQYIpy/fr1QOCvkI9RAYfi4mLzuym9wgmQf1Yn6tmzZz1Qk0SkZhYsWACEalPKx4kfrVkFGFes\nWGEWkr6nvWDy5MlMnjy5Aa6yZhpso7z22muB4INek8moCgiZmIoQXhj4STcU8Zcze/PmzVZJIpdF\n9ayD999/Hwhz2Lp3724RXAUX0v2eXo7t27cD8NxzzwFYpdPcuXOZNm0aEOao6sPu1MygQYMAOHDg\nAADjx49vMlkZbno7juNEkBRFGU8HkClTpgBhysCFyMQRqsM9fvw4vXv3TsRlNkmU8iClXT23TPl5\nTzzxhD1farN67p4c6Krrlnujffv2Sammaip88sknQKjau3fvTp8+fQAsZ3LDhg0AzJ8/n9/97ndA\nuM6HDx/Ogw8+CMB9990HJDZFpakjt5E+27NmzeK6665ryEuKG1eUjuM4ESRFUcYTppfPcdasWXG9\nptIAOnbsaGkG586dA8KKinRA6So7duwAYOvWrXY/ajqd77nnHiBUlhDeL6l5PV511VWW0C7/pVRA\nqlNcXMzhw4eB8/25Ut1KX1FSetu2be15Ko54++23WbFiBQA//elPAXj00UeBQGFWrxNPN6qqqmyd\nyuIcPnz4Jf26ubm5FvxpDLiidBzHiaBZVQMVSiuamJ2dbTXbtUWqqmXLlhelF6QisVjM0qreeecd\nAN566y3mzJkDwD//8z/X6vWU4K+o9+nTp833Kd9cZWVlo+0R2NjYs2cPP/nJTwBYuHAhEHZE/+pX\nv2r+S/nn04mKigrreLVu3Tog6NWgghJ97+WXXwaCghT5d9V16dprr+VLX/oSEFpK9UW9b5QyV5Yv\nXw4Ef3z37t2v6DU3bNhgpqJufKqmvChws2zZMiBoHvLII48k9XfGYjEgvdva1RY1bfjKV74CBO/X\npEmTAPje974HwI033tgwF9cAlJSUmFti7dq1QBDoUmBRLiXdt5KSEltvcg1p7QNcc801AMybNw+A\nxx57LKnX71LBcRwngnpXlL/61a+AMAVjxowZV6woIWxkK6QsU60uXKdqTk4OADfccEO9Kb1kjvhI\ndX784x/z1FNPAWEw7YknnmDu3LlA6t/ToqIi6xokhXj06NHzxmhAWIgyevRoW9da83v37rXA2dKl\nS4GwHeOIESP4/ve/D2DKPZG4onQcx4mg3hSl0k7UNUi13OoGcqVo+NOqVavO+36q+YE07kF+w3jV\nePXhWPpaKVoXJvc7yeH1118HsIDEuHHj+Nd//VcgTJNLVWVZVVVFaWkpEMYP8vLyLFAoy08qsnpT\nY631yspK9u/fDwRpcQCLFy8GgsCurMiHHnoIgFtuuSVh199gUe9k8corrwBhRPy73/1uQ15Oo0R5\nbDJ7VPUUi8VssaVzhU6y+c1vfgPA73//e8t9VQNbtczT7B3nfLRp6rD/4IMPgKCngUSScl2//vWv\nM378+IT8Xje9HcdxIki5sguZ9moO6lyMzB2135cj/cCBA5anKTNp9OjRjapCIhWQadipUyebgaRg\npGrKXVHWjExzParzWLt27Sxn9eOPPwYC89wVpeM4Tj1R792D5BKVYkn0JLqXXnoJiD/I0dQ5efLk\nRQ7w2qoRJev36dPHJjouWrQICOaAq1451VGys2qSs7KyrNY7GcybN4+3334bCDsXqQ9rKvqI4+kq\nVltU1de8eXNmzJgBhA2Cb7vttoT9HleUjuM4EdR796ALeyeOHDkyob9bEdxx48Yl9HUbC1Liiurn\n5uZaes/EiROBaEUpVV99VjrApk2bWLJkCRB2xFHXplTn7NmzbNq0CQjrs4cNG2azp+Mpic3Pzzc1\nqnWoexuLxbj11lsv+hklTGsMbqre76qqKlPLdVXpVVVVF6VPaa/p2rWr9QC9HFLuSk+Ml3oP5ij9\nRDXfiUK1z8rLlAxPJSorK20UgYIue/bssdEOqmpQCsWOHTvMLFdLutzcXDMx1ZJfG8O5c+fIy8sD\nwk3329/+dnL/qEZCRUWFfQjVnPfVV1+1JtE33HADEAYPmjVrZpug7m1JSYmN49D39F5oDMKFyF2S\naMHQ2CgvL7e1pYO6tnPO65JjqvaCEmh1zVN109txHCeCeleUSkVRrXciOHfunAUhVOcZb0PgpkRJ\nSYnVySoF4siRI6YkdW9XrlwJBBMupRZl4pWXl5vq6dixIxC2VBswYIBVSNx1111A6laKXEhmZiY3\n3XQTEKq8X/ziFzz77LNAmCSugWJ9+/a1QIIUUocOHew+y/0h07Bz584XtbUDGDVqFBCO40jVtKDy\n8nLrFKQWi1pjyWL37t2WkC7XhgbB1RZXlI7jOBE06YRzqaulS5eao/gLX/gCEJ78qUTr1q2tmamU\n30033XTRjOnp06cDQWNenaDyB7Vu3dpST+QXSxfVGC9aO9OmTbOgoBLCpdrbtm1r91EKcfjw4ebT\nlK9c5XQ1UVFRYao0VZWkKCoqsjpt+ca3bNliPl9ZgLXt9iV1umnTJgtOygd/7tw5pk6dCsDAgQOB\nsAlwbWlytd5FRUXnBTIgmHeiG64opczKVKKiooJt27YBcOjQISAI4NT1zXcalpqiuKlKYWGh1WKr\nce+SJUts89Q2pPneo0aNsui4GsHk5eVZVoGCMxIAXbt2NVEgATBt2jT73oVNN2qLm96O4zgRNCpF\nKWd38+bNTT4rBUNt2mKxGMePHwfCU6J///6WIpPqyN2g+9O7d+9ap1k4Tn1TWVl5UVrahg0bzELS\naBg9p6ioyD7zCjb27NnT0qxkSutxyJAhpkAVuGnTpk3CRsK4onQcx4mgUShKVTHoBKmqqrJEXV2e\nUjY6depkfoZUHSAWD0pohtQbd+GkJvos67GsrMx8jkJTQGOx2EX+xPbt29vPKr1Q9ePVPwPJ8Pu6\nonQcx4mgUShKx3GcxowrSsdxnAh8o3Qcx4nAN0rHcZwIfKN0HMeJwDdKx3GcCHyjdBzHicA3Ssdx\nnAh8o3Qcx4nAN0rHcZwIfKN0HMeJwDdKx3GcCHyjdBzHiaDWM3PU1qipoWl4juM4tcV3D8dxnAjq\nfQqjFKkmpp04cYKcnBwAPv30UwAOHjxojWkvnAPer18/m3A3ePBgIBgwVNehQamIGiAXFBRYI1Q1\nM23fvr21ynfip6KiAgjWKwTz1I8dOwaEaxnCBtNar927dweCoVlu1dQNzaGvqSOkppIm+/Pv75zj\nOE4ESVGUOn01azsvL4/du3cDgVqEcMBQbm6ujazU/504ccJOiuHDhwPhGNouXbrYcLF27doB589Y\nTnWkXqQQjx07RmlpKQDl5eVAOFrj2LFjdm81lCkzM9Pum4+5PR8p8SNHjgBw5swZG3YlNSjlMmjQ\nIK6++uoGuMr0ozGMfnFF6TiOE0GtR0HEE/WWopGKXL9+PW+++SYQDi6XmhkzZgzjx48Hzh9LqWFB\nUpY60WOxGC1btgwu/v9VVWZmZuRpkyr+oWXLlgHh+7Bp0ybz8X7+858HAiUEwf3RCGCppaysLHr0\n6GH/D+FJ3blzZ7vfGoGbLkodMEtF1s7gwYPJzMxsyEtKafLz88nKymroy4iL1Ng9HMdxkkhSfJQd\nOnQAYNiwYQBkZ2ebQty+fTsAM2bMAOC2225LyO9sqvmdtSUvLw+AJUuWAMHIT/mEpYSuu+46+78F\nCxYAod+tqKiIjh07AmGENjs7G4CJEycyceJEIL2UpOjWrdt5j3Xh6NGjAKxZswYIVBMEY5Z79eoF\nhPe7b9++puDTkZKSElauXAnA9OnTG/hqLk9STO/LmbkKRsh8TkRY/0qvqSmxd+9eAB577DEgCJhp\n4xOf+9zngCAlRS4PmZB9+/Zl7NixQOD2cOpGTk4OTz75JACLFy8G4NChQ5YOpNS1O++8EwgOr9Gj\nRwM0GXMz2ZSWlnL//fcD8NprrzXw1Vye1Ng9HMdxkki9K0qxbds2oPaqJhaLWRCiNqSKohQ//OEP\nAfjOd77DtddeC4QBsr/6q78CYO7cuQ1zcSnE4cOHAXjllVcA+PjjjwE4deoUmzZtAjCFPmHCBCZP\nngzA1KlTAejatStAndZsOqBUtR/96EcA/N3f/V1DXs4lSa3dw3EcJwkkJZijJF2dpjXx4YcfArB6\n9WruvfdeIPSjlZWVmXK9MBXIT+aAKVOmANC7d2+7J9dccw2AqRqndsjKUYCloKDA0tkUgFHa1NCh\nQ+nduzcQBi8zMzNtvTrxIYN20aJFQONVlEnZKLXJFRYWAmHda3UU7XrppZfMkdu3b18g+PCrEkeP\nPXv2BPCF+P8oi+DGG2/k3XffBYJDB8INs2vXrg1azdDU0NpSDXfz5s2t+kZVZspL7dOnD7FYDAiz\nB3xt1h5Fu3ft2nXJ5+je60BqCNz0dhzHiSDhirKqqspOVlWMnDx5koEDB573PCnEAQMGWO7Znj17\ngKDWWykv06ZNA2D27NkAzJw50ypL1FkoncxxdVhSfl5ZWZnVyI8aNQqArVu3ArBz506rdpLTvG/f\nvtZ9yTkfKUN1rtKac5LH9ddfD1w+b7chlaRwRek4jhNBwtODKioqyM3Nta8h8KdJBV4OpWK89NJL\nvPXWW0CYYC3f0LRp0/jiF78IYGkx3bp1i0xcT4X0oFgsxosvvgiEAbOjR4/y6quvAvDVr34VgP79\n+wNBHfjGjRuB0O/WqlUrZs6cCcAdd9wBhKo9namoqDBfmNd31x9K1pcF2VgrdJr+7uE4jpNkGizh\nPF4UHX/hhRcAeO+996wETMroz/7sz0xFXUpZpoKiBFi6dCkQ1m63adPG/JbqEKQenrt37za/r1T+\n0aNHrYx0wIABQBA5h+A+uv/y8iiFqGPHjrbmnLqxcOFC+1qlno2VhAdzEr0hSYrrcc2aNfziF78A\nMJPz5MmTVo2iRhypmqqhw0HpFOvXr7cNUu4NpQf9xV/8hf2cUrW2bt3KG2+8AYTpRM8++ywQtNyf\nM2cOEJrv2kzTkby8PJ5++mkAfvnLXwJhS7qsrCxrciEX0Jw5cxqt6dgYKSkpaTIjXFJDZjmO4ySR\npJjeSsSNV9Up6KN0n3if//jjjwPw+uuvW9u2v/zLvwSwdmEXNv5NFRScWbFihXWqqW1FzoYNGwBM\nNb355pvjDIpkAAAStElEQVTW9k4diGbOnJk2wQ2Z1XL3fPTRR5a6pooc1XD37t3bAowKQu7fv9/e\nAzWjnjRpUj1dfdNjy5Yttlfo83o5KisrL/s5rmv/iHhIrd3DcRwnCSTcR1laWmrNZZVEqh591dFJ\nUj1Z/NSpU0DgK1OJ3oW9FiFUnuqgk5GRYWkG8nmobFKBjVRD/sj77ruvzq+h8rz/+I//AILuQ3/8\n4x+B0BoYMGCA+eXGjRtX59/V2Dl79qylXMkvO2zYMFtPUopalxBaVwqUHThwwMafLF++HDg/rc05\nn379+pnvPB6irELFJ7Zs2QIkdr0mxfROBKqO0FxqNSC4FGqi+t577wFhIEMNN1LN9E4GhYWFPPro\nowC2YT711FN06tQJgHnz5jXYtTUVNHdHVWnKzRw0aFBaB8bqE2WELFmyxISEqq7qiu8ejuM4ESSl\ne1AikIkj813Biw4dOpxn/oiHHnoICOrEIVSkTvx06dLFgg+ffPIJEDjIP/OZzzTkZTUpNG9Hub4K\nMOzcudOqzDRj3RVmclDa1s9//nNz8Sl9sK64onQcx4kgKYpS6TtSdWfOnLGv1QlEJ2+887gV1Dl8\n+LA52HUyQxgwuuWWWxLyNzRWdB+rq+raplddin379lm3odtvvx0I7ne6DCHTvZWfsUuXLrZea5sY\nrXXbr18/IBg8pnnrriQDYrFYUhPO161bx/79+xPyWq4oHcdxIki4oozFYpYepDb6R48eNR+jxkNo\n9rS6mkPYFr68vNyihfI56mQ4dOiQRd5Vuldd8cjHVlpamuC/rOGJxWKWJK4k8OzsbItKx6Moy8rK\nTMXrnkpJFRQUUF5eDoRlec2aNUsLBVRSUmLRUq0vJZnHy+nTpy3dpbi4GAhTh6qPCa4JqU3V4V/J\nbPGmwocffmhpV8noxD9hwgQbEXylJHyjLCsrs1C8HNrbt2+3cQ/KM7v55puBoHGnHK7KoywoKLCN\n9eTJk0C4kAYOHMisWbMA7ENdnQvbudVXOlN9UFBQwAcffACEaVODBg2y+mLlkVVHB41MnIKCAmv0\nq41Az+nZsycjRowAwjzUjRs3Wk14Krs1WrVqFVdDEJnl+/fvt81Q+ZedO3c2U12Hi+7npdi+fTsQ\n5GACl91MU4358+fb57OmGnn9X02pfdozajLd169fD8CPf/xj641wpbjp7TiOE0HCFWX79u0t0KA5\n0yNHjjS1+G//9m8APP/88wAsWLDAAjVKBerfv78lmCu7Xh1xBg4caGrncs2AU0lJio4dO5qrYv78\n+UBQj/zMM88AcPfddwOhW6N169am7qV+OnToYCpdprpM95YtW9oJ/dvf/hYI3Btq8JvKxFuQIJM4\nEabx2rVrra5c7qOowopUokWLFlyu3uVy78nlgkDJqK93Rek4jhNBwksY4z2ZN2/eDAS+R/l1pECV\nonIlXHidqVLCqMDLU089BQSdk5QeJB+l/GNVVVWmQDVcLDMz056vwEN1/6V8w3ovHnjggfPSsJy6\n84c//AGAHTt2AMEalfpR49orLbVrSqxcudJ8jWoe3VhJ+EZZXl5urc0agpqabUDqbJRi586dQBA5\nVDBGb6UyBqqqqqwpgzIEWrVqZd+Tya2g21VXXWXR7iFDhtTHn9FoOHPmjGUB6ODo1KmTHeLxRMAP\nHz5sm6CaSksQFBYWWk6lXCM33HCDTSFsKg1sE0lubq6t3caeq5tau4fjOE4SSLiiLC4utnQJmW+X\nm9mbKBLlEmhqVFRUWFBGSlG5e/v37zdlreBD8+bNzZRORxVzKQoLC60uWzXZ69atM3Wu/Ea5Jtq0\nacPHH38MhDm7LVu2tPdC6lFNlWfMmGGpQnqUok9Xjh8/blZQTa0YGxOpuXs4juMkkKT0o9Tpq8BD\n69atE3p66pJrc+mpqiidxKEkflXonDhxwhLCVW2m9V9SUmIWU/Ug5IWpa6pEy8jISNmBd1eC9ggp\ncT0WFBScd+8aGt89HMdxImi0Hc4TjStKx3Hqiu8ejuM4EfhG6TiOE0GtTW/HcZx0wxWl4zhOBL5R\nOo7jROAbpeM4TgS+UTqO40TgG6XjOE4EvlE6juNE4Bul4zhOBL5ROo7jROAbpeM4TgQJn8Lo1B1v\nOOI4jZN63yjLy8uBsOff1q1b2bhxI4B1mD5w4IANwNIoW83UGD9+PFdffTWADc7Kysq6aEZOOpKs\nDaupbuANibrNq5fiqVOnbJaUxjk7tSMWi5Gfnw9gI5fVXf7s2bO2V6g/aHZ2dsJ+t0sBx3GcCJLS\nFOPUqVMAHDx4EAiUYk5ODgBbtmwBsH9/8skndjroJJgyZQq33347ANOnTwdg4sSJib7MRkc8yq0h\nzNx0nUdUHX1M3HJpWKorSAjHMJeXl1sH+VatWgGJ7Yye+ivccRznCkmKoly/fj0Ay5cvB2DhwoUU\nFhYCoTrRJMDx48cza9YsIJhznM40BkWpa9DJ3aJFi8iTOVUUpaaHalaL0/iIxWJAMMERglnqEMxl\n11am2MXgwYMTZgGkxgp3HMdJIkmJeitiLd9jly5duOeeewC46667gHDecV2Q6lFksU2bNnV+rXRB\nJ/COHTts2qBUfq9evYDAR6z5341pAl59IZ96IhXlmTNngMCnpgmN+nx06NAhYb8nXVCMQz5K+SN7\n9uxpk16VVZBIf3JSTO+jR48C2ID4zMxMJk+enJDXrqystIWm0ZapYvrFY3prvKcOiaysrEs+d//+\n/XzrW98CgqAZQH5+PkVFRUCwuCA0VcaOHWupVyNHjgRg4MCBtG3b9rLXlCr3f/fu3QDs2rULwAKK\nNbF582ZzMX344YdAcI91L5SaMm3aNCAISo4fPx5InfuVSLSu165dCwTjfrUG9+/fDwSbpNwjQilB\nrVu3ZsCAAQD07t0bwA6mRODvmOM4TgSNfmZO9aACpLYpGI+i1OkqJ3ZeXp6dqlLaSrN6++23WbJk\nCRCa2T169GDEiBEATJgwAYBJkyYB0L9/fzp37lzr604VhVRSUgLAgw8+CMCMGTN4+OGHz3vOf/7n\nfwLw+9//3szq0aNHA4FqHDduHBAqSaWsOJfn5ZdfBuCZZ54BYMiQIdx3331AuL5OnjxpZnWPHj2A\nMCjctWvXpKZupcYKdxzHSSINVuutUsa8vDxzyEo9dujQwRJJVfblBEg9KiBz5MgR8+9IDUrVPPbY\nY8yYMeO8n8vOzjant3yPXqIYIN9tcXExAI888gh//ud/DoQBgkceeQQI/LtSkt27dwew++rUHt3z\nVatW2fe0B2jtjh49mszMTKD+A2H1slEePHjQoq7bt28H4NVXXwWCCh390UOGDAECs0WLUNFxBS20\nqaYrMqsHDhwIBItIi+zWW2+96PmqbLocMm3SfcPUfbj55puB4H4oIDl16tTznnv33Xdf9POxWMwO\nsEQGEtKBmTNnAmEu9ciRI828VuAyJyfHzOt+/foBYdAs2fuCm96O4zgRJEVR7ty5EwhzngoLC+10\nUAj/lltuAeDQoUOWtyZllJOTY8EHpWxIbfbs2dNeI5UDOzVR3Vmt1InqNa5XSk1BmXRSmTLx7rzz\nTiAw/WqzxjIyMkxJnjhxAgjvn8xzp2aU0nPdddcBMGrUKAs6ivz8fNsPtLfs2bMHCJTl8OHDgeQE\nF11ROo7jRJDw9KDi4mKWLVsGwMqVKwG49tprrTInHo4cOcIHH3wABMEeCKslOnfubF8rUTo7Ozsl\nurokuktPotRmunQPkhVT3b+YqK5B1YOWCkg4F/Puu+8CQf/ZmlS4Am7Hjh0DwvfsxIkTFpzs06cP\nEPrxE0FqrHDHcZwkknBFWVlZyd69e4EwKVS+n7rw6aefAkHJGAQpGKpN1smfkZFhv6spq5t4/IFK\nKs/NzQWC9ClFWtesWQPA4sWL7Tnq49m/f38ARowYYf5NKfIrvb6mfM+rI39XPH0IDh06ZJ359fxR\no0bF9Xu8S9HFqM5e5aOjRo2Kq0P5vn37gCAZXX5h0adPH/NbXqlF0Ogrc4RM8I0bN5o5JAdw+/bt\nLd9STR2aYhpR1EZZWVlpdfRaFB999BGbNm0Cwo1Sh0pNr9mxY0dLxVCO5dy5c4GwyqG215kqG6VS\n2F577TUgWEuqz/7Vr34FhLXIEOb96pAePny45Vt+85vfBMJcwJo4efKk5V6myj2sK6rIkVti8uTJ\ntlHGe6DIbNeYmY4dO9ohNmzYMKDu+0J6vzuO4zhx0GQUpaioqLBkdY2cGDt2rCVi6wTXKRTV+aYx\nEaUoS0pKzESRS2LHjh3WxUYmuNIqBg8ebEpFzu9NmzZZoEzt6VTzfdddd/GVr3yl1tebKmpIKv3J\nJ58EYMOGDVazrXWkAEPPnj05cOAAECoYmYEQKiM9/uAHP6jxd17YLqwpWkJXggpQpNT1OR4zZoy5\ni2prNr/11ltA0BdBrydlOXToUKD2A95SY4U7juMkkXpTlPo18uvo5LwSNbJo0SIgUJkaZ6vUgKbY\nHDWeNBzVyG/duhUIkvuVqC/Vo96fQ4YMsdNY97+wsNB8mOos9P777wPBeyKf3E9+8pO4rzdVFKWC\nM1LcXbt25ZprrgEun2qiLkKbN282haTRy0qQbtWqFfPnz0/OhTdRTp8+bfdLn1OVJiaibv7tt982\n/71ed+zYsUA4/jpempzpXR0FNnbv3m0SWxHdpkh9VcFUb1sF8Kc//cke5c7QQvr7v//7S75Oqm2U\niUAHmQ4jbb6LFy820/69995rmItLQ/7xH/8RCA+zOXPmAGEAM158hTuO40TQYG3WEoFyCXv06GEm\ntxONlKDass2ePdv+LSWkQM8rr7zCvHnzGuAqmyaqhJL7QylsLVu25LnnngPgG9/4BgBPP/10/V9g\nmqHg2+uvvw6EaUK1xRWl4zhOBElRlKq/VAVCaWmppe3IaVt9KFBtKC8vtzEIat2v6pN0oDb+wCif\np9zTUo8zZsyw1CIpy4yMjITVOzdFFHysawNpDXAbP348X/jCF4Cwv2JVVdVF91TpQk0pCHkl6P7K\nh6i/O1GTVfX62pPq6k93Rek4jhNBwhXluXPn2LBhA4CF5iEcYKXw/OW62uTn51s5mRKr1eNy3bp1\n1iPwH/7hHy75GoowKhqeClxJdFnKJUoVqmyspjpbpSTpdI6ne3pToqYofqJGkYwbN87UpZL/a3ov\n0i3hXMPyVKKsvrPx1HnHg4oB9Hp1VaoJ3yjLysrsj9cHq3nz5rYo1q1bB4SpPUVFRTbzRR/A/fv3\n20anHEE1wBgzZgxf+9rXgJpNblWnNOGsp4SRqLQdmeGrV68GwsMulSgvLzeT+HJVG7FYDKi9SyIj\nI+Oyh5BQG7F02TCVL6lqGr0HGRkZZoZfSXWd3H/XX389UPv8SeGmt+M4TgQJV5Rt2rSxQUwKxQ8Y\nMMBqlHVyHDp0CAhOUCXp6iSfOXPmRfOR1S4pCinKVGy9X1VVZUni9TW86oUXXrDqCaVgXUnbvMZK\neXk5OTk5wOVVR/XREFca3KoezFGAUulE6YLUtVrUqTJMwwSvhOeff95cJ9deey0Qf4esC3FF6TiO\nE0GTLmGsjhzk8mvUtjtIYyAqnaeoqMiS7KVASktLrZGxOqTU1qcjP87evXutI46GNxUXF1uDX6l8\nJVPLMZ4qJYwaXbJw4UIAfvjDHyb09WU5LV26FAhUk/qnJnJsQVNEMQsFvK7kNdSJaOvWrVbSrLWr\n7kG1pUlX5oiCggIzRVPdCa58M21kq1atsmCYPnQKtkycONFqt/VzFRUV57k94PzAmoJnag82ceJE\nez3NWk9UjltjQxu+umw//PDDPPTQQ0C4kekeR7FgwQIg/NDu2rXLGvzedNNNQJCRke4bpKjtBql1\nrfds37595jpRxsz48ePNlL/Syr3UkAKO4zhJpEma3tUrGyBQkalQNRJPm7X8/HwgUJIQmIsyFdXS\nS47wNm3amAKqfs9039SKTmq8W7duF5nXo0ePthZVLVrUbICkguldWVlpeb9vvvkmEIzZUD28GkHL\nxdO2bVtTLnKDxGIxc3vo+QpWzJ492+5pPDN50g25f3T/WrZsad/Tujty5IhZPFKUcmcobQtC87p3\n794Jq3Bq+ivccRwnyTRJRZmqxNOPUupNJ+vhw4fN53hhs9jTp09b7bBSenr06GHBGaWi6LFXr14W\nBFNtfjxqMRUUZXV0/3bv3m3jHeT/kj+4VatWptble+zVqxcDBgwAwoqwVKoMqw9UoZOfn28+dH0u\n2rZta1/rvqrCLzMzMyEz7C9Faq1wx3GcJOCKshFRXx3OE02qKUrHuRBf4Y7jOBH4Ruk4jhOBm96O\n4zgRuKJ0HMeJwDdKx3GcCHyjdBzHicA3SsdxnAh8o3Qcx4nAN0rHcZwIfKN0HMeJwDdKx3GcCHyj\ndBzHicA3SsdxnAj+DzxuN7DL2uNTAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x250f5474320>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_conv_output(layer_output2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above images are the output of the first convolutional layer of the test image of 5."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's see the outputof the second convolutional layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "output_conv2 = K.function(inputs=[layer_input.input],\n",
    "                          outputs=[layer_conv2.output])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1, 14, 14, 36)"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layer_output3 = output_conv2([[test_images[0]]])[0]\n",
    "layer_output3.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAADuCAYAAACEaORrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmQVNXZx/GvIqKgCCKbCAqyuIEKKBDhBZeoqCkTI2os\nlVRU1LjEJcQ1LlEoY9SUS2mZ0ihojAvGxKVGXABFRZR9FRHZZWecYZhhYJh5/7Cec2+Ps/Ryuu/t\nmd/nH253354+092cee5ZnmePqqqqKkREJCN7Rt0AEZGGQJ2piIgH6kxFRDxQZyoi4oE6UxERD9SZ\nioh4oM5URMQDdaYiIh6oMxUR8UCdqYiIB3tF3YDaVFRUuOO99sp9M8Ovv2PHDgCaNm0KQLNmzXLe\nnnxR/b1q0qRJlM1pFKqqqiguLna3v/vuOwDee+89AN566y0A5syZ486xz2nPPX+Mp9q2bese69On\nDwADBgwA4Gc/+xkAffv2dee0b9/e7y+Rhq+//hqAsrIyAI4//vgom6PIVETEB3WmIiIe7OEra9TW\nrVsB2L59OwCdO3eu9VwLzwGmT58OwIEHHgjAiSeeCER7GVFVVUVJSYm7XVhYCEC7du2A4NIofI61\nX+KlsrIy4RK4VatWEbYmv2zevBkI/m/vscce7rG2bdtG/l6uXbsWgBkzZgDB8FunTp3cOXZs/z/n\nzZvnHrOhDutzMqXIVETEg7Qi06KiInd8wAEH1HhO+MfaXwP7S/Dpp5+6xxYtWgTA//3f/wEwcuTI\nVJsjUqvdu3drEiwLqqqqEiLVOJg2bRoA33//vbuvsrISCCax7777bvfYggULAPjggw8AOO200zJ6\nfUWmIiIepByZVlZWujFDH2w8snXr1rWes3PnTvbee29vrymSDItqUv2+l5eX53z53MqVK93xxIkT\nAdiyZQsAN954IwD77rtvTtsUR+PHj3fHzZs3B+D888/38rMVmYqIeJByZLpjxw722WeftF5s/vz5\nANxxxx3uvj//+c/AT2fUbKYOYN26dfTv3z+t14wTWzRtUcuRRx4ZZXMSZrlbtmwZYUtE6ldaWkp5\nebm7XdfVbBQUmYqIeKDOVETEg5Q3vVe/xLd9sckMbl911VUAHH300e6+2hbMLly40B136dIl1WbG\nyi9/+UsAZs2aBQR7pqMWvrRfs2YNALNnzwZg/fr1AHTv3t2dY8tLbPna/vvv7x6bO3dudhubQ/Ye\nRL3XOyo2HNe7d++IW5KoSZMmCUNT1k6bKDzhhBMAaNGihbfXTGVYU5GpiIgHaaVjsr8EkFxE+tFH\nHwEwfPhwIJh0qsnUqVMBaNOmjbvviCOOSKeZWWMZperKZnXeeee5Y4vannrqKQCOOuqoLLYuPYcc\nckjCv6a0tNQdP/744wAsX74cgBEjRuSodf7Zd3jChAkAXHjhhe6xSy65BIAXX3zxJ8+bMmUKEGw8\nuf3224H8z44VXuhu2zPjFpk2a9aMQw891N0OH/u0c+dOd5zKkkxFpiIiHnhLdFLdhg0b3HGPHj2A\nYMvoO++8U+vzXnvtNQB+/etfu/vy6a++jT3269fP3XfqqacC8PLLL0fSJl9Gjx4NBNHrH/7wh5Se\nn+4i+Gx64YUXAFixYoW77957742kLRIPu3fvdsep9D3x+VaLiOQxdaYiIh5krR5I+BJw27ZtQN2X\n9za5ccYZZwD5dWkfdvbZZwOJy7nGjRsXVXMyZvlmw1K5vE93MD/bZs6cCQTL1ez7J5Ju36PIVETE\nA++R6aRJk4DECagxY8bUev66deuAILO+TVblG1sqY7lb77//fveYFZfLR0uWLHHHd955Z8rPj9vv\nbpML9nuNGjUqyuZIjNh3Q5GpiEiEvC2NsnpI1157LRCUiQX4/e9/X+vznnzySSBYNB0uOZtP/vSn\nPwHBWOl1110XZXMil8zGhihYlQhrX3hziDRu1hWmW0FAkamIiAcZR6ZWjXTy5MlAEImceeaZtT7n\n/fffd8c2PmEL2031ZsWt3gzAbbfd5o7POussINiY0FjZuJMtzI/j55Ysq08Wx+2/qbDfw+YzAI45\n5hgAhg0bFkWTGiRFpiIiHqgzFRHxIOPZAbucO/bYYwHo3Llzredu2rQJgM8++8zdl4+lnW3Bdzh7\nVmO/vK8uny/vLb9Cvl/eG/s9/v3vf7v7bOliY7vMtz4IgqFEm/TO9DuryFRExIOsZY0SEWlMFJmK\niHigzlRExAN1piIiHqgzFRHxQJ2piIgH6kxFRDyIV0qfBiS84iy8uB+C7PPJlMmWzJWXl7tjy/Zf\nfYH2qlWrfnJseSdatGjhHjvssMOAINtUs2bNgCALVfg18pV9P+P2e2zdutVtmIHEzwyCxfcdOnRw\n91kWN/t8wp+TfZa+KDIVEfFAnamIiAcp74AqLCxMKGXx5ZdfAsHlzsCBA4Fgr76IKSsrAzS84dv2\n7dtdmj0IhpXCCdobgl27djFhwgR328ohLVy4EAiGJY444gh3zuGHHw4EqUHDj1111VUAdOrUyUv7\nFJmKiHiQcmS6atUqduzY4W7bYPW3334L4B4bMmSIO8dXzy/+zJkzh6VLl7rb5557LuB/0uHVV191\nx5a9qHfv3l5fo7FbtmwZr732mrv91VdfAUE0Zv83Z8+e7c6xz+CFF14AoG/fvrloalYVFhYCcMst\nt7j7rNClTRgOGjTIPXbCCScAMHz4cABatWqV0esrMhUR8SDlyLS8vNyNj0r++u6775gxY4a7bTk8\nLaqxCPWKK65w54SvNpL16KOPuuPzzz8fCJar5EJ5eTmffPKJuz1//nwgWBq1ZcsWAKZPn+7O+fDD\nD4GgtMfYsWPdY7/4xS+y2+AsuOuuuwD4/PPP3X0nnXQSEERsa9eudY+dffbZwE9znYa7inzOV5st\nikxFRDxIedG+otLsWLduHR07dszZ63Xr1o1u3brV+riNr+2///5p/Xwr733jjTe6+3IZkZpNmzYl\njA3bmH7z5s2BH993gPXr17tzWrduDQQrUlKNRsvKymK1YuGBBx7w8nOijkYLCgq45ppr3G37zOzK\n48QTT4ykXUaRqYiIB+pMRUQ8iE3ZksceewwIBr+7d+8eZXMkRZs3bwaC/dGjRo0C4JlnnomsTcko\nLi4GEoczMr2c1SRtNGx52KxZs9x9Dz74YM5eX5GpiIgHsYlMBw8eDARbxIYOHRplc9L23HPPASRs\neysoKKjx3PDmh3322Se7DcsyWxy9evVqAH7zm99E2RyRnFNkKiLiQeT5TEeOHAkES65si1e+ufnm\nmwGYO3cuAM8++2y9z7Htfg2BjT0qIm0cKisr2XPPeMRi27ZtA9JfxudLPN4NEZE8p85URMSDyK8z\nS0pKALjpppuAYGdKPrjsssvcse1ztwm0rl271vq877//HoCDDz44i63LvgULFrjjhpY7U+oWh0t8\nK0fTtGnTiFvyo+jfERGRBiCSpVH33nuvO7YcguE93HFnGePDUfQ555wDwNtvv13v83fv3g1AkyZN\nstC63Jk0aZI7PuWUUyJsieRC3LJGxa1ygyJTEREPMh4z3bhxIxBk37GxlHDUZRnWX3/9dSBYygD5\nFZEa+0s4btw4d194/LQ+cYxIayprXJsVK1YAikbjyrIp2RZfK2m83377pfXzrDxy3JbyxSUiNYpM\nRUQ8yPhPjf11qKsaaVFREQAXXHABkJi5PNO6K1Fq37591E3wxjLt79q1CwiyzIdZRGqRTj6bPHky\nAFOnTgWC1SQQ/eLvTHXo0CHhX2O/KwQrSax6Z13iFpHGlSJTEREP1JmKiHiQcfyezCWRFSg78MAD\ngfzNCFVdspe7W7duBYLfP4569epV4/1WJhgaRo7Zb775BgiWsLVs2RKo+3tsS9kgnpOHyUqnIGKc\n2Wf51FNPAXDaaacBwTLFXFNkKiLiQWzymYqI5DNFpiIiHqgzFRHxQJ2piIgH6kxFRDxQZyoi4oE6\nUxERD7TpVhq10tJSIMi0BMEifcsbYRnd4cdCcuHn2e2DDjrIndOuXbsstvinysvLE/bP5/PGgvqE\nM87FLYeCIlMREQ/UmYqIeKDL/BoUFRUlXDYlkzBZamaJhSGaVG7Lli3jf//7n7s9b948APbee28A\nzjjjDAAGDx7szrHPe/Xq1UBiKrvWrVtnt8FpqKioYNWqVe62DUvss88+QFBiJJwyMt1E0VGy9JDZ\nZkU+IbX3SZGpiIgHsdmbbwPLcRhU3rhxIwsXLnS3LXOSlWj+4YcfEm5DkBHq9NNPB+Dcc891j8Wh\nLG51hYWFLF682N2232XYsGFAMKGSbGRpEzJWZHDmzJlAzeVrLCrMlXBE8+abbwLw7rvvAjBx4kQA\nNmzY4M6xbGB33HEHAFdeeWUumpm20tJStmzZ4m7bd3fKlClAEI23bdvWnWOTZPZv+OrLskv16NED\nCCLcqM2fP59Zs2a52yNHjszK6zz88MPu+Oqrr046Oo3f/3IRkTyUcmS6adMmNm3a5G5brs7wmFMq\nPv30UwCmTZsGwOjRo9P6OT7t3r074S99YWEhEBQNfP/99wGYMGGCO2fp0qVAMMYSXipjf+Evvvhi\nAHbu3AlA//79E87JZVReWFjoPjuAQw45BIBmzZp5+flW4iT8Gn379vXys7Nh0aJF7tiuJCwqDy+b\nsu+rfT8ee+yxhOfE0V//+lcgMeKyAor2edvVFkDPnj0BOO6444AfozOAk08+OfuNrcO4ceMSrhit\nrwhH3JmwsdK7777b3Tdy5Mg6SzKFxfcbICKSR1KOTNetW5fwV+zQQw8FYMeOHQBMnz4dSCzUZX/p\nzEMPPeSOb731ViAo9lVThLtq1Sq6dOmSSjNjzSKd8BhkOJIViVJxcTEQVMiwK6Z+/fq5c1q2bNlg\nC+0999xz7rhz585uHqQ+ikxFRDxQZyoi4kEkS6Puu+8+d1x9csaE9+AuWrSIAQMG5KZxjdTy5csB\nsK9Dt27domyOpGDHjh1uUhOCIoGSmhdffBGATz75xN03atQoTjjhhKSer8hURMSD2Czary78l7ak\npCTWZZKTZQvDw9v6olJSUuIWc0OwsHvZsmVAsHRmzZo17hwrj2zLqGy7ZdzZ5CgEy19sKZt9JgMH\nDnTnhLePVmfRi5UV7tixo9/GpmHXrl2UlZW52zZhZP+147xsKxUbNmxg7ty57rZNlHXt2hUINoXY\nMksIsnrZ51UXW/oVngweN26c24hSn4bxLouIRCynkelFF10EwCuvvJKrlxSP1q5dC0CnTp3Ser59\n1SzxhmSXRWUNJTKtjSV2sX5l/Pjx7rGPPvqo3uf//e9/B+Dmm28GoKCgwD125plnJt2Ohv0ui4jk\niDpTEREPcrKFYdCgQQBcfvnluXg58cgmaiBYxpYuXd7nhpVdaWgTULWxZX2WK8OyfiXrqaeeAuBX\nv/oVEGROS1XDfpdFRHIka5Hp5MmT3fGXX34JJC5ZyCeWq9P+0jemzPuZRqM2CQLxiJBsCdG+++7r\n9efG6fe0/K25zhsblQ8++AAIcuiG8wzXxTYP2RK/Z599Fkg/f2v0324RkQYga5GpLf6GYAwnX9kC\ndos+GmJkalG376gm6iitumyVQY7T79m0aVMgXm3KBsurvHnzZiDINZssq0RgFRVsbiddDfvdFhHJ\nkYwjU9ueZ2NQtg30iiuuyPRHx4ZFabYt0cZkwlsJbabaIjuLDvKFVRGwXLW2NS9dcV0wnkzEbTl5\n7d8bbrih1nPtqitbEW86UmmLVTXNx3zBFllaXtVkkvOEtw1bNQhb5J/p1Vi8vukiInlKnamIiAcZ\n7823UHnlypUADB06NKXnZ2upikRD++8lV2xIqlWrVkk/J5x9ziae/vjHP3ppjyJTEREPYpvPVEQk\nnygyFRHxQJ2piIgH6kxFRDxQZyoi4oE6UxERD9SZioh4oM5URMSDnJQtkUS2tNeSwsCPySksYUMu\nlJWVNYpdZ0VFRcybN8/dtgqrloilZcuWQLCbBmDGjBkAfPbZZ0CQvAagT58+AAwZMgQI0rYdc8wx\nWWl/FCyVXd++fQGSrhvf2CkyFRHxQJ2piIgHGW8n/eabbwDo2bOnlwbVpKKiIqeXwBUVFS7PI8CH\nH34IwMaNGwE45ZRTgMTM3JbX0v5t1qxZva8Trhu0Y8eOnF5OrV+/nsWLF7vbVkenTZs2ANxyyy05\na4tIMoqLi93/QQiGY77++msgSJYUHr6yxCbHHXcckN3hGEWmIiIeeEt08uCDDwLQvn17AC644AL3\nWKY1k7Zt28b++++f0c/IxFdffQXA888/D8Dbb78NwJo1a9w5lqn8oIMOAmDw4MHuMUv1Ze9NHKxf\nv54OHTq424WFhQC88sorQFCLPBx9H3300UB2r0Ike7744gt3bHWTzjnnHAAefvhh99jo0aMBuP76\n6wF4/PHHc9XEem3bts0dL1y4EICpU6cCMGnSJAC+/fZbd47ViSoqKgKCCBXg6quvBoKqIFYVwqpO\nwI991wEHHJBU2xSZioh4kLUUfC+99JI77t69e8K///znP91jt956KxDUc0k1ubSkZ+3atXTq1Kne\n8x555BF3bOPA5513HpAYAdx7770AXHPNNQCMGDHCV1Mjd//997tjq6lu0Zs0PFu3bnXH++23X9K1\noRSZioh4oOTQkrSCggIANmzYAATVWiGISOM4ziaSC4pMRUQ8UGcqIuJBxpf5dqlnA/MitVm6dCkA\nPXr0iLglIslJJYeFIlMREQ/SikzDT8lFffSdO3cmvTzBh8rKSreAV/yx7bN6byVO5s+fD0Dv3r1/\n8timTZto27ZtUj9H32oREQ/Syh6Si2i0rKwsp68XFkXklOv8opWVlTW+r9l8rxWRSpw8/fTTAJx2\n2mkJ94cTEKUyF6Rvt4iIB+pMRUQ8iG3ZkvDumtatW0fYkuzZvn27O840s5aI1M9yn0LQr1Rfqrdl\nyxZ3nOzkEygyFRHxInaRqeUr3G+//SJuSfY1adIk6iY4paWlQFApoKZCc61atcroNWxJXa4nFKVh\nqqioAEipCodVzQC48sora/x56V4lKjIVEfEgdpGpLUto2rRpxC3JHlv2FadSy9YmqxRgFixY4I7D\n1QMkv1100UUA/OMf/wCCK5F8YldRdUWm9v21Khm9evVyj1Wv02ZXYdX/DyRLkamIiAc5iUz/8pe/\nAEFkY9U9w0pKSoCGPau9c+dOID6L18Pbgq1GTvW/yplGo+EF0CbuY6bt2rUDSKiE2RAcddRR7tjG\nC/MxIjXJVAC2rPm2MH/48OE/Occi3EzrzMXjf7WISJ5TZyoi4kFOypZYGG3lhMOXknZ5b8sSMl1+\nE2e2SD8OQxmZ7M23ySpbTgXQpk2bWl+nurgMc9TG3gMrmvfQQw9F2RxvFi1a5I7Dl/wNjQ2nAcye\nPRuAAQMG1Hq+DXElW9K5NvH+VouI5AkV1BMR8UCRqYiIB+pMRUQ8UGcqIuKBOlMREQ/UmYqIeKDO\nVETEg5T35peVlfHtt9+621988QUAy5cvB4JMSOGMSLb/18o1hxd49+zZE4DDDjsMSG6/bVT++9//\nAjBmzBgAZsyY4R6z383aH87HeuCBBwJw8MEHAzBo0CAAzjvvPHfOsccem61mp2TDhg0ALFmyBIBd\nu3a5xywzua2mCxc9rJ4Jyz5X2+ceVwUFBUDiZ9m8eXMgyEZkm00AVq9eDQTvy5FHHgnAz3/+c3dO\n//79s9ji3LNsSuF9/HHfeBEFvSMiIh6oMxUR8SDjHVC2p/69994DgmSz77zzjjvHXsJSXIWTufbu\n3RuAK664AoAhQ4YA0KFDB3dOKrWrc8l+dwj2AM+bNw8IUn8BvPXWWwAsXLgQgHPPPReASy+91J1T\nU1rCbCoqKsp4L3JjtGLFCgCmTZsGwPHHHw/A4Ycf7s5paInN161bB0DHjh0jbkm8KTIVEfEg8r35\na9asAeCQQw6JshkJZs+enVD+tSEW9yspKWmQv5cEk2NWsFGTRbmhd1lExIPII9M4WrZsGd988427\nXVOpg3xXUVGRUolcaTws766VRbYx/qi9//773Hbbbe62fX8feeQRIJhviYoiUxERDxSZiojbfAPB\nppKbbroJgEcffTSSNlVXVVWVUAnCVtOsXLkSSFxRkYxkSq4vXrzYbcyojyJTEREP1JmKiHiQ8QzE\n4sWLAdi8eTMAb7/9NgBz5sxx59hefluqMWLECPfY2LFjM21Cg/DDDz80yGKCq1atcsddunSJsCXZ\nYcXbbBE/QOfOnenWrVtUTUpLOP/AGWecAcTn8t5UL/ZoE1CpXN4/8cQT7viGG24Agk1FNVm/fr0u\n80VEcimtCajwXzHbalZcXAwEg8KWRQmCrDOnnnoqEESoDVF4G6mxrFHVhaO24uJijjnmmKy1y6dl\ny5YBdUcENnnRr18/d98ll1yS3YZ58sYbbwDBkpvPP/+81nPvv/9+IDE71tChQzniiCOy2EL/fve7\n37lj29qcL59XKqysM9Re2tm2hMOPW9ktA1p9FJmKiHiQ1pip5bUMH1dWVgKNY+va7t27gZojbFvs\nP3DgwHp/Tvgv4LBhw/w0zhPLTztu3Dh33/PPPw8ES1FqcvvttwPB72a34+6jjz5yx48//jgABx10\nUK3nT5w4EQiim3BiHstbmw/uu+8+IPFKsiFGpNdffz2Q+H/u448/rvHccAKjzp07J/0aDb/nExHJ\nAXWmIiIeZLw0qjFd3te1YyKVy/uvv/4aSLw0jEsGp++++w6ACRMmAMFlIMCdd95Z6/NmzZoFwJtv\nvgnAddddB8S/bMnUqVMBeO6559x9n3zyCVD3khmbhLPcpeGlUOHyHnE1c+ZMILjsDX/ODYmVGpoy\nZQoA8+fPr/XctWvXAtC2bVt3X127o6pr+D2giEgOZByZNoaI1NSV8b+uyYrqbEmULRWLg5KSEiCY\neLKqACeffLI754EHHqj1+Q8//DAQDNiPHDkyK+30xQrj2SRE+/bt3WO1TUwAPPnkk0Cw3M+uRHr1\n6pWVdmbL5MmTATjrrLMA8mZZXqq+/PJLAF588cV6z7VikukWt2w8PaGISBYpoWUSFixYAATLYGpa\nLlHbwvywSZMmAcHSmThtXrDI1DZkWET629/+Nqnn26YNW6xv9b7iysatbXzsrrvuSup5FsFZhQjb\nuBBeWhRXNk4KQZnxfNv2miobu69ruZptPLI5jHT/XyoyFRHxQJFpEjZt2gQEM9wWoVoUliyb5Y3j\n+JRtRDjssMMA6N+/f63nWoR99913u/v69OkDBGNwcVdaWgrAkiVL6j336aefdsctWrQAgjFSe7/y\nQbi6aD6sOKiPJVcaP348ACeddBIAAwYMcOcks4HCamZ16tQpo/YoMhUR8UCdqYiIBypbkoKCggIg\nuNwPL/Sui+37Pv7444HkJqvizBYyhzcdvPvuuwAcddRRkbQpVTNmzADqHs645557gGBhP8DVV18N\n/JgZChKXVMVdeXm5O37ppZcAuPzyy6NqTsYsZ7LlOU11SZNtNOnevTuQ+dCHIlMREQ8UmYqIeKDI\nVETEA3WmIiIeqDMVEfFAnamIiAfqTEVEPFBnKiLiQVp787/66it3bIu0bc+ySGNg+7ktp0FduW6z\nrby8nI0bN7rbqRSBk7pVVlYmnbNZkamIiAfqTEVEPEh5B9Ty5cvp2rVrttojkhes9Mwbb7wBpJ6O\nUfJDRUUFe+2V3GioIlMREQ9iszf/b3/7GwCjR4+OuCU/ZUloUymaF3cLFizgrbfecrdXrFgBBBMp\nF1xwAQCDBw/OedsknmzSDWDbtm15n/0sGVVVVS4rVX0UmYqIeBCbyDRO5s2bxy233OJu27KT6dOn\nA9Eug/GltLSU5s2b/+R+GwP817/+BcAHH3zgHrOie1YewsoeA/Tu3RuouxiZlQqp6XVF6lPbdzYu\nFJmKiHigyLQGs2fPdkX0AE4//fQIW5MfLGqfO3cuAKNGjQKCksgQZHq38si5sm3bNndsY8MWSaeq\nWbNmQGLW+jhYu3atO7axzerF/ioqKtxxsjPUkjxFpiIiHqgzFRHxQJf5Ig3ckiVLABKW+PTs2TOq\n5jRYikxFRDyIXWT66quvAnDhhRdG1obt27e7hfoARUVFACxbtgwIthIOGjTInWNlY6+55hoAJk6c\n6B6rPoH1+uuvAzBixAjfTU9JeBF2cXExAE2bNgUyL3tr70ffvn0z+jkiPoWjc99dnyJTEREPvK2P\nsCUxxx13HAArV650j3Xp0iXp51t+yCi1aNGixvysffr0qfU5CxcuBIKos67lVP369cuwhX5YFArQ\npk0bLz+zsrISSIx689XFF18MQK9evdx999xzT1TNSVlZWRkA++67b8QtiZ59lv/5z3+y9hqKTEVE\nPFBnKiLigbcJKJuMsR0iU6ZMSen5AwYMAIKdNPkgPEl16aWXAlBQUFDr+RMmTADg/PPPz27DIjR+\n/HgALrvssohbkjmbrIjZHG29bIilsLAQgHbt2kXZnEi99957ADz00EMATJo0KWuvpchURMSDjCeg\nLJK0Qe50e/4xY8Zk2pScs2VQUHem9Z07dwJw6qmnZr1NUbFlY926dYu4JZmzyTjLL5BvbGKxMUek\n5tprrwWC72c2KTIVEfEg4zFTG58JL7NJxfXXXw/AE088kUkzcuqZZ54BoFOnTu6+c845p9bzbRx1\n+PDh2W1YBix6njNnDgAnnnhiSs+fNm0akLiRIV8NGTIEgKlTp0bckux56aWXALjkkksibol/tmEE\n4LHHHgNg3LhxWX9dRaYiIh5kPGaaTkR68sknu+PJkydn2oSs2759OwC33347ADNmzADg888/T+r5\ncY5IjeXD7NGjR63n2OywjY+//PLL7jGL5vLVsGHD3PHAgQOja0gWWPWE8CqScP2vhubdd991x2PH\njs3Z6yoyFRHxQJ2piIgHOa1d8MADDwCw55751YdbPki7vE3m8j5c9qRt27bZaVgGqpew7dq1a73P\nsRIkq1evBqBVq1busdqGB2qa30y2dG4unHLKKQB8/PHH7r5UN5zE1cyZMwF44YUXgMTvbUOYKKzO\nCl9aTgIP6ItkAAAATklEQVRInCTOtvzq1UREYip2+UxFRPKRIlMREQ/UmYqIeKDOVETEA3WmIiIe\nqDMVEfFAnamIiAfqTEVEPFBnKiLigTpTEREP1JmKiHjw/6Jc1Pv0qq3VAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x250f52099e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_conv_output(layer_output3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above images are the output from 36 channels of the second convolutional layer of test image of 7."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1, 14, 14, 36)"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layer_output4 = output_conv2([[test_images[15]]])[0]\n",
    "layer_output4.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAADuCAYAAACEaORrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHndJREFUeJzt3Xu0zXX+x/Fn7rqolEtuSRFGhTRiWj+SKIsWGrNGRcxq\nGtOaSlNT0aK1uhkzYZimocy4DI1JmNRCGqKURUphUSIRoZL75eDs/fuj9f58v9vt7LP3Z+/v3ue8\nHv/4nv39nr0/+5yzP97fz+X9Pisej8cREZG0lIm6ASIiJYE6UxERD9SZioh4oM5URMQDdaYiIh6o\nMxUR8UCdqYiIB+pMRUQ8UGcqIuKBOlMREQ/K+X7Chx9+GICRI0e6x37yk58A0KFDBwD69evnzrVs\n2TLh+/fs2QNAxYoV3WMVK1akTJno+v158+YBcOTIEQC6d+8eWVuk+I4fP+6O58yZA8DatWsBGDhw\nIACVKlVK6zWOHTvmjsuXL5/Wc0nxvPfee0Dwc7/++uuT+j7bSX/WWWd5aYciUxERD9SZioh4cJbv\nrFEbNmwAoG7duu6x8C376dht1/79+wGoVauWO1erVi3Kli3rs5lSitjQEcC3334LQKNGjVJ6rn/8\n4x8ArF69GghuKW+44QZ3TZ06dVJ6bknN999/D8DKlSsBuPnmm1N6nqNHjwKJwzTFGQJQZCoi4oH3\nyDQZK1ascMebN28G4MILLwSgXr16QOL/7hUrVvQ2SCxS0m3btg2Ajz/+GIDWrVsDUL169cjalA0W\nWVaoUCGS11dkKiLiQSSRab7asWMHADVr1oy4Jf7Nnz8fgE6dOkXcEskV48ePB4JlX23btnXnmjVr\nlhfzGHYXbOOg4TY3a9bM62spMhUR8SDtyLRv374ANG7cGIDBgwen9Dw2i3/eeeel05yM+vOf/wzA\nH/7wh4hb4t+WLVsS/g3PTpd2U6ZMccfr168HoEuXLsCpF4jHYrFIN5lkypIlS9xx9erVU14REQWL\nUG0DDgQrOzp27AjAbbfdltZrlLzfuIhIBNSZioh4kPZt/qxZswDo0aOHlwb973//A6BFixbusSpV\nquTEfmfLLbBw4cKIWyJRs/3gtWvXBqBBgwZRNueUxo4dC8Bvf/tbAMaMGePO3X///Wk994EDBzj3\n3HPTeo4ofP755+54wIABACxatAiAUaNGAUG+huJSZCoi4kFWlkaNHj0agAcffLDIa21LmC3eB7jo\noosy07BisoxYI0aMiLglIsn74osvAGjYsOFJ52zTzOzZs91j/fv3B8jLyDMVN954IwDVqlUD4NVX\nX03peRSZioh4oEX7xTBs2DAABg0aFHFLonHgwAEgWL720EMPuXPh/LWSW2bMmAEECUEANm3aBMDO\nnTuBYNwwfN3TTz8NpD6GmGlr1qxJ+Ldz585AsDU9VbFYzB0XZ4mbIlMREQ/UmYqIeJDV2/x169YB\nibeHn376KRAMkufioPc333wDwD333APAU089BSQukXr00UeL/bzhTFj5ONoSnoibOnUqEGQqyiXh\nfKYXXHBBkdfbXvRcWI4nRfvggw8A2LVrFwDdunWLpB2KTEVEPEg7MrWoLZwZ/0QzZ84Egn3fhw4d\ncueqVq0KBAtoT2XUqFEJ0axIrqhcuTKQuK/75ZdfpkqVKlE1qdTaunUrkHgnkm5mqIKCgqQqhYAi\nUxERL1KKTA8fPuyO7X9m38LLMW699Va37EEkF9jW4nfeeQfIrTFvq081btw4IPi8Wv0qSd6RI0eS\nLgOuyFRExIOUIlOb7YRg0W9xKjLu3r3bHdtsfvv27ROusUXFAJdddllxm5gXbLwZzjzmnA2TJk0C\n4O677460HeKf1YSypCw+HDx4kHPOOcfb8yUjFou5vMcQZM1PZgWQRefhFRrW9dlj9nWq9eYUmYqI\neKDOVETEg7SXRtltvoXR9evXB4KyqxBd6dUobdiwwR1bJizLpXjNNdcA0S0uPpNevXoBULduXQAu\nvfRSd86OW7VqBUC5cuXcuYMHDwLBvn0rKxxepmJ/aununc4Fls/U9oXbzwt+LDxnS/7ykQ3D2QYM\nG9YLL/dq1KgRF198cdbbFu5XrO/57rvvgGAYo0aNGllvFygyFRHxQlmjREQ8UGQqIuKBOlMREQ/U\nmYqIeKDOVETEA3WmIiIeqDMVEfGgXNGXJKegoACAoUOHAol765999lng1KVm80km9jiXFPazWbt2\nLQCXXHIJkJhXIdt7ubMpvMIw1b3d6Qjny/BVIeCTTz5JeO4rrrjCnYtq40V4b/7y5cuBYFNMixYt\ngMR2hjedZJoiUxERD9SZioh4kNIOqPXr17vjRo0aeW1QLrMCcoWFhUBqRfRyid2SAzRt2rTI661w\n2WuvvQbA4MGD3bko9mln2r///W8Atm/f7h6zPBM2ZHX55ZcDiSkUzz777Gw10bG8DxAUlmvbtm2R\n3/frX/8agPHjx7vHbA9+nz59ALj33nsBuPrqq/00toRSZCoi4oH25hfDkCFDAHjmmWeA3CpVkYrP\nPvvMHTdu3DjClmRW+Pe0Y8cOIJhUsSxXZcoEccX5558PwN69ewH4+uuv3bnjx48DQfYiS0wcnmjL\nVCkfyW2KTEVEPEg7Mo3FYkDwP/vIkSMBePjhh901+R7BSdGsfLcty/G1PMcHW7YHsG/fPgCqVavm\n9TXCS5PKli2bEOlGxZYRWfRdErz77rvueP78+UCQO/eBBx446fqePXsCcOeddwJw++23Z6xt0f/G\nRURKgIyNmVoWbIgu87WcXjweT2pxuUWcEM0sdT7KdrG5WCyW8HtKpsCcsTHkmjVrFvs1cyH6TpYV\n7hw0aJB7bO7cuQA8+eSTADzxxBNA6ndV+fPTEBHJYepMRUQ88LY3/0Q2MSW5KRaLsXXrVvf1iXuY\np0+fDsBXX33lHrOcBFYIMB8nNuz9WLE4K8ZmNdghuOVt0qQJkLghwZZNnUm2cxCUKVOmWLf2YbNm\nzQJg0qRJ7rGHHnoICCZvTnXbm0+3+BAUsZwzZ457zG75//WvfwHBRNZ9992X0mvk109ERCRHadG+\nFJttXliwYIF7zLaa5qLw9udly5YBQfRo26FtyRQE0ap9NJo3b+7OWQQfRWYoCSJpgBkzZgDBBLdt\n9w6zEvTJbKSYOXMmQEKZ7vbt2yfdNkWmIiIeKDKVrHnxxReB1MekssU+Ehs3bgQS82NKtFasWOGO\nLd+qjXF37dr1pOu//PJLAEaPHg3ALbfc4s7deuutp3yNcGIby8ubDEWmIiIeqDMVEfFAt/mSEZ07\ndwaC/dMAr776KgC9evWKpE0lWWncqbZ7924AFi9eDAQZvcITSJUqVQKCScT333/fnbv55puLfI13\n332X//u//0uqPYpMRUQ8SGnR/po1a9yxZcuZN28eEGRhv+6669w1Y8eOLfZrbNiwwR1rAsC/Q4cO\nub3JEOzntogyVa+88goA33//PQBvv/22O9exY8e0njsXfPvtt0BQbeFUExS7d+/OasG5wsJCN9EC\n8MILLwDw0UcfAVCnTh2AhGvsM2WTgmfKn2ERX7lyGdvjkxL7GXfv3j3h8aVLl7pj21hi0Xoy0WiY\n/b6TochURMSDlMZMw1sM69ev77E5AdvuB9CyZcuMvEZpFovFEmpAzZ49GwjyRVodoPDykbvuust9\nL0DFihWz0tZ0ffPNN+7Ylr3YtlA7N23aNHeNbTncvHkzkBi1/fKXv0z4t27dukBiftRYLObG6nLB\ngQMHAHjjjTfcY/axX7duHZBYTcCO7ffdv3//rLQzF3355Zc0aNAgqWsVmYqIeKDZfElgY+C2pdKi\nGgi2YFrCk3yxZcsWd2zRuGXftzpP4TshG1u0BCfh7aQWrdmMsVUCDc+gqwZUZsTjcXcHBTBu3Dgg\n+F1MmTLlpO+xKNvutJJJVBO2a9cuLrrooqSuVWQqIuKBOlMREQ90mx+hcM7XfMsPWRQbLggvp1Gm\nJckEK5FkQ1LhfLKWNSpcirsoR44cccfFmUgsWZ9gEZGIKDIVEfFAkamIiAfqTEVEPFBnKiLigTpT\nEREP1JmKiHigzlRExIPcSlCYQywfJwR7tNNl+8BtX7jlEIXcy46+Y8cOAPbv33/SuYYNG572+yzP\np73XcNbzqNh7Afjhhx+AIM+l5asMbygoTpYy+12GVxhGkTEqnEPB3qP9Tdnv4OjRo+4a20xh7zv8\n916hQgUgeB+5lGsg/HO2fLyWm9XKdkdFkamIiAfqTEVEPPC+A2ry5MlAYoGvAQMGFPt5ws2KYk93\nuDSLJcO22wgr43DllVe6a6xUwssvvwzAqlWr3Dm7pbz66qsB6NatGwBt27Y96ZpssvRxEKSdswTC\nH374IUBC+Y09e/YAQfLk0uKvf/2rO/7iiy8AqFWrFgBt2rQB4Nprr3XXhIdvssX2oAOsX78eCIY3\nLIVg+fLl3TWWks5yQoRTEFoCbRveOlUC7KgSg9vfJcCIESOAIMG3JTZPlQ1phYe27PecDEWmIiIe\neI9MLep688033WP5uP3fJlAADh48CBTvf6l8EJ6YsSxPFoWI5KJnnnnGHQ8ZMgSA3/zmN0BqhTt9\nUmQqIuKBskaJeBD1GL9ET5GpiIgHWrSfAlupYKWAAQYNGgTA66+/DkD79u3dueHDhwPw05/+NEst\nTN0LL7wABDO5NgYO8NZbbwGwaNEiIHEMuWzZskCwaL9jx44AtGrVKrMN9uRUqzesnLONmUPw87Cx\nO1uhEXU0apsHILmZ9g8++ACA+fPnA4kbFdq1awcUv/hcaafIVETEA3WmIiIeaALqNGxhPiQWhQsL\nL/9asGABEBT32rp1qztne4fvuOMOILgFjtp///tfd7x8+XIAnnvuOS/PbXvew5s3bGNCsnXIsyl8\nm9+sWTMAdu/eDcCMGTPcOdu3ftVVVwFQv359IHFzQz6x5UXhJUe2oN+Grvr16wdAzZo1s9u4ExQW\nFvLss8+6r20YJpN78gsLC90QVlEUmYqIeJBSZJrJEsVjxowBEidwbJC/pNi4cSMAl19+ecQtya6X\nXnoJgHvvvTfilvhXkpZG2QTcqFGjgCAKtzssSPx8lmQDBw7kL3/5S1LXKjIVEfEgpaVRvqPRsDp1\n6gC5l9/TJ4tIw+OyK1as4Prrr4+qSc6UKVMAmDBhAgCDBw9252y5VNOmTYHExBnJCCeGKWmijEbj\n8XhCPlOLkm3ss7hsHHj06NEJj4cT3Pzwww85kas20y699NKkr1VkKiLigTpTEREPtDTqNGxpD0D1\n6tUz/nrz58+nU6dOGX+dMCtvAfDKK68AsHbtWiC4lZ89e7a75u233wbg6aefBuDnP/+5O2dlL264\n4YYMtrj4YrGYy0EKMG/ePCBY4mS5R8PDSvZebCLmgQcecOcuueSSjLY3FfF4POF3aflMbblWgwYN\ngOA9+3Dw4EHOOeccb8+XLHtvEAxJ/e53vwMy8zn9+OOPky5jo8hURMSDYkem8Xg875d+FCUejydk\nyrcs+tu2bQOC4mW2xAmCSTnbf3/TTTe5czYRYAvDbUF/uFBfnz59/L6JIsRiMRYvXuy+fv/99wG4\n+eabAWjduvVJ3/OnP/0JgJUrVwKJGeUt/0CuT0pYtGlRtm3ICEd2FonceOONRT7f73//ewCeeOIJ\n91i2NyWc+Jm0zPPvvfceELy38IShTYJa9vzatWu7c7m6AaGgoIDp06e7r+1uomfPnhl7zTvuuMPd\ntRVFkamIiAfFjkwLCwtdFANBRh37X/xMZW4/++wzABo3blzshkbJohmLKC0inTlzprvGaidZxp7w\nMqfu3bsDQTYeiwqiqPsUVpIWmmfSli1b3LH9DdgWQxtj7t+/f/YbliSr7xUe/7Ztvh06dABOfUdh\n2zTtMx7FGGnYwYMHee2119zX9jnK5Dh9p06dXGatoigyFRHxoNiRaSwWSxhPtC1mNn5mY1GbNm1y\n19j/jDZ2GF4ILiKSS44ePeqO9+7dm1CV9UwUmYqIeKDOVETEAy3aj4DlyQzfTpx77rmRD/AD7Nq1\nCwiGbcIlMKwctC0V2759uzt34kJx29McXv5leRdKUjlpKwkezrNQpkyZrC4visfjHDlyxH1tv7NM\n5tCQk+mnLSLigSJTEREPFJmKiHigzlRExAN1piIiHqgzFRHxQJ2piIgH6kxFRDxQZyoi4kFK1UlL\ng1gs5o4tzdrhw4cBuO6660663nYHWcJl+xqgVq1aANSrVw+ACy64AIi+oqWlVoNgJ4+9x3CNdCn5\nnn/+eXc8Y8YMINgFZ8mXu3Tp4q4pTtVOn2z3IASpAW331759+4DEnYUTJ04EgnSZ4V2Gt9xyCxCk\nD73mmmuAxB16xamSrMhURMQDdaYiIh5Esp3UspUDfPfddwDUqFEDCG6Jc9FHH30EwLXXXlvkteHq\npiYbVU6TFY/HXS0rCG5/Fi5cCASVAsJ1nkoSq4zQrVu3iFsixVFQUOCO7e832zW3TkeRqYiIB94j\nUxsgztUKh/KjE9O2Va5cGQjuFObOnQtA3759i/W8GzZsAIJJtnCdK6vbnkv1pp566ikAhg4dGnFL\nJN8pMhUR8SBjS6PWrVvnjps0aZKpl4mELXuyJVMQLKtIxp49ewASliZVrVrVRYfZYpFimNW7KSws\nBGDEiBHu3IABA4AzV6nMtyVV+/fvj7oJ4sm8efMA+OSTT9xjLVu2BII+KJOJyRWZioh4kNXZfKtc\n2rBhQ/dY/fr1i/y+eDyeU+Nsp2Kz97ZIeOrUqe7cTTfdBMDIkSMTviccFe3Zsyer5Tzy4Wcqflgp\nmuLOelu9eFvR0bZtW78Ni4DdFUIwru+LIlMREQ/UmYqIeJDV23ybcLnhhhvcYzZYvHr1agCaNWuW\nreZk1MaNG93x/fffD8B9990HQNeuXSNpU1gsFktYAJ3K5Ndjjz3mjidPngzAmDFjAOjVq1eaLcwd\nttwLYPDgwQC8+uqrUTUnaane3p/Ihq5sLztAzZo103rOkkiRqYiIBzlTnbRDhw4A9OnTB4D+/ftH\n2RwpJtueedtttwEwYcIEAPr16xdVk5Jif/7JTsY9/vjjAAwfPjzh+0UUmYqIeJAz+UwtwcacOXMi\nbknp4HtplCUMsUjNtmdu27bNXVO7dm1vr5cqG0e0rbTWJtukAFC2bNnTfv8f//hHIDFnppRchw8f\nTno+QZGpiIgH6kxFRDxIewLq+++/B4JyF9ncxZNJVgIB4OOPPwbg0KFDQGL5hnxVWFiYcGtrO7hW\nrVoFlIz3eCo7duwAgtwRVrJC8pf1PTY8c6qcE9mgyFRExANvS6OWLl0KBBNIAwcOdOdyJRN2ul54\n4QUAnn76aSAxm75tRLjrrrsA6N27tztXpUqVUz5fuOhe+fLl/Ta2CMePH0/4uly5H+ciV65cCcCy\nZcuAYMkaQKNGjbLUuszbvn07ELzP7t27R9mcrJs1axYQTLyFN5L06NEDCLIw5eryL8ud/PLLLwMw\nbtw4AL788kt3jWU4s/f05JNPunO+M5wpMhUR8SBji/YXLVrkjtu3b5+Jl5A0HD9+nDJlgv9Lw8dh\nS5YscceWPah58+YpvWZxF8jnk507d7pjq2cmpYsiUxERD3JmO2lpd/z4cTdumatsBtyisHDGfcsN\neeL4sKI0KS0UmYqIeKDOVETEg7Rv821pxZn2M6fr2LFjWV86lA179+5NOK5Xr16ErRGRdCgyFRHx\nQBNQIiIeKDIVEfFAnamIiAfqTEVEPFBnKiLigTpTEREP1JmKiHiQ25vBJaO2bt3qjuvUqRNhSyQb\nNm3a5I4tF2/r1q2jak6Jo8hURMQDdaYiIh5oB9RpzJw50x337NkzwpZkzvr1691xrVq1gCABdEkS\nLtFi77lp06ZRNUcyxEontWnTJpLXV2QqIuKBJqBOY+3ate74mWeeAYKSz6maNm0aAJUrVwaCInwQ\nTdHBgoICd1wSI1ITTrp92WWXpfVcc+fOBeDZZ58FoG3btgA88sgj7prq1aun9RpyelbWGYLPkYkq\nIjWKTEVEPMjqmGm/fv0AmDRpknvs4osvBuDRRx8FoHPnzkDimFYU5TwsTyvA3XffDcCUKVNSei4r\nKLh48WIArrvuOgAeeOABd82dd96Z9UJz4ci0YsWKWX3tqOzYsQOAmjVrenm+CRMmAIk/vx49epwU\nNUVp+PDhADz22GMRt6RkU2QqIuJBzszmW0nh888/H4CrrroqyuYkeP755wHo3bs3ALVr107r+V58\n8UUgiJIAOnXqlDCGKv7s2bPHHS9YsACA22+/PWOvt3Tp0sjH78Ks2GH451AS/epXvwLgn//8ZySv\nr8hURMQDdaYiIh7kzG1+rtm1a5c7/vvf/w7AoUOHAHjuuee8v97OnTtVYz5Djhw54o5teZstaSoN\n9u/fD8B5550XcUvSU1hYmPC7XLNmDRAMl9nGE5vgzTZFpiIiHniLTA8ePAjAOeecc9prjh49CkCF\nChV8vGTWfPrpp0CwSLhRo0anvdYW+2u7okjposhURMSDtCPThQsXAsGCdlt+ER7/s6VEtlA6vGC6\nUqVKAHTs2DGdZkRi2LBh7njdunUAfPXVV0Di1tNvvvkGgCpVqmSvcSJpsk0kl1xyCRD8HcupKTIV\nEfFAnamIiAdp3+bv3bsXgGXLlgE/LvEJ/wvB3vqf/exnQOJtftWqVQEoX748kH+TUyIlVbt27YAg\nQ5Z26J2ZIlMREQ+0aP80bKE+BPu4S1qeyi1btrjjevXqZeQ1wnlhbZIuk/vic0GvXr2YPn161M3Q\nMr0sU2QqIuKBItPT+PDDD93x3/72NwAmTpyYsddbuXIlLVq0yNjzRyWco7Vr164AvPHGG1E1J2N+\n8YtfuOOvv/7a1SMqCWy5X/369SNtx969exk6dKj7esyYMQA0adIEgBUrVgBw9tlnZ79xKDIVEfEi\n8sjUtmra7H/Lli2BIAN/Lvjuu+8AqFatmtfnPXDggDtetGiRi9xKAquVFK7yOnjwYCD9Oky5xMad\n33rrLfdYPB7n3nvvjapJpcaGDRuA4G/snnvuAYIVQtmmyFRExAN1piIiHni7zd+9ezcAP/zwAwBv\nv/22O/f6668DsHXrViAxr+Kll14KBPklrdxD48aN3TUltQzx119/7Y4rVarkfRhBMs/+7r/44gv3\nWJkyZWjVqlVUTZKIKDIVEfEg8gkoEZGSQJGpiIgH6kxFRDxQZyoi4oE6UxERD9SZioh4oM5URMSD\ncql805AhQ9zxqlWrgGBhfjIWLVrkjidPngzAqFGjADj//PNTaZKIpCGc23bOnDlAULbdspmFcyqc\nqaR7aaXIVETEA3WmIiIeaAdUMRw+fBiAYcOGAdCpUyd3Lh+LjVkeBQhSIH7++efAj7kCAG655ZZi\nPacl1S4sLASCmusAdevWBX7cu56vVq5cCUDZsmUBuOKKK4DoEhKna82aNQBceOGF7rHatWtH1Zy8\nlr9/1SIiOUSR6Wls2rTJHT/yyCNAkIS2b9++AEyaNCn7DfPo2LFj7thKbcvJwmXLa9SoEWFL5Eys\nRE5UXZoiUxERD1JaGlUahHONTpkyBYDKlStH1ZyMUDSaHEWj+cFKiUdFkamIiAcaMy2lYrFYwqz6\nkiVLAKhXrx4Ajz32GADTpk1z12zcuBGABg0aZKuZecNWQQBceeWVEbZEoqLIVETEA3WmIiIe6Db/\nND766CN3/PjjjwPB3uSXXnopkjb5FIvFXDE4CJZJ1axZ87Tf8+abbwIwa9YsAOrUqePOdejQAYB2\n7dp5b6ukZsWKFQn/DhgwIMrmZMx9990HBJ/ZZcuWRdIORaYiIh4oMj2FWCzG1KlT3dfNmzcH4Kqr\nroqqSeKZTRjZFmH7HZcEmzdvBqBixYrAme82SoLVq1cD0X8+FZmKiHigyFRExANFpiIiHqgzFRHx\nQHvzU/DWW28B0Llz54hbIpJ577zzDgBt2rRxjxUWFqp0yQkUmYqIeJAXkelZZ50VWY5CCJbPWNao\nkhCRFhQU8P7777uvt23bBsDRo0cB2L59OwC33367u6Z69epAkFGruMuJDh06BORvVvrSokuXLgDM\nnTsXgP/85z9AUH1BTk2RqYiIBzm7NGrEiBHuePz48VnNVVhQUOBKT0OwfXL69OlAkFkpn8XjcZeZ\nPMy2mPbu3RsIolEIagNZtNqqVavTPv/EiROBIMIFuOeeewCoVq1aGi3PvjFjxrjjBx98EAi2FlvZ\n8nz7mzhy5AgA+/btAxJzgVapUgUISjznCqsrZqwOV65QZCoi4kHORqZbt251x+GEGtlQWFjI8uXL\n3dcWgfnOTB+O2sJVPEuCPXv2AFChQgX3WL6MldoY+dixY4HEqDM8hpyrjh8/DgSRm92BxGIxd00+\nV4jNVfqJioh4oM5URMSDnL3NL4k2bNgAwKpVq4BgEgOgYcOGnHvuuZG0S+RUbLigXLm8WEEZOUWm\nIiIeKDIVEfFAkamIiAfqTEVEPFBnKiLigTpTEREP1JmKiHigzlRExAN1piIiHqgzFRHxQJ2piIgH\n6kxFRDz4fzWcaDUz0SVIAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x250f5287198>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_conv_output(layer_output4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above images are the output from the 36 channels of the second convolutional layer of test image of 5."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 12. Save the entire model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:TensorFlow optimizers do not make it possible to access optimizer attributes or optimizer state after instantiation. As a result, we cannot save the optimizer as part of the model save file.You will have to compile your model again after loading it. Prefer using a Keras optimizer instead (see keras.io/optimizers).\n"
     ]
    }
   ],
   "source": [
    "model.save('mnist_functional_model.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, our model is successfully saved."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}