KNN_1.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "KNN_Demo_Winter_22.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# $KNN$ K-Nearest Neigbor "
      ],
      "metadata": {
        "id": "xrgBYPZfFgOf"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5UqG_1_x7K8r"
      },
      "source": [
        "# Author: Alexander Le\n",
        "# Date: April 27th, 2022\n",
        "\n",
        "# import\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "from sklearn.neighbors import KNeighborsClassifier\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.datasets import load_iris"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "URpaOLlN78qZ"
      },
      "source": [
        "# load iris dataset with load_iris\n",
        "iris_data = load_iris()"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a6Uvq4rv-eCF",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c6436498-710a-4313-c753-d4001bdd7116"
      },
      "source": [
        "# documentation\n",
        "print(iris_data.DESCR)"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            ".. _iris_dataset:\n",
            "\n",
            "Iris plants dataset\n",
            "--------------------\n",
            "\n",
            "**Data Set Characteristics:**\n",
            "\n",
            "    :Number of Instances: 150 (50 in each of three classes)\n",
            "    :Number of Attributes: 4 numeric, predictive attributes and the class\n",
            "    :Attribute Information:\n",
            "        - sepal length in cm\n",
            "        - sepal width in cm\n",
            "        - petal length in cm\n",
            "        - petal width in cm\n",
            "        - class:\n",
            "                - Iris-Setosa\n",
            "                - Iris-Versicolour\n",
            "                - Iris-Virginica\n",
            "                \n",
            "    :Summary Statistics:\n",
            "\n",
            "    ============== ==== ==== ======= ===== ====================\n",
            "                    Min  Max   Mean    SD   Class Correlation\n",
            "    ============== ==== ==== ======= ===== ====================\n",
            "    sepal length:   4.3  7.9   5.84   0.83    0.7826\n",
            "    sepal width:    2.0  4.4   3.05   0.43   -0.4194\n",
            "    petal length:   1.0  6.9   3.76   1.76    0.9490  (high!)\n",
            "    petal width:    0.1  2.5   1.20   0.76    0.9565  (high!)\n",
            "    ============== ==== ==== ======= ===== ====================\n",
            "\n",
            "    :Missing Attribute Values: None\n",
            "    :Class Distribution: 33.3% for each of 3 classes.\n",
            "    :Creator: R.A. Fisher\n",
            "    :Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)\n",
            "    :Date: July, 1988\n",
            "\n",
            "The famous Iris database, first used by Sir R.A. Fisher. The dataset is taken\n",
            "from Fisher's paper. Note that it's the same as in R, but not as in the UCI\n",
            "Machine Learning Repository, which has two wrong data points.\n",
            "\n",
            "This is perhaps the best known database to be found in the\n",
            "pattern recognition literature.  Fisher's paper is a classic in the field and\n",
            "is referenced frequently to this day.  (See Duda & Hart, for example.)  The\n",
            "data set contains 3 classes of 50 instances each, where each class refers to a\n",
            "type of iris plant.  One class is linearly separable from the other 2; the\n",
            "latter are NOT linearly separable from each other.\n",
            "\n",
            ".. topic:: References\n",
            "\n",
            "   - Fisher, R.A. \"The use of multiple measurements in taxonomic problems\"\n",
            "     Annual Eugenics, 7, Part II, 179-188 (1936); also in \"Contributions to\n",
            "     Mathematical Statistics\" (John Wiley, NY, 1950).\n",
            "   - Duda, R.O., & Hart, P.E. (1973) Pattern Classification and Scene Analysis.\n",
            "     (Q327.D83) John Wiley & Sons.  ISBN 0-471-22361-1.  See page 218.\n",
            "   - Dasarathy, B.V. (1980) \"Nosing Around the Neighborhood: A New System\n",
            "     Structure and Classification Rule for Recognition in Partially Exposed\n",
            "     Environments\".  IEEE Transactions on Pattern Analysis and Machine\n",
            "     Intelligence, Vol. PAMI-2, No. 1, 67-71.\n",
            "   - Gates, G.W. (1972) \"The Reduced Nearest Neighbor Rule\".  IEEE Transactions\n",
            "     on Information Theory, May 1972, 431-433.\n",
            "   - See also: 1988 MLC Proceedings, 54-64.  Cheeseman et al\"s AUTOCLASS II\n",
            "     conceptual clustering system finds 3 classes in the data.\n",
            "   - Many, many more ...\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0um__UcQ8N61",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4d2f138f-39c3-4cf7-d10a-587a0a03133c"
      },
      "source": [
        "# feature names\n",
        "print(iris_data.feature_names)"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j1TpcGw7bPju",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7eb53072-c252-43ad-8c53-bd91029a5b6d"
      },
      "source": [
        "# target names\n",
        "print(iris_data.target_names)"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['setosa' 'versicolor' 'virginica']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6Y41iRD_8OoC"
      },
      "source": [
        "# assign the features to X and the targets to y \n",
        "X = iris_data.data\n",
        "y = iris_data.target"
      ],
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5dWy68H08V4s"
      },
      "source": [
        "# split train and test data with train_test_split\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, shuffle=True, random_state=24)"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8kdPQhEK_2s8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 406
        },
        "outputId": "76428db4-1e39-4df4-e0d6-61fc0c222ea5"
      },
      "source": [
        "# plot train data\n",
        "y0 = (y_train == 0)\n",
        "y1 = (y_train == 1)\n",
        "y2 = (y_train == 2)\n",
        "\n",
        "# visualize the first 2 features\n",
        "plt.figure(figsize=[8, 6])\n",
        "plt.xlabel('sepal length (cm)')\n",
        "plt.ylabel('sepal width (cm)')\n",
        "plt.scatter(X_train[y0, 0], X_train[y0, 1], c='b', label='Iris-Setosa')\n",
        "plt.scatter(X_train[y1, 0], X_train[y1, 1], c='g', label='Iris-Versicolour')\n",
        "plt.scatter(X_train[y2, 0], X_train[y2, 1], c='r', label='Iris-Virginica')\n",
        "plt.legend()"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f7c045474d0>"
            ]
          },
          "metadata": {},
          "execution_count": 9
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 576x432 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfEAAAFzCAYAAAAuSjCuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXgc5ZXv8e/RAqaBGAN+COBYYrIxIGPZFoZgFoOHJMQEEiABruDGTIgyUhZI7rCNM8Q4KDATJpjcAScCso00bE5g2C4QCAY7C4lsg2WbkAnEMg6+F2OMg1EgtnzuH9WStXRLXVJXd1Xr93mefrrr7eq3T3VV66i63qpj7o6IiIgkT1mxAxAREZGRURIXERFJKCVxERGRhFISFxERSSglcRERkYRSEhcREUmoimIHENaBBx7o1dXVxQ5DRESkYFasWPGau08c2J64JF5dXU17e3uxwxARESkYM+vM1K6f00VERBJKSVxERCShlMRFREQSKnHHxEVEJDc7duxg48aNvP3228UORXI0btw4Jk2aRGVlZU7zK4mLiJSojRs3su+++1JdXY2ZFTscGYa7s2XLFjZu3Mhhhx2W02v0c7qISIl6++23OeCAA5TAE8LMOOCAA0L9cqIkLiJSwpTAkyXs+lISFxGRyOyzzz5ZnzvuuONy7qerq4v6+nqmTJlCTU0Nxx9/PNu3bx/yNd/85jdz7j+plMRFRKSgdu7cCcAvf/nLnF9z0003cdBBB9HR0cGaNWu4/fbbhx38pSQuMka1tUF1NZSVBfdtbcWOSCR6UW73S5cu5YQTTuCMM87giCOOAHbvpW/atIkTTzyR2tpaampqWLZs2aDXb9q0iUMPPbR3+oMf/CB77rknAK2trcycOZPa2lo+//nP093dzZVXXslf/vIXamtrqa+vB+Db3/42NTU11NTUsGjRIgDeeust5s6dy9SpU6mpqeGuu+4CYOHChRx99NHU1NTQ0NCAu+fvw8gnd0/UbcaMGS4SpdZW91TKHXbfUqmgXSRJ1q1bl/O8UW33e++9t7u7P/nkk55Kpfyll14a9NwNN9zg1157rbu779y50//85z8P6mfVqlU+ceJEP/bYY33+/Pn++9//3t2DZTz99NP9r3/9q7u7NzY2+o9+9KN+/bu7t7e3e01NjW/fvt3ffPNNP+KII3zlypW+ZMkSv/jii3vne+ONN9zdfcuWLb1tF1xwgd9///2j+yBCyLTegHbPkBO1Jy4ywPz50NXVv62rK2gXKVWF2O5nzpyZ8dSpo48+mh/84AcsWLCAjo4O9t1330Hz1NbW8tJLL3HZZZfx+uuvc/TRR/P888/zxBNPsGLFCo4++mhqa2t54okneOmllwa9fvny5Xzyk59k7733Zp999uGss85i2bJlTJkyhZ/97GdcccUVLFu2jPHjxwPw5JNPcswxxzBlyhR+/vOfs3bt2vx9EHmk88RFBtiwIVy7SCkoxHa/9957Z2w/8cQTefrpp3nooYeYN28eX/3qV9l333255pprALjtttuoq6vrTb5nnXUWZWVlPPzww+yxxx585jOf4brrrhtRTB/4wAdYuXIlDz/8MF/72teYM2cOl19+OU1NTbS3t/Oe97yHBQsWxPaCOdoTFxlg8uRw7SKloJjbfWdnJwcddBCf+9znuPjii1m5ciWf/OQnefbZZ3n22Wepq6vjF7/4BVu3bgXgr3/9K+vWraOqqoo5c+awZMkSXn31VQBef/11OjuDgl+VlZXs2LEDgBNOOIH77ruPrq4u3nrrLe69915OOOEEXnnlFVKpFBdccAGXXXYZK1eu7E3YBx54INu3b2fJkiXRfwgjpD1xkQGam6Ghof9Pi6lU0C5Sqoq53S9dupRvfetbVFZWss8++/DjH/940DwvvvgijY2NuDu7du1i7ty5nH322ZgZ1157LR/+8IfZtWsXlZWV3HzzzVRVVdHQ0MBRRx3F9OnTaWtrY968ecycOROAiy++mGnTpvHoo49y2WWXUVZWRmVlJYsXL2a//fbjc5/7HDU1Nbz73e/m6KOPjv5DGCHzuI64y6Kurs5VT1yi1tYWHAvcsCHYE2luhvQAV5HEeP755/nbv/3bnOfXdh8Pmdabma1w97qB82pPXCSD+nr98ZKxR9t98uiYuIiISEIpiYuIiCSUkriIiEhCKYmLiIgklJK4iIhIQimJi4hIZPJRivSiiy7ie9/7Xr+2++67j9NOO21UsbW3t/PlL395RK+trq7mtddeG9X754OSuMSKqoeJlL6wpUjPP/987rzzzn5td955J+eff36o9xuorq6O73znOzn1EZVsseVKSVxio60tuGJUZ2dQQ6mzM5hWIhcpjLaONqoXVVN2TRnVi6pp68jfl280pUjnzJnD7373OzZt2gQE5UMff/xxPvGJT7BixQpOOukkZsyYwUc+8pHeeWbPns2ll15KXV0dN910E/fccw81NTVMnTqVE088sTem008/HYDt27dz0UUXMWXKFI466ih+8pOfAHDHHXcwZcoUampquOKKKzIuW6YSp+vXr6empqZ3nhtuuIEFCxZkjG00dLEXiY2hqijpAhQi0WrraKPhgQa6dgRfws5tnTQ80ABA/ZT8fAFXrlzJmjVrBlUy+8///E8+8pGPMH/+fLq7u+ka8IegvLycs88+m7vvvptLLrmEBx54gNmzZ7PXXnvxpS99if/6r/9i4sSJ3HXXXcyfP5/vf//7QHCN9Z4rfE6ZMoVHH32UQw89lDfeeGNQbN/4xjcYP348HR0dAGzdupVXXnmFK664ghUrVjBhwgQ+/OEPc9999/GJT3yi93UrVqzgBz/4Ac888wzuzjHHHMNJJ53EhAkThvws+sY2GtoTl9hQ9TCR4pn/xPzeBN6ja0cX85/IXy3S0ZQi7fuTes9P6S+88AJr1qzh1FNPpba2lmuvvZaNGzf2vubcc8/tfTxr1izmzZvHrbfeSnd396D+H3/8cb7whS/0Tk+YMIHf/va3zJ49m4kTJ1JRUUF9fT1PP/10v9dlK3E6nL6xjYaSuMSGqoeJFM+GbZn/W87WPhLDlSI99NBDmTdvHj/+8Y+59957qa2tpba2lvb2do477jg2bdrEc889xy9/+Uvmzp2Lu3PkkUf2Vjvr6Ojgsccey/h+3/3ud7n22mt5+eWXmTFjBlu2bMnbcmVSUVHBrl27eqcHljLN9lmEpSQusdHcHFRN6kvVw0QKY/L4zP8tZ2vPp1xKkZoZ5557Lp/5zGc47bTTGDduHB/84AfZvHkzv/rVrwDYsWMHa9euzfgeL774IscccwwLFy5k4sSJvPzyy/2eP/XUU7n55pt7p7du3crMmTN56qmneO211+ju7uaOO+7gpJNO6ve6bCVODzroIF599VW2bNnCO++8w4MPPpjnTy2gJC6xUV8PLS1QVQVmwX1Li46HixRC85xmUpX9/4tOVaZonhP9f9FLly5l6tSpTJs2jbvuuotLLrkk43znn38+zz33XO+o9D322IMlS5ZwxRVXMHXqVGpra7OOeL/ssst6B6gdd9xxTJ06td/zX/va19i6dWvv4Lcnn3ySgw8+mOuvv56TTz6ZqVOnMmPGDM4888x+r5s+fXpvidNjjjmmt8RpZWUlV199NTNnzuTUU0/l8MMPz8MnNZhKkYqIlKjQpUg72pj/xHw2bNvA5PGTaZ7TnLdBbZI7lSIVEZHQ6qfUK2knjH5OFxERSSglcRERkYRSEhcREUkoJXEREZGEijyJm1m5ma0ys0EnyZnZnmZ2l5n9wcyeMbPqqOMREREpFYXYE78EeD7Lc58Ftrr7+4AbgX8pQDwiIlIgUZciHWk50Vze++KLL2bdunWh+y6kSJO4mU0C5gK3ZZnlTOBH6cdLgDlmZlHGJCIixZXPUqTZyokOV+Izl/e+7bbbeiuuxVXUe+KLgMuBXVmePxR4GcDddwLbgAMGzmRmDWbWbmbtmzdvjipWEZGxra0NqquhrCy4z2Md4KhKkfYtJ7pgwQIuvPBCZs2axYUXXsjmzZs59dRTOfLII7n44oupqqritdde6/feS5cuZfbs2Zxzzjkcfvjh1NfX03MRtNmzZ/dWGnvkkUeYPn06U6dOZc6cOQD85je/4UMf+hDTpk3juOOO44UXXsjb55WryC72YmanA6+6+wozmz2avty9BWiB4IpteQhPRET6amuDhobd9YA7O4NpyNu1j/NdivRd73rXoPdYt24dy5cvZ6+99uKLX/wip5xyCldddRWPPPIIt99+e8a4Vq1axdq1aznkkEOYNWsWv/jFLzj++ON7n9+8eTOf+9znePrppznssMN4/fXXATj88MNZtmwZFRUVPP744/zTP/1Tbx3yQolyT3wWcIaZrQfuBE4xs9YB8/wJeA+AmVUA44FoS8uIiMhg8+fvTuA9urqC9jzJdynSTM444wz22msvICgTet555wHw0Y9+NGuN75kzZzJp0iTKysqora1l/fr1/Z7/9a9/zYknntgb+/777w/Atm3b+NSnPkVNTQ1f+cpXshZfiVJkSdzdr3L3Se5eDZwH/NzdLxgw2/3AZ9KPz0nPoz1tEZFC25Cl5Gi29hHIdynSMO8xlD333LP3cXl5+bDH03v88z//MyeffDJr1qzhgQceGFRutBAKfp64mS00szPSk7cDB5jZH4CvAlcWOh4REQEmZyk5mq09j0ZainQ4s2bN4u677wbgscceY+vWrSOK79hjj+Xpp5/mj3/8I0Dvz+nbtm3j0EMPBeCHP/zhiPoerYIkcXdf6u6npx9f7e73px+/7e6fcvf3uftMd3+pEPGIiMgAzc2Q6l+KlFQqaI/YSEuRDufrX/86jz32GDU1Ndxzzz28+93vzvhT/XAmTpxIS0sLZ511FlOnTuXcc88F4PLLL+eqq65i2rRpOe+955tKkYqIlKiwpUhpawuOgW/YEOyBNzfnbVBbMbzzzjuUl5dTUVHBr371KxobG3n22WeLHdawVIpURETCq69PdNIeaMOGDXz6059m165d7LHHHtx6663FDinvlMRFRKQkvf/972fVqlXFDiNSKoAiIiKSUEriIiIlLGnjnsa6sOtLSVzGhKYmqKgAs+C+qanYEYlEb9y4cWzZskWJPCHcnS1btuR0+lwPHROXktfUBIsX757u7t49fcstxYlJpBAmTZrExo0bUc2J5Bg3bhyTJk3KeX6dYiYlr6IiSNwDlZdDkU7tFBEJJdspZvo5XUpepgQ+VLuISFIoiUvJKy8P1y4ikhRK4lLyeqop5touIpIUGtgmJa9n8FpLS/ATenl5kMA1qE1Ekk5JXMaEW25R0haR0qOf00VERBJKSVxERCShlMRFREQSSklcREQkoZTERUREEkpJXCKn4iO7tbVBdTWUlQX3bW3FjkhkDIrqi1iEL7hOMZNIqfjIbm1twfnpXV3BdGfn7gvO1NcXLy6RMSWqL2KRvuAqgCKRUvGR3aqrg+/1QFVVsH59oaMRGaOi+iJG/AVXARQpChUf2W3DhnDtIhKBqL6IRfqCK4lLpFR8ZLfJk8O1i0gEovoiFukLriQukVLxkd2amyGV6t+WSgXtIlIgUX0Ri/QFVxKXSN1yCzQ27t7zLi8PpsfaoDYIxra0tASHyMyC+5YWDWoTKaiovohF+oJrYJuIiEjMaWCbiIhIiVESFxERSSglcRERkYRSEhcREUkoJXEREZGEUhIXERFJKCVxiVwcKnfFIQYRkXxTFTOJVBwqd8UhBhGRKOhiLxKpOFTuikMMIiKjoYu9SFHEoXJXHGIQEYmCkrhEKg6Vu+IQg4hIFJTEJVJxqNwVhxhERKKgJC6RikPlrjjEICISBQ1sExERiTkNbBMRESkxSuIiIiIJpSQuIiKSUEriIiIiCaUkLiIiklBK4hKaiomIiMSDCqBIKComIiISH9oTl1Dmz9+dwHt0dQXtIiJSWEriEoqKiYiIxIeSuISiYiIiIvGhJC6hqJiIiEh8KIlLKComIiISHxqdLqHV1ytpi4jEgfbERUREEkpJXEREJKGUxEVERBJKSVxERCShlMRFREQSKrIkbmbjzOw3Zvacma01s2syzDPPzDab2bPp28VRxSMiIlJqotwTfwc4xd2nArXAR83s2Azz3eXutenbbRHGI3kStopZmPnjUiEtLnGIiAwlsvPE3d2B7enJyvTNo3o/KYywVczCzB+XCmlxiUNEZDgW5NqIOjcrB1YA7wNudvcrBjw/D7gO2Az8HviKu788VJ91dXXe3t4eTcAyrOrqIKkNVFUF69ePbv6wfUclLnGIiPQwsxXuXjeoPcok3ufN9wPuBb7k7mv6tB8AbHf3d8zs88C57n5Khtc3AA0AkydPntGZ6S+sFERZGWTaZMxg167RzR+276jEJQ4RkR7ZknhBRqe7+xvAk8BHB7Rvcfd30pO3ATOyvL7F3evcvW7ixInRBitDClvFLEx7XCqkxSUOEZHhRDk6fWJ6Dxwz2ws4FfjdgHkO7jN5BvB8VPFIfoStYhZm/rhUSItLHCIiw4lyT/xg4EkzWw38FviZuz9oZgvN7Iz0PF9On372HPBlYF6E8UgehK1iFmb+uFRIi0scIiLDKcgx8XzSwDYRERlrinpMXERERPJPSVxERCShlMRFREQSSklcREQkoZTERUREEkpJvMiiLLQRVd9joQBKqWvraKN6UTVl15RRvaiatg590CKJ5O6Jus2YMcNLRWureyrlHlzkM7ilUkF7XPsO22+Y+aP8PGS31tWtnmpOOQvovaWaU966Wh+0SFwB7Z4hJ+o88SKKstBGVH2PhQIopa56UTWd2wZ/0FXjq1h/6frCByQiw9J54jG0YUO49jj0HbbfMO1Rfh6y24ZtmT/QbO0iEl/DJnEzqzOzr5jZt9KXTP20mU0oRHClLspCG1H1PRYKoJS6yeMzf6DZ2kUkvrImcTO7yMxWAlcBewEvAK8CxwOPm9mPzEzf+lGIstBGVH2PhQIopa55TjOpyv4fdKoyRfMcfdAiiZPpQHn6OPkXgL2GeL4WmJPt+ahupTSwzT0YtFVV5W4W3OdzEFdUfYftN8z8UX4eslvr6lavurHKbYF51Y1VGtQmEnNoYJuIiEgyZRvYVpHDCw8DvgRU953f3c/I9hoRERGJ3rBJHLgPuB14ANgVbTgiIiKSq1yS+Nvu/p3IIxEREZFQckniN5nZ14HHgHd6Gt19ZWRRiYiIyLBySeJTgAuBU9j9c7qnp0VERKRIckninwL+xt3/GnUwIiIikrtcLru6Btgv6kCkdKkymWSz/PomNu5fwS4zNu5fwfLrm4od0vC0QUuM5LInvh/wOzP7Lf2PiesUMxlWWxs0NEBXVzDd2RlMA9TXFy8uKb7l1zcx7erF7L0jmJ60tZsJVy9mOXD8lbcUNbastEFLzAx7sRczOylTu7s/FUlEw9DFXpJFlckkm437VzBpa/fg9gnlTHp9ZxEiyoE2aCmSEV/sBdgAbHL3t9Md7QUclOf4pESpMplkc0iGBD5Ueyxog5aYyeWY+D30v8hLd7pNZFiqTCbZvDKhPFR7LGiDlpjJJYlX9B2Znn68R3QhSSlRZTLJZv3lDbxV2b/trcqgPba0QUvM5JLEN5tZ7yA2MzsTeC26kKSU1NdDS0twyNAsuG9p0RggCQavrVrYyMYJ5ewiOBa+amFjfAe1gTZoiZ1cBra9F2gDDkk3bQQudPcXI44tIw1sExGRsWbEA9vSyfpYM9snPb09gvhEREQkpKw/p5vZBWbW+7y7b++bwM3svWZ2fNQBioiISGZD7YkfAKwysxXACmAzMA54H3ASwXHxKyOPUERERDLKmsTd/SYz+3eCQiezgKOAvwDPExwT14mRIiIiRTTkMXF37wZ+lr6JiIhIjORyipkQXc2DsP3GofZCHGKQwdo62qheVE3ZNWVUL6qmraPEVkyYDS8uG2mEcUS2vuPy2Ulu3D1RtxkzZnihtba6p1LusPuWSgXthew3qjiijFkKo3V1q6eaU84Cem+p5pS3ri6RFRNmw4vLRhphHJGt77h8djII0O4ZcuKw54nHTTHOE4+q5kHYfuNQeyEOMchg1Yuq6dw2eMVUja9i/aXrCx9QvoXZ8OKykUYYR2TrOy6fnQyS7TzxXC72sidwNlBNn2Po7r4wzzHmpBhJvKws+Jd0IDPYtWtwe1T9RhVHGHGIQQYru6YMZ/CKMYxdXy+BFRNmw4vLRhphHJGt77h8djJItiSeyzHx/wLOBHYCb/W5jRlR1TwI228cai/EIQYZbPL4zCsgW3vihNnw4rKRRhhHZOs7Lp+d5CyXJD7J3c91939193/ruUUeWYxEVfMgbL9xqL0QhxhksOY5zaQq+6+YVGWK5jklsmLCbHhx2UgjjCOy9R2Xz05yl+lAed8b0AJMGW6+Qt2KMbDNPRjXUVXlbhbc52ucR9h+o4ojjDjEIIO1rm71qhur3BaYV91YVTqD2nqE2fDispFGGEdk6zsun530Q9iBbWbWATjBcfD3Ay8B7wAW5H4/qhD/ZAykAigiIjLWjKQAyukRxiMiIiKjNNRlVzsBzOw/3P3Cvs+Z2X8AF2Z8oYiIiBRELgPbjuw7YWblwIxowhEREZFcDVWK9CozexM4ysz+nL69CbxKcNqZiIiIFFHWJO7u17n7vsC33P1d6du+7n6Au19VwBhFREQkg6zHxM1sevrhPX0e93L3lZFFJSIiIsMa6pj4v6VvNwPPEJwvfmv68c3RhzY2xKVgUFziEIm7poeaqFhYgV1jVCysoOmhprz1vfz6JjbuX8EuMzbuX8Hy6/PXt5SmoUannwxgZj8Fprt7R3q6BlhQkOhKXFsbNDRAV1cw3dkZTAPU14+9OETirumhJha3L+6d7vbu3ulb5t4yqr6XX9/EtKsXs/eOYHrS1m4mXL2Y5cDxV46ubylduRRAWevuA0eoD2orlFK62EtcCgbFJQ6RuKtYWEG3dw9qL7dydl69c1R9b9y/gklbB/e9cUI5k14fXd+SfCO52EuP1WZ2G9Canq4HVuczuLFqw4Zw7aUeh0jcZUrgQ7WHcUiGBD5Uuwjkdp74RcBa4JL0bV26TUYpLgWD4hKHSNyVW3mo9jBemZC5j2ztIpBDEnf3t939Rnf/ZPp2o7u/XYjgSl1cCgbFJQ6RuGuY0RCqPYz1lzfwVmX/trcqg3aRbIa62Mvd6fsOM1s98Fa4EEtXfT20tATHns2C+5aWwg8mi0scInF3y9xbaKxr7N3zLrdyGusaRz2oDYLBa6sWNrJxQjm7CI6Fr1rYqEFtMqShqpgd7O6bzKwq0/M911YvtFIa2CYiIpKL0APb3H1T+uHfAU+7+39HFZyIiIiEl8vo9MnA98ysGlgBPA0sc/dnI4xLREREhpHLwLavu/spBNXMlgGXESRzERERKaJh98TN7GvALGAfYBXwjwTJXERERIool5/TzwJ2Ag8BTwG/cvd3Io1KREREhpXLz+nTCQa3/QY4Fegws+XDvc7MxpnZb8zsOTNba2bXZJhnTzO7y8z+YGbPpI+7F0TYgh9JLBDS1AQVFcFpYxUVwbTkX1tHG9WLqim7pozqRdW0dRRn4whVPCPEBp3IohwhN/5Q6zCJfwzCxJzE5RvL3H3IG1ADNAJ3An8AngQW5vA6A/ZJP64kqH527IB5moDvph+fB9w1XL8zZszw0WptdU+l3GH3LZUK2vMxfxw0NvaPt+fW2FjsyEpL6+pWTzWnnAX03lLNKW9dXdiNY9l1jb69sv/K3l6JL7suwwoPsUGH6jcuQm78odZhEv8YhIk5ics3RgDtniEn5lIA5UGCEenLgd+6+46w/yiYWSr9+kZ3f6ZP+6PAAnf/lZlVAP8XmOhDBJWP88TDFvxIYoGQigroznDJ5fJy2KlaCnlTvaiazm2DN46q8VWsv3R9weIIVTwjxAadyKIcITf+UOswiX8MwsScxOUbI7KdJz5sEh/lm5YTjGR/H3Czu18x4Pk1wEfdfWN6+kXgGHd/bcB8DUADwOTJk2d0ZtrIQigrC/7FHBwv7No1+vnjwCz7cxGu8jGn7JoynMEfqGHs+nrhNo5dZhmPje0Cygau8BAbdKh+4yLkxh9qHSbxj0GYmJO4fGNEtiSeSwGUEXP3bnevBSYBM9O1yEfST4u717l73cSJE0cdV9iCH0ksEFKepWZCtnYZmcnjM28E2dqjEqp4RogNOpFFOUJu/KHWYRL/GISJOYnLN8ZFmsR7uPsbBMfSPzrgqT8B7wFI/5w+HtgSdTxhC34ksUBIQ5aaCdnaZWSa5zSTquy/caQqUzTPKezGEap4RogNOpFFOUJu/KHWYRL/GISJOYnLN9ZlOlCejxswEdgv/XgvgnPLTx8wzxfoP7Dt7uH6zcfANvdgnEZVlbtZcD/cuI2w88dBY6N7eXkwNqW8XIPaotK6utWrbqxyW2BedWNVwQe19Vh2XaO/PKHcu8FfnlA+9OCzEBt0qH7jIuTGH2odJvGPQZiYk7h8YwBhB7aZ2QOQ4UDR7uR/xlD/HJjZUcCPgHKCPf673X2hmS1MB3O/mY0D/gOYBrwOnOfuLw3VrwqgiIjIWBO6AApww2je0N1XEyTnge1X93n8NvCp0byPiIjIWDVUFbOnChmIiIiIhJPLtdPfD1wHHAGM62l397+JMC4REREZRi6j038ALCa4fvrJwI+B1iiDEhERkeHlksT3cvcnCC4M0+nuC4C50YYlIiIiw8mlitk7ZlYG/LeZfZHg3O59og1LREREhpPLnvglQAr4MjADuBD4TJRBxZEK+0g2YSpgxaXiWRhhY45qGZP42YlELedrp5vZuwB39zejDWloxThPvK0tuNhTV9futlQKWlqgvr6goUjMtHW00fBAA107dm8cqcoULR9voX5K/YjnjYuwMUe1jEn87ETyacQFUMysjmBw277ppm3A37v7irxHmYNiJHEV9pFswlTAikvFszDCxhzVMibxsxPJp5Fc7KXH94Emd1+W7uh4gqR+VH5DjK8NG8K1y9ixYVvmjSBTe5h54yJszFEtYxI/O5FCyOWYeHdPAgdw9+UEp5uNGSrsI9mEqYAVl4pnYemOmf4AABSnSURBVISNOaplTOJnJ1IIuSTxp8zse2Y228xOMrNbgKVmNt3MpkcdYByosI9kE6YCVlwqnoURNuaoljGJn51IIeTyc/rU9P3XB7RPIyiQckpeI4qhnsFr8+cHP6FPnhwkcA1qk55BVfOfmM+GbRuYPH4yzXOaMw62CjNvXISNOaplTOJnJ1IIOY9OjwtVMRMRkbEm28C2YX9ON7ODzOx2M/s/6ekjzOyzUQQpIiIiucvlmPgPgUeBQ9LTvwcujSogERERyU0uSfxAd78b2AXg7juB7kijEhERkWHlksTfMrMDCAaxYWbHElzwRURERIool9HpXwXuB95rZr8AJgLnRBqViIiIDGvYPXF3XwmcBBwHfB440t1XRx2YSGKEqI7T9FATFQsrsGuMioUVND3UVLAwRyp0zFFVC2pqgooKMAvum7LHEZdiKXGJIzbiUkkqLnHkg7sPeQM+Beybfvw14KfA9OFeF9VtxowZLhIbra3uqZQ77L6lUkH7AI0PNjoLGHRrfLCxCIHnJnTMIT6PcIE09u+z59Y4OI7W1a2eak71izfVnPLW1aOMIaS4xBEbUW0bSY0jJKDdM+TEXAqgrHb3o9LXTP8GcANwtbsfE+U/F9noPHGJlRDVcSoWVtDtg8eElls5O6+O55WMQ8ccVbWgigrozjCetrwcdvaPIy7FUuISR2zEpZJUXOIIacTnibN7JPpc4FZ3fwjYI5/BiSRWiOo4mZLhUO1xEDrmqKoFZUrgWdrjUiwlLnHERlwqScUljjzJJYn/ycy+B5wLPGxme+b4OpHSF6I6TrmVZ5w1W3schI45qmpB5VneL0N7XIqlxCWO2IhLJam4xJEnuSTjTxNc7OUj7v4GsD9wWaRRiSRFiOo4DTMaMnaRrT0OQsccVbWghizvl6E9LsVS4hJHbMSlklRc4siXTAfK43zTwDaJndZW96oqd7PgfogBMo0PNnr5NeXOArz8mvJYD2rrETrmEJ9HuEAa3cvLg4FI5eUZB7X1hrC61aturHJbYF51Y1XRBpPFJY7YiGrbSGocITDSgW1xo4FtIiIy1oxmYJuIiIjEkJK4iIhIQimJi4iIJJSSuIiISEIpiYuIiCSUkriIiEhCKYmLjFKUlaqWzj2SnWWGm7GzzFg698i89Bub6lqlVE1KpAiUxEVGoa2jjYYHGujc1onjdG7rpOGBhrwkxaVzj+Skh9dR4WBAhcNJD68bdSKPMuZwgbQFV1zr7AxqSXV2BtNK5CI508VeREYhykpVO8uMigxfz50GFbtG/r2NTXWthFaTEikGXexFJAJRVqoqz5Kns7XnKjbVtUqsmpRIMSiJi4xClJWqui1ce65iU12rxKpJiRSDkrjIKERZqWr5aUcwcKfb0+2jEZvqWqVWTUqkCJTERUahfko9LR9voWp8FYZRNb6Klo+3UD+lftR9z35oLU997Ah2WpC8dxo89bEjmP3Q2tjGHC6QemhpCY6BmwX3LS1Bu4jkRAPbREREYk4D20REREqMkriIiEhCKYmLiIgklJK4iIhIQimJi4iIJJSSuEgGURUIiU3hkTBUpERKTQlt0xXFDkAkbnoKhHTt6ALoLRACjOpc6qj6jVRPkZKuIObeIiWg87klmUpsm9Z54iIDRFUgJDaFR8JQkRIpNQndpnWeuEiOoioQEpvCI2GoSImUmhLbppXERQaIqkBIbAqPhKEiJVJqSmybVhIXGSCqAiGxKTwShoqUSKkpsW1aSVxkgKgKhMSm8EgYKlIipabEtmkNbBMREYk5DWwTEREpMUriIiIiCaUkLiIiklBK4iIiIgmlJC4iIpJQkSVxM3uPmT1pZuvMbK2ZXZJhntlmts3Mnk3fro4qHhERkVIT5Z74TuB/ufsRwLHAF8zsiAzzLXP32vRtYYTxyFAirOqTxMpdSYw5kUqompRIMURWxczdNwGb0o/fNLPngUOBdVG9p4xQhFV9kli5K4kxJ1KJVZMSKYaCXOzFzKqBp4Ead/9zn/bZwE+AjcArwD+6+9qh+tLFXiIQYVWfJFbuSmLMiZTQalIixZDtYi+R1xM3s30IEvWlfRN42kqgyt23m9nHgPuA92foowFoAJic0IvUx1qEVX2SWLkriTEnUolVkxIphkhHp5tZJUECb3P3nw583t3/7O7b048fBirN7MAM87W4e527102cODHKkMemCKv6JLFyVxJjTqQSqyYlUgxRjk434HbgeXf/dpZ53p2eDzObmY5nS1QxSRYRVvVJYuWuJMacSCVWTUqkGKLcE58FXAic0ucUso+Z2T+Y2T+k5zkHWGNmzwHfAc7zpFVkKQURVvVJYuWuJMacSCVWTUqkGFTFTEREJOZUxUxERKTEKImLiIgklJK4iIhIQimJi4iIJJSSuIiISEIpiYtk0PRQExULK7BrjIqFFTQ91FTskEREBon8sqsiSdP0UBOL2xf3Tnd7d+/0LXNvKVZYIiKDaE9cZICWFS2h2kVEikVJXGSAbu8O1S4iUixK4iIDlFt5qHYRkWJREhcZoGFGQ6h2EZFi0cA2kQF6Bq+1rGih27spt3IaZjRoUJuIxI4KoIiIiMScCqCIiIiUGCVxERGRhFISFxERSSglcRERkYRSEhcREUkoJXEREZGEUhKPQFsbVFdDWVlw39ZW7Ijyq62jjepF1ZRdU0b1omraOkpsAaVgtC2JjI4u9pJnbW3Q0ABdXcF0Z2cwDVBfX7y48qWto42GBxro2hEsYOe2ThoeCBawfkoJLKAUjLYlkdHTxV7yrLo6SNwDVVXB+vWFjib/qhdV07lt8AJWja9i/aXrCx+QJJa2JZHc6WIvBbJhQ7j2pNmwLfOCZGsXyUbbksjoKYnn2eTJ4dqTZvL4zAuSrV0kG21LIqOnJJ5nzc2QSvVvS6WC9lLQPKeZVGX/BUxVpmieUyILKAWjbUlk9JTE86y+HlpagmPgZsF9S0tpDGqDYMBRy8dbqBpfhWFUja+i5eMtGogkoWlbEhk9DWwTERGJOQ1sExERKTFK4iIiIgmlJC4iIpJQSuIiIiIJpSQuIiKSUEriImPQ8uub2Lh/BbvM2Lh/Bcuvb8pb3ypqIlI4KoAiMsYsv76JaVcvZu8dwfSkrd1MuHoxy4Hjr7xlVH2rqIlIYek8cZExZuP+FUza2j24fUI5k17fOaq+VdREJBo6T1xEADgkQwIfqj0MFTURKSwlcZEx5pUJ5aHaw1BRE5HCUhIXGWPWX97AW5X9296qDNpHS0VNRApLSVxkjDn+yltYtbCRjRPK2UVwLHzVwsZRD2oDFTURKTQNbBMREYk5DWwTEREpMUriIiIiCaUkLiIiklBK4iIiIgmlJC4iIpJQSuIiIiIJpSQuIiKSUEriIiIiCaUkLiIiklBK4iIiIgmlJC4iIpJQSuIiIiIJpSQuIiKSUEriIiIiCaUkLiIiklBK4iIiIgmlJC4iIpJQSuIiIiIJpSQuIiKSUJElcTN7j5k9aWbrzGytmV2SYR4zs++Y2R/MbLWZTY8qHsmfto42qhdVU3ZNGdWLqmnraCt2SCIiY1JFhH3vBP6Xu680s32BFWb2M3df12ee04D3p2/HAIvT9xJTbR1tNDzQQNeOLgA6t3XS8EADAPVT6osZmojImBPZnri7b3L3lenHbwLPA4cOmO1M4Mce+DWwn5kdHFVMMnrzn5jfm8B7dO3oYv4T84sUkYjI2FWQY+JmVg1MA54Z8NShwMt9pjcyONFjZg1m1m5m7Zs3b44qTMnBhm0bQrWLiEh0Ik/iZrYP8BPgUnf/80j6cPcWd69z97qJEyfmN0AJZfL4yaHaRUQkOpEmcTOrJEjgbe7+0wyz/Al4T5/pSek2ianmOc2kKlP92lKVKZrnNBcpIhGRsSvK0ekG3A487+7fzjLb/cD/TI9SPxbY5u6boopJRq9+Sj0tH2+hanwVhlE1voqWj7doUJuISBGYu0fTsdnxwDKgA9iVbv4nYDKAu383nej/Hfgo0AVc5O7tQ/VbV1fn7e1DziIiIlJSzGyFu9cNbI/sFDN3Xw7YMPM48IWoYhARESllumKbiIhIQimJi4iIJJSSuIiISEIpiYuIiCSUkriIiEhCKYmLiIgklJK4iIhIQimJi4iIJJSSuIiISEJFdtnVqJjZZqCz2HHk6EDgtWIHEaFSXz4o/WXU8iVfqS9jqS8f5LaMVe4+qIxn4pJ4kphZe6Zr3ZaKUl8+KP1l1PIlX6kvY6kvH4xuGfVzuoiISEIpiYuIiCSUkni0WoodQMRKffmg9JdRy5d8pb6Mpb58MIpl1DFxERGRhNKeuIiISEIpieeJmZWb2SozezDDc/PMbLOZPZu+XVyMGEfKzNabWUc69vYMz5uZfcfM/mBmq81sejHiHI0clnG2mW3rsw6vLkacI2Vm+5nZEjP7nZk9b2YfGvB8otdhDsuX9PX3wT6xP2tmfzazSwfMk9h1mOPyJX0dfsXM1prZGjO7w8zGDXh+TzO7K73+njGz6lz6rYgi2DHqEuB54F1Znr/L3b9YwHjy7WR3z3Ye42nA+9O3Y4DF6fukGWoZAZa5++kFiya/bgIecfdzzGwPIDXg+aSvw+GWDxK8/tz9BaAWgh0G4E/AvQNmS+w6zHH5IKHr0MwOBb4MHOHufzGzu4HzgB/2me2zwFZ3f5+ZnQf8C3DucH1rTzwPzGwSMBe4rdixFMmZwI898GtgPzM7uNhBScDMxgMnArcDuPtf3f2NAbMldh3muHylZA7worsPvOhVYtfhANmWL+kqgL3MrILgn8xXBjx/JvCj9OMlwBwzs+E6VRLPj0XA5cCuIeY5O/0T1xIze0+B4soXBx4zsxVm1pDh+UOBl/tMb0y3JclwywjwITN7zsz+j5kdWcjgRukwYDPwg/Qhn9vMbO8B8yR5HeayfJDc9TfQecAdGdqTvA77yrZ8kNB16O5/Am4ANgCbgG3u/tiA2XrXn7vvBLYBBwzXt5L4KJnZ6cCr7r5iiNkeAKrd/SjgZ+z+byspjnf36QQ/133BzE4sdkARGG4ZVxJc9nAq8L+B+wod4ChUANOBxe4+DXgLuLK4IeVVLsuX5PXXK32o4AzgnmLHEoVhli+x69DMJhDsaR8GHALsbWYX5KNvJfHRmwWcYWbrgTuBU8yste8M7r7F3d9JT94GzChsiKOT/i8Sd3+V4DjVzAGz/Ano++vCpHRbYgy3jO7+Z3ffnn78MFBpZgcWPNCR2QhsdPdn0tNLCJJeX0leh8MuX8LXX1+nASvd/f9leC7J67BH1uVL+Dr8O+CP7r7Z3XcAPwWOGzBP7/pL/+Q+HtgyXMdK4qPk7le5+yR3ryb4Gejn7t7vP6wBx6XOIBgAlwhmtreZ7dvzGPgwsGbAbPcD/zM9OvZYgp+KNhU41BHLZRnN7N09x6fMbCbBd2fYL1gcuPv/BV42sw+mm+YA6wbMlth1mMvyJXn9DXA+2X9qTuw67CPr8iV8HW4AjjWzVHoZ5jA4D9wPfCb9+ByCXDLshVw0Oj0iZrYQaHf3+4Evm9kZwE7gdWBeMWML6SDg3vR3pwL4T3d/xMz+AcDdvws8DHwM+APQBVxUpFhHKpdlPAdoNLOdwF+A83L5gsXIl4C29M+VLwEXldg6HG75kr7+ev7BPBX4fJ+2klmHOSxfYtehuz9jZksIDgnsBFYBLQPyxO3Af5jZHwjyxHm59K0rtomIiCSUfk4XERFJKCVxERGRhFISFxERSSglcRERkYRSEhcREUkoJXGREpau/JSpsl7G9jy83yfM7Ig+00vNrC6H1x2cj3jMbKKZPTLafkSSQklcRPLpE8ARw8412FeBW0f75u6+GdhkZrNG25dIEiiJixRR+mpxD6WLOqwxs3PT7TPM7Kl0QZZHe676l96zvcmCespr0leuwsxmmtmv0gVAftnn6mW5xvB9M/tN+vVnptvnmdlPzewRM/tvM/vXPq/5rJn9Pv2aW83s383sOIIrEn4rHd9707N/Kj3f783shCxhnA08ku673MxuSC/fajP7Urp9vZldl+673cympz+bF3suCpJ2H1Cf6/KLJJmu2CZSXB8FXnH3uRCU1TSzSoICD2e6++Z0Ym8G/j79mpS711pQpOX7QA3wO+AEd99pZn8HfJMgMeZiPsElHv/ezPYDfmNmj6efqwWmAe8AL5jZ/wa6gX8muD75m8DPgefc/Zdmdj/woLsvSS8PQIW7zzSzjwFfJ7iOdC8zO4ygjnJPfYEGoBqoTS/P/n1m35Be9hsJajHPAsYRXCb3u+l52oFrc1x2kURTEhcprg7g38zsXwiS3zIzqyFIzD9LJ8FygvKFPe4AcPenzexd6cS7L/AjM3s/QVnVyhAxfJigiM8/pqfHAZPTj59w920AZrYOqAIOBJ5y99fT7fcAHxii/5+m71cQJOeBDiYoJdrj74Dvpssx0vM+afen7zuAfdz9TeBNM3vHzPZL1xF/laBSlEjJUxIXKSJ3/72ZTSe45vW1ZvYEQRW1te7+oWwvyzD9DeBJd/+kmVUDS0OEYcDZ7v5Cv0azYwj2wHt0M7K/GT19ZHv9Xwj+cQjT164Bse3q0/e4dJ8iJU/HxEWKyMwOAbrcvRX4FsFP1C8AE83sQ+l5Ks3syD4v6zlufjxBpaptBGULe8pOzgsZxqPAl9LVlTCzacPM/1vgJDObYEHJxL4/279J8KtAGL+n/x76z4DPp/tmwM/pufgAgyvtiZQkJXGR4ppCcAz6WYLjxde6+18JKjb9i5k9BzxL/9rDb5vZKoJjwJ9Nt/0rcF26Peze8jcIfn5fbWZr09NZpWuvfxP4DfALYD2wLf30ncBl6QFy783cw6D+3gJeNLP3pZtuIyjduDq9/P8j3OJwMvBQyNeIJJKqmIkkiJktBf7R3duLHMc+7r49vbd8L/B9d793FP19Epjh7l/LQ2xPEwwK3DravkTiTnviIjISC9K/HqwB/khwWteIpf8BWD/aoMxsIvBtJXAZK7QnLiIiklDaExcREUkoJXEREZGEUhIXERFJKCVxERGRhFISFxERSSglcRERkYT6/+9QXFhY4Pq1AAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Un39g4GIBT6e"
      },
      "source": [
        "# create a KNN model\n",
        "knn = KNeighborsClassifier()  # default value of k is 5"
      ],
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s92c9uSbcFcV",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "eeb7bd43-a6d3-4c78-8998-fedb18db965c"
      },
      "source": [
        "# fit the KNN model to the train data\n",
        "knn.fit(X_train, y_train)"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "KNeighborsClassifier()"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yTQD59DXcPyE",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "bb01f98d-27ed-4ad1-9d5d-ecd5df26f175"
      },
      "source": [
        "# make predictions on the test data\n",
        "y_pred = knn.predict(X_test)\n",
        "y_pred"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([2, 2, 1, 1, 2, 0, 0, 2, 0, 2, 0, 0, 0, 2, 2, 0, 2, 0, 2, 2, 2, 2,\n",
              "       2, 1, 2, 0, 2, 2, 0, 2, 1, 0, 1, 1, 2, 2, 0, 1, 1, 1, 1, 0, 0, 0,\n",
              "       1, 2, 1, 0, 0, 0])"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "y_test"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "W81ZFkPHj9cO",
        "outputId": "3516f3fd-d775-4f8a-d326-13533d469057"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([2, 2, 1, 1, 2, 0, 0, 1, 0, 2, 0, 0, 0, 2, 2, 0, 2, 0, 2, 2, 2, 2,\n",
              "       2, 1, 2, 0, 2, 2, 0, 2, 1, 0, 1, 1, 2, 2, 0, 1, 1, 1, 1, 0, 0, 0,\n",
              "       1, 2, 1, 0, 0, 0])"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nHCpU5Dkcj-G",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "707566f1-77ba-4445-a295-d9095c1b3822"
      },
      "source": [
        "# accuracy of the KNN model\n",
        "accuracy = (y_test == y_pred).sum()/y_test.shape[0]\n",
        "print('Accuracy:', accuracy)"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy: 0.98\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TClpEyVvc87E",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4ade68f5-5a0c-419c-8838-0ce2a129a7b1"
      },
      "source": [
        "# Can you find the best k(s) for this dataset?\n",
        "accuracies = []\n",
        "\n",
        "for k in range(1, 20):\n",
        "  knn = KNeighborsClassifier(n_neighbors=k)\n",
        "  knn.fit(X_train, y_train)\n",
        "  accuracies.append(knn.score(X_test, y_test))  # another way to compute the accuracy\n",
        "\n",
        "print('Best k:', np.array(accuracies).argmax() + 1)"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Best k: 8\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2A-KzTaRenRo",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 406
        },
        "outputId": "2fe4e0ee-eaf9-491a-ed72-6dc070a18636"
      },
      "source": [
        "# plot k vs accuracy\n",
        "plt.figure(figsize=[8, 6])\n",
        "plt.xlabel('k')\n",
        "plt.ylabel('accuracy')\n",
        "plt.plot(np.arange(1, 20), accuracies)"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f7c04001590>]"
            ]
          },
          "metadata": {},
          "execution_count": 16
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 576x432 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAf4AAAFzCAYAAADfQWsjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXSk9X3n+/dXu1RSL1IVWy+0ShDbfRMCuI2dxQMmE0/j5JolvgkkucGejBlPwr2Z40tiOJwhPsxwGCdMPPGE6zlkgm0SD9hDvJA5OIAxxDM3xpd22CEN3SXobnUDVepVpV36zh/1lLospO5qtZ56lvq8zqnTVc/G96HU/dXv9/y+v5+5OyIiItIcWqIOQERERBpHiV9ERKSJKPGLiIg0ESV+ERGRJqLELyIi0kSU+EVERJpIW9QBNEI2m/UtW7ZEHYaIiEhD/OhHPyq5e26pfU2R+Lds2cKOHTuiDkNERKQhzOyN5fapq19ERKSJKPGLiIg0ESV+ERGRJqLELyIi0kSU+EVERJqIEr+IiEgTUeIXERFpIkr8IiIiTUSJX0REpImEmvjN7F4ze9vMXlxmv5nZF8xsl5k9b2YX1+y73sxeC17X12x/r5m9EJzzBTOzMO9BREQkTcJu8X8Z2H6C/VcA5wevG4AvAphZP/CHwPuBS4A/NLP1wTlfBD5Zc96Jri8iIiI1Qp2r392/b2ZbTnDIlcB97u7AU2a2zszOBi4DHnP3gwBm9hiw3cyeBNa4+1PB9vuAq4DvhHYTIjEwcniCV986FnUYTWEo28vmgZ6owxAJTdSL9GwA9tZ83hdsO9H2fUtsfwczu4FKLwKbN29evYhFIvDJr+zg5QNHow6jKWwZ6OHJ3/9Q1GGIhCbqxB8ad78HuAdg27ZtHnE4Iis2OzfPa28f41cu3shvfkC/xIbpr/9hH3/11B4mpufo7miNOhyRUESd+EeATTWfNwbbRqh099dufzLYvnGJ40VSa9+hCWbmnPcP9nPR5vUnP0FWbOTwBH/11B6GS2W2nrMm6nBEQhF1Od9DwG8Fo/s/ABxx9wPAI8CHzWx9MKjvw8Ajwb6jZvaBYDT/bwHfjix6kQYolMYAyOcyEUeSfvlsL3D8/7lIGoXa4jez+6m03LNmto/KSP12AHf/z8DDwEeAXcA48Ilg30Ez+7fA08Glbq8O9AN+h0q1QDeVQX0a2CepViiWAcjneiOOJP0Gs5VfroaD/+ciaRT2qP7rTrLfgd9dZt+9wL1LbN8B/OSqBCiSAIVSmXU97fRnOqIOJfW6O1rZsK6bQkmJX9Ir6q5+ETmJQnGMfFbd/I2Sz2UoFNXVL+mlxC8Sc4ViWd38DZTPZigUy1Q6JEXSR4lfJMaOTc7w9rEpDexroHyul2NTsxTHpqIORSQUSvwiMTYcPGuujjaX8FV/ySpogJ+klBK/SIxVk8+QWvwNUx3Zr8QvaaXELxJjheIYLYbmjm+gc9Z209XeogF+klpK/CIxtrtUZuP6HjrbNH1so7S0GFsGMirpk9RS4heJscqIfnXzN9pQrlctfkktJX6RmJqfd4ZLYxrYF4F8LsPeQxNMz85HHYrIqlPiF4mpA0cnmZyZV4s/Avlchrl5Z89BdfdL+ijxi8RUtatZib/xqr0suzWyX1JIiV8kpo6X8qmrv9FUyy9ppsQvElOF4hiZjlbO6OuMOpSm09fVTq6vUwP8JJWU+EViqlCqzNFvZlGH0pTy2czCzIkiaaLELxJTKuWLVj7Xq1p+SSUlfpEYmpyZY/+RCZXyRWgol+FgeZrD49NRhyKyqpT4RWJouFTGXSP6o1T9f6+R/ZI2SvwiMVQdTa7EH53BoLdFA/wkbZT4RWKommyqK8VJ421a3017q+k5v6SOEr9IDBVKZc5e20VPR1vUoTStttYWNvf3qMUvqaPELxJDheKYuvljIJ/r1SQ+kjpK/CIx4+6VUj6N6I9cPpfhjdFx5uY96lBEVo0Sv0jMFMemODY1qxZ/DAxle5mem2ffofGoQxFZNUr8IjFzfES/WvxR05z9kkZK/CIxs5D4NaI/ctVfvnZrgJ+kiBK/SMwUimN0trWwYV131KE0vf5MB+t62lXSJ6mixC8SM4VSmcFshpYWLc4TB/lsRiV9kipK/CIxM1zS4jxxks/1apU+SZVQE7+ZbTeznWa2y8xuXmL/uWb2uJk9b2ZPmtnGmn2fM7MXg9ev1Wz/spkNm9mzwevCMO9BpJGmZ+fZc3BcpXwxks9leOvoFGNTs1GHIrIqQkv8ZtYK3A1cAWwFrjOzrYsOuwu4z90vAG4H7gzO/SXgYuBC4P3ATWa2pua833f3C4PXs2Hdg0ij7TlYqRlXiz8+qr+EDWtkv6REmC3+S4Bd7l5w92ngAeDKRcdsBb4XvH+iZv9W4PvuPuvuZeB5YHuIsYrEQvVZskr54mOoWtJX0nN+SYcwE/8GYG/N533BtlrPAdcE768G+sxsINi+3cx6zCwLfAjYVHPeHcHjgc+bWWc44Ys0XnX0uFr88bF5oIcW0/K8kh5RD+67CbjUzJ4BLgVGgDl3fxR4GPh74H7gB8BccM4twLuB9wH9wGeWurCZ3WBmO8xsR7FYDPcuRFZJoThGtreTNV3tUYcigc62Vjau12I9kh5hJv4RfryVvjHYtsDd97v7Ne5+EXBrsO1w8OcdwTP8XwQMeDXYfsArpoAvUXmk8A7ufo+7b3P3bblcbrXvTSQUhaJG9MdRPpfR7H2SGmEm/qeB881s0Mw6gGuBh2oPMLOsmVVjuAW4N9jeGnT5Y2YXABcAjwafzw7+NOAq4MUQ70GkoQqlsmbsi6F8tlLSN6/FeiQFQlvs291nzexG4BGgFbjX3V8ys9uBHe7+EHAZcKeZOfB94HeD09uB/1HJ7RwFftPdq7U0XzWzHJVegGeBT4V1DyKNdHh8moPlabX4YyifyzAxM8ebRyc5RzMqSsKFlvgB3P1hKs/qa7fdVvP+QeDBJc6bpDKyf6lrXr7KYYrEwu6FOfo1oj9uahfrUeKXpIt6cJ+IBI6X8qnFHzdDQXmlSvokDZT4RWKiUCrT1mJs6u+JOhRZ5Iy+TjIdrRrgJ6mgxC8SE4XiGJsHemhv1V/LuDEz8rleLc8rqaB/YURiolAs6/l+jKmkT9JCiV8kBubmnTdGxxemh5X4yWd72X9kgsmZuZMfLBJjSvwiMTByaILpuXkN7IuxfC6DO7w+qla/JJsSv0gM7C5pcZ64qy3pE0kyJX6RGCgs1PCrxR9Xg9lq4tcAP0k2JX6RGCgUx1jb3U5/piPqUGQZPR1tnL22Sy1+STwlfpEYqC7OE0xTLTGVz2XYXVLil2RT4heJgUJpTKV8CZDP9lIojuGuxXokuZT4RSI2NjXLW0enNKI/AfK5DMcmZymNTUcdisiKKfGLRGxYA/sSo1p1oQF+kmRK/CIRK6iULzGqv5wV9JxfEkyJXyRiu4tlzODcAS3OE3cb1nXT2daiFr8kmhK/SMQKxTE2ru+mq7016lDkJFpajMGs5uyXZFPiF4mYFudJlnwuo65+STQlfpEIzc87w6WyRvQnSD7by56D40zPzkcdisiKKPGLROjNo5NMzMxpYF+C5HMZ5uadPQfHow5FZEWU+EUiVH1WPKRSvsRQSZ8knRK/SISGVcqXONXFeob1nF8SSolfJEK7i2UyHa2cuaYz6lCkTmu728n2dmhkvySWEr9IhAqlMoNanCdx8tnehYmXRJJGiV8kQoWiFudJonxOtfySXEr8IhGZnJlj5PCESvkSKJ/LMFqe5sj4TNShiJwyJX6RiLw+WsZdA/uSqNpLs1vd/ZJASvwiESloVb7EqvbSqLtfkkiJXyQi1TrwQSX+xNnU30Nbi6mWXxIp1MRvZtvNbKeZ7TKzm5fYf66ZPW5mz5vZk2a2sWbf58zsxeD1azXbB83sh8E1v2ZmHWHeg0hYCsUyZ63pItPZFnUocoraW1vYPNCjFr8kUmiJ38xagbuBK4CtwHVmtnXRYXcB97n7BcDtwJ3Bub8EXAxcCLwfuMnM1gTnfA74vLufBxwCfjusexAJ027N0Z9oKumTpAqzxX8JsMvdC+4+DTwAXLnomK3A94L3T9Ts3wp8391n3b0MPA9st0qx8+XAg8FxXwGuCvEeRELh7pVSPiX+xBrKZXh9dJy5eY86FJFTEmbi3wDsrfm8L9hW6zngmuD91UCfmQ0E27ebWY+ZZYEPAZuAAeCwu8+e4JoisVcam+bY5Kxq+BMsn8swPTvPyKGJqEMROSVRD+67CbjUzJ4BLgVGgDl3fxR4GPh74H7gB8DcqVzYzG4wsx1mtqNYLK5y2CKnpzooTC3+5KqWYaqkT5ImzMQ/QqWVXrUx2LbA3fe7+zXufhFwa7DtcPDnHe5+obv/ImDAq8AosM7M2pa7Zs2173H3be6+LZfLreZ9iZy2QrDAy5Bq+BOrWoapAX6SNGEm/qeB84NR+B3AtcBDtQeYWdbMqjHcAtwbbG8NuvwxswuAC4BH3d2pjAX4WHDO9cC3Q7wHkVAMl8p0tLVwzrruqEORFerPdLCmq21hhUWRpAgt8QfP4W8EHgFeAb7u7i+Z2e1m9tHgsMuAnWb2KnAmcEewvR34H2b2MnAP8Js1z/U/A3zazHZReeb/F2Hdg0hYCsUxBgcytLZocZ6kMjPyuV61+CVxQi0gdveHqTyrr912W837Bzk+Qr/2mEkqI/uXumaBSsWASGIVimXedVZf1GHIacrnMvz9rtGowxA5JVEP7hNpOjNz8+w5OK6BfSkwlOvlzaOTlKdmT36wSEwo8Ys02J6D48zOu0r5UqA6wG+4pO5+SQ4lfpEGW1icRy3+xFso6dOc/ZIgSvwiDbZQw68Wf+KdO9CDmUr6JFmU+EUarFAsM5DpYG1Pe9ShyGnqam9l4/ruhXkZRJJAiV+kwQolzdGfJvlsr5bnlURR4hdpsEKxrG7+FMnnMgyXylTmFxOJPyV+kQY6Mj7DaHlaLf4Uyed6GZ+e482jk1GHIlIXJX6RBqou6JLXHP2pMaQ5+yVhlPhFGkilfOlT/SVOz/klKZT4RRqoUByjrcXY3N8TdSiySs5c00lPRyu71eKXhFDiF2mgQrHM5v4e2lv1Vy8tzIzBbEYlfZIY+tdHpIGGS2V186dQPter5XklMZT4RRpkbt4ZHi1rYF8K5bMZ9h2aYHJmLupQRE5KiV+kQfYfnmB6dn5hYRdJj3wugzu8MToedSgiJ6XEL9Ig1YVc1OJPnyGN7JcEUeIXaRCV8qXXYLWWXwP8JAGU+EUapFAaY01XGwOZjqhDkVWW6WzjrDVdWp5XEkGJX6RBCsXKwD4zizoUCUE+l9HsfZIISvwiDVJZnEfd/GlVSfxjWqxHYk+JX6QBylOzvHl0Us/3Uyyf7eXo5Cyj5emoQxE5ISV+kQYYLlUH9mlEf1pVf6lTd7/EnRK/SAMcL+VTiz+tVNInSaHEL9IAhWIZM9gyoMSfVues66ajrUUlfRJ7SvwiDVAoldmwrpuu9taoQ5GQtLYYWwZ61OKX2FPiF2mAQnFMz/ebQD7bq2f8EntK/CIhc/fKqnwq5Uu9fC7DnoPjzMzNRx2KyLKU+EVC9tbRKcan5xjSwL7Uy+d6mZ139h7UYj0SX6EmfjPbbmY7zWyXmd28xP5zzexxM3vezJ40s401+/7IzF4ys1fM7AsWTHcWHLfTzJ4NXmeEeQ8ip6ugxXmahkr6JAlCS/xm1grcDVwBbAWuM7Otiw67C7jP3S8AbgfuDM79WeDngAuAnwTeB1xac95vuPuFwevtsO5BZDXsLmlxnmYxlA1K+koa4CfxFWaL/xJgl7sX3H0aeAC4ctExW4HvBe+fqNnvQBfQAXQC7cBbIcYqEppCcYyejlbOWtMVdSgSsrU97QxkOtTil1gLM/FvAPbWfN4XbKv1HHBN8P5qoM/MBtz9B1R+ETgQvB5x91dqzvtS0M3/b0wrnkjMFYplBrMZLc7TJLRYj8Rd1IP7bgIuNbNnqHTljwBzZnYe8B5gI5VfFi43sw8G5/yGu/8U8MHg9X8udWEzu8HMdpjZjmKxGPZ9iCyrUFIpXzPJZ3vV1S+xFmbiHwE21XzeGGxb4O773f0ad78IuDXYdphK6/8pdx9z9zHgO8DPBPtHgj+PAf+VyiOFd3D3e9x9m7tvy+Vyq3tnInWanJlj36EJBlXK1zTyuQylsWmOTMxEHYrIksJM/E8D55vZoJl1ANcCD9UeYGZZM6vGcAtwb/B+D5WegDYza6fSG/BK8DkbnNsO/DLwYoj3IHJa3hgdxx2V8jWRvObsl5gLLfG7+yxwI/AI8ArwdXd/ycxuN7OPBoddBuw0s1eBM4E7gu0PAruBF6iMA3jO3f+GykC/R8zseeBZKj0Ifx7WPYicroVSvqy6+puFSvok7trCvLi7Pww8vGjbbTXvH6SS5BefNwf8yyW2l4H3rn6kIuGoLtgyqBZ/09jc30Nri+k5v8RW1IP7RFJtd3GMM9d00tsZ6u/YEiPtrS1s7u9Ri19iS4lfJESFYlnd/E0on1VJn8SXEr9ISNw9WJVP3fzNJp/LMDxaZm7eow5F5B2U+EVCMlqe5ujkrGr4m1A+18v07Dz7D09EHYrIOyjxi4RkWHP0N63qEszVwZ0icaLELxKSainfkJ7xNx3V8kucKfGLhKRQLNPR1sKG9d1RhyINlu3toK+rTQP8JJaU+EVCsrtYZstApaZbmouZkc9pzn6JJyV+kZAUSmMq5WtiQyrpk5hS4hcJwczcPHtGxzWwr4nlcxkOHJlkfHo26lBEfowSv0gI9h4cZ3beVcrXxI4P8FOrX+JFiV8kBNV/7LUcb/NaWKxHJX0SM3UlfjP7hpn9Us0SuiJyAtVBXVqOt3ltGchgppI+iZ96E/n/C/w68JqZ/Xsze1eIMYkkXqFYpj/TwbqejqhDkYh0tbdyztpudfVL7NSV+N39u+7+G8DFwOvAd83s783sE2bWHmaAIklUWZxHrf1ml89lVNInsVN3172ZDQAfB/4F8Azwp1R+EXgslMhEEqxQ0uI8AkO5XoaLZdy1WI/ER12LhJvZN4F3AX8J/O/ufiDY9TUz2xFWcCJJdGRihtLYtEb0C/lchvL0HG8dneKstV1RhyMC1Jn4gS+4+xNL7XD3basYj0jiVQdzqatfqhM4FYpjSvwSG/V29W81s3XVD2a23sx+J6SYRBKtOphLLX6pPu7ZrZI+iZF6E/8n3f1w9YO7HwI+GU5IIsk2XCrT2mJs7u+JOhSJ2Flruuhub2VYI/slRupN/K1mtrDSiJm1AqpTEllCoTTG5v4eOto07UWza2kxBrMa2S/xUu+/TH9LZSDfL5jZLwD3B9tEZBGV8kmtfE6L9Ui81Jv4PwM8Afyr4PU48AdhBSWSVPPzznCprFI+WZDP9bLv0DhTs3NRhyIC1Dmq393ngS8GLxFZxsjhCaZm5zWwTxYM5TLMO7wxOs5PnNkXdTgidc/Vf76ZPWhmL5tZofoKOziRpKkuyKKufqmqLekTiYN6u/q/RKW1Pwt8CLgP+KuwghJJqoUafrX4JTBYLenTc36JiXoTf7e7Pw6Yu7/h7p8Ffim8sESSqVAs09fZRrZXRS9S0dvZxhl9nRrgJ7FR78x9U8GSvK+Z2Y3ACKAmjcgi1Tn6a6pfRbRYj8RKvS3+3wN6gP8beC/wm8D1JzvJzLab2U4z22VmNy+x/1wze9zMnjezJ81sY82+PzKzl8zsFTP7QnUeATN7r5m9EFxzYbtIHBSKZXXzyzvkc70UtFiPxMRJE38wWc+vufuYu+9z90+4+6+4+1N1nHc3cAWwFbjOzLYuOuwu4D53vwC4HbgzOPdngZ8DLgB+EngfcGlwzhepzBp4fvDaXtedioRsfHqWA0cmNbBP3iGfzXBkYoaD5emoQxE5eVe/u8+Z2c+v4NqXALvcvQBgZg8AVwIv1xyzFfh08P4J4FvV/yzQRWV2QAPagbfM7GxgTfWXDjO7D7gK+M4K4msqU7NzqLERrp1vHgM0sE/eaSj4mdj55jEuPrfeJ6zp1t7aQmuLOmyjUO9P4DNm9hDw34CFESru/o0TnLMB2FvzeR/w/kXHPAdcA/wpcDXQZ2YD7v4DM3sCOEAl8f+Zu79iZtuC69Rec0Od99C0vvz/DfPZv3n55AfKqhg6Qy1++XHnnVFJ/L/+X34YcSTxcUZfJ9//gw/R1d4adShNp97E3wWMApfXbHPgRIm/HjcBf2ZmHwe+T2XQ4JyZnQe8B6g+83/MzD4ITNR7YTO7AbgBYPPmzacZZrI9VThIrq+TT/zclqhDSb3+ng7epUlaZJFN/T3c9X/8NG8fm4w6lFjY9dYY33hmhNdHy7z7rDVRh9N06p257xMruPYIsKnm88ZgW+1191Np8WNmvcCvuPthM/sk8JS7jwX7vgP8DPCXHP9lYMlr1lz7HuAegG3btjV1J3ehNMZPb1zL71x2XtShiDStj71348kPahIvjhzhG8+MUCgq8Ueh3pn7vmRm9y5+neS0p4HzzWzQzDqAa4GHFl03G5QJAtwCVK+5B7jUzNrMrJ3KwL5X3P0AcNTMPhCM5v8t4Nt13WmTmpt3Xh8d13NnEYmNwWAArGYzjEa9Xf3/veZ9F5Xn8ftPdIK7zwY1/48ArcC97v6Smd0O7HD3h4DLgDvNzKl09f9ucPqDVB4rvEDlkcLfuvvfBPt+B/gy0E1lUJ8G9p3AyKEJpmfnNdJcRGIj09nGWWu6NKlRROrt6v/r2s9mdj/wP+s472Hg4UXbbqt5/yCVJL/4vDngXy5zzR1USvykDrtLmkJWROInn8uwu6TEH4V6J/BZ7HzgjNUMRMIxHPxGrWViRSRO8rkMw8UxTWoUgbpa/GZ2jEqXe9WbwGdCiUhWVaE0xpquNgYymjteROIjn+3l6OQso+Vpsr2dUYfTVOrt6ld9UkJVp5DVzMYiEifVXshCsazE32D1juq/2szW1nxeZ2ZXhReWrJZK4lc3v4jES3U2Q43sb7x6n/H/obsfqX5w98PAH4YTkqyW8tQsbx6dXPgLJiISF+es66ajrYWCBvg1XL2Jf6njNOF0zA0Hf6FUyicicdPaYgwOZNTij0C9iX+Hmf2JmQ0Frz8BfhRmYHL6dhdVyici8ZXPZVTLH4F6E///BUwDXwMeACY5PtmOxFShWMYMzh3oiToUEZF3GMxm2HNwnJm5+ahDaSr1juovAzeHHIusskKpzIZ13Vr9SkRiKZ/rZXbe2XNwXGORGqjeUf2Pmdm6ms/rzeyR8MKS1VAojqmbX0Riq7akTxqn3q7+bDCSHwB3P4Rm7os1d2e4VNbAPhGJraGsSvqiUG/inzezhUXtzWwLPz6Tn8TMm0cnGZ+eY0g1/CISU2t72hnIdKjF32D1luTdCvxPM/s7wIAPAjeEFpWctsLCHP3q6heR+MrnMhRKavE3Ul0tfnf/W2AbsBO4H/h/gIkQ45LTVFgo5VOLX0TiK5/tVYu/wepdpOdfAL8HbASeBT4A/AC4PLzQ5HQUSmV6Olo5a01X1KGIiCwrn8swumOaI+MzrO1pjzqcplDvM/7fA94HvOHuHwIuAg6f+BSJUqFYZjCb0eI8IhJr1ceR6u5vnHoT/6S7TwKYWae7/yPwrvDCktNVKKmUT0TiTyV9jVdv4t8X1PF/C3jMzL4NvBFeWHI6Jmfm2HdoQqV8IhJ7m/t7aGsxtfgbqN6Z+64O3n7WzJ4A1gJ/G1pUclreGB3HXQP7RCT+2ltb2NzfoxZ/A53yCnvu/ndhBCKrpzqiX1NgikgSaLGexqq3q18SpLq+9aC6+kUkAfK5XoZHy8zNa164RlDiT6HdxTHOXNNJpvOUO3RERBpuMJthenae/Yc1PUwjKPGnUKFYJp9VN7+IJEN1IPJuzdnfEEr8KePuwap86uYXkWRYqOXXc/6GUOJPmdHyNEcnZ1XDLyKJke3toK+rTSV9DaLEnzLHF+dRi19EksHMyOc0Z3+jKPGnzEIpn57xi0iCDGVV0tcoSvwpUyiV6WhrYcP67qhDERGpWz6X4c2jk5SnZqMOJfVCTfxmtt3MdprZLjO7eYn955rZ42b2vJk9aWYbg+0fMrNna16TZnZVsO/LZjZcs+/CMO8haQrFMbYM9NDaosV5RCQ5quOShktq9YcttMRvZq3A3cAVwFbgOjPbuuiwu4D73P0C4HbgTgB3f8LdL3T3C6ks/TsOPFpz3u9X97v7s2HdQxIVSirlE5HkWVisR4k/dGG2+C8Bdrl7wd2ngQeAKxcdsxX4XvD+iSX2A3wM+I67j4cWaUrMzM2zZ3RcA/tEJHG2DGQwOz5OScITZuLfAOyt+bwv2FbrOeCa4P3VQJ+ZDSw65lrg/kXb7ggeD3zezDpXK+Ck23twnNl5VymfiCROV3srG9Z1a4BfA0Q9uO8m4FIzewa4FBgB5qo7zexs4KeAR2rOuQV4N/A+oB/4zFIXNrMbzGyHme0oFoshhR8vKuUTkSTL53pVy98AYSb+EWBTzeeNwbYF7r7f3a9x94uAW4Nth2sO+VXgm+4+U3POAa+YAr5E5ZHCO7j7Pe6+zd235XK51bmjmKv+hVEpn4gkUT6bYbhYxl2L9YQpzMT/NHC+mQ2aWQeVLvuHag8ws6yZVWO4Bbh30TWuY1E3f9ALgJkZcBXwYgixJ1KhWGYg08HanvaoQxEROWVDuQzl6TneOjoVdSipFlrid/dZ4EYq3fSvAF9395fM7HYz+2hw2GXATjN7FTgTuKN6vpltodJj8HeLLv1VM3sBeAHIAv8urHtImkKxrG5+EUms43P2q7s/TKGu2+ruDwMPL9p2W837B4EHlzn3dd45GBB3v3x1o0yPQmmMy999RtRhiIisyGB1lb5SmZ89LxtxNOkV9eA+WSVHJmYojU1rRL+IJNZZa7robm9Viz9kSvwpUf2LUl3XWkQkaVpajEHN2R86Jf6UOF7Kpxa/iCRXPpdRSV/IlPhTolAao7XF2NzfE3UoIiIrls/1su/QBJMzcyc/WFZEiT8lCsUym/t76GjTVyoiyTWUy+AOb4xqlvawKEukRKFY1vN9EUm86iJjGg121OEAABRxSURBVOAXHiX+FJibd4ZHVcMvIsk3qFX6QqfEnwL7D08wPTuvgX0ikni9nW2cuaZTI/tDpMSfAtXfjNXVLyJpkM9qsZ4wKfGnwEINv1r8IpIC+Vylll+L9YRDiT8FCsUyfV1tZHs7og5FROS05XO9HJmY4WB5OupQUkmJPwUKpTHyuV4qCxaKiCRbXgP8QqXEnwKFYpkhPd8XkZQYUklfqJT4E258epYDRyYXVrUSEUm6Deu76Wht0cj+kCjxJ5zm6BeRtGltMc4d6GG3En8olPgTbqGUT5P3iEiKaLGe8CjxJ1yhOIYZ6uoXkVTJ53rZMzrOzNx81KGkjhJ/whWKZc5Z201Xe2vUoYiIrJp8NsPsvLP3oBbrWW1K/AlXKeVTa19E0qU6bkkD/FafEn+CuTvDxTJDGtgnIikztFDLr+f8q02JP8HeOjpFeXpOLX4RSZ11PR30ZzrU4g+BEn+CVX8Trq5fLSKSJvlsRrP3hUCJP8GO1/CrxS8i6VNdrEdWlxJ/ghWKZbrbWzlrTVfUoYiIrLp8rpfS2BRHJ2eiDiVVlPgTrFAaYzCboaVFi/OISPrkg/lJ1OpfXUr8CVYoltXNLyKpdbykTyP7V5MSf0JNzc6x79C45ugXkdTa3N9Da4upxb/KlPgT6o3Rceb9eK2riEjadLS1sLm/R7X8qyzUxG9m281sp5ntMrObl9h/rpk9bmbPm9mTZrYx2P4hM3u25jVpZlcF+wbN7IfBNb9mZh1h3kNcVbu+NEe/iKTZYFYj+1dbaInfzFqBu4ErgK3AdWa2ddFhdwH3ufsFwO3AnQDu/oS7X+juFwKXA+PAo8E5nwM+7+7nAYeA3w7rHuKsulylEr+IpFk+m2G4VGZ+3qMOJTXCbPFfAuxy94K7TwMPAFcuOmYr8L3g/RNL7Af4GPAddx83M6Pyi8CDwb6vAFeteuQJUCiWOaOvk76u9qhDEREJTT7Xy9TsPCOHJ6IOJTXCTPwbgL01n/cF22o9B1wTvL8a6DOzgUXHXAvcH7wfAA67++wJrtkUtDiPiDSD/MKc/eruXy1RD+67CbjUzJ4BLgVGgLnqTjM7G/gp4JFTvbCZ3WBmO8xsR7FYXK14Y8Hdg1I+jegXkXRbSPwq6Vs1YSb+EWBTzeeNwbYF7r7f3a9x94uAW4Nth2sO+VXgm+5enbZpFFhnZm3LXbPm2ve4+zZ335bL5U7/bmLkYHmaIxMzC5NbiIikVa63k77ONg3wW0VhJv6ngfODUfgdVLrsH6o9wMyyZlaN4Rbg3kXXuI7j3fy4u1MZC/CxYNP1wLdDiD3Wql1eWo5XRNLOzCpz9qukb9WElviD5/A3UummfwX4uru/ZGa3m9lHg8MuA3aa2avAmcAd1fPNbAuVHoO/W3TpzwCfNrNdVJ75/0VY9xBX1S4vPeMXkWaQz/Wqxb+K2k5+yMq5+8PAw4u23Vbz/kGOj9BffO7rLDFwz90LVCoGmlahVKajtYWN63uiDkVEJHT5bIZvPjPC+PQsPR2hpq2mEPXgPlmBQrHMuQOVqSxFRNKuOpB5WCP7V4USfwIViirlE5HmcXxkvxL/alDiT5jZuXn2HNTiPCLSPAazGcyU+FeLEn/C7D00wcycq5RPRJpGV3sr56zt1sj+VaLEnzDHR/SrxS8izSOf02I9q0WJP2GqP/hajldEmslQrpdCcYzKdC5yOpT4E6ZQGmN9TzvreppyNWIRaVKD2Qzl6TnePjYVdSiJp8SfMLs1R7+INKHqyP7dmrP/tCnxJ0yhWNbAPhFpOtUGj57znz4l/gQ5OjlDaWxKLX4RaTpnr+miq71FiX8VKPEnSPUHXpP3iEizaWkxBrO9KulbBUr8CVIt5dOIfhFpRirpWx1K/AlSKJZpbTE29yvxi0jzGcpm2HdonKnZuahDSTQl/gQplMbYtL6bjjZ9bSLSfPK5XuYd3hgdjzqURFMGSZCCSvlEpIlpsZ7VocSfEPPzzuujKuUTkeY1GPz7pwF+p0eJPyH2H5lgcmZeLX4RaVp9Xe2c0depFv9pUuJPCJXyiYhUR/arxX86lPgT4viqfEr8ItK88rleCiW1+E+HEn9CFEpl+jrbyPV2Rh2KiEhk8tkMh8dnOFiejjqUxFLiT4hCscxgLoOZRR2KiEhkjo/sV3f/SinxJ0ShOKYR/SLS9PJZLdZzupT4E2B8epb9RyY1ol9Emt7G9d20txq7VdK3Ykr8CTBc0oh+ERGAttYWzh3QnP2nQ4k/ARZK+bJq8YuI5LMq6TsdSvwJUE38g3rGLyJCPtfLnoPjzM7NRx1KIinxJ0ChNMaGdd10d7RGHYqISOTyuQwzc87eQxNRh5JISvwJUFmcR619ERGAIZX0nZZQE7+ZbTeznWa2y8xuXmL/uWb2uJk9b2ZPmtnGmn2bzexRM3vFzF42sy3B9i+b2bCZPRu8LgzzHqLm7gyXtDiPiEhVdbzTsGbwW5HQEr+ZtQJ3A1cAW4HrzGzrosPuAu5z9wuA24E7a/bdB/yxu78HuAR4u2bf77v7hcHr2bDuIQ6Kx6YYm5pVKZ+ISGB9poP1Pe3s1sj+FQmzxX8JsMvdC+4+DTwAXLnomK3A94L3T1T3B78gtLn7YwDuPubu4yHGGlu7tTiPiMg75HO96upfoTAT/wZgb83nfcG2Ws8B1wTvrwb6zGwA+AngsJl9w8yeMbM/DnoQqu4IHg983sxSPXl9dd1ptfhFRI7LZzNarGeFoh7cdxNwqZk9A1wKjABzQBvwwWD/+4A88PHgnFuAdwfb+4HPLHVhM7vBzHaY2Y5isRjmPYSqUCzT1d7C2Wu6og5FRCQ28rleisemODY5E3UoiRNm4h8BNtV83hhsW+Du+939Gne/CLg12HaYSu/As8FjglngW8DFwf4DXjEFfInKI4V3cPd73H2bu2/L5XKrfW8NUyiOMZjtpaVFi/OIiFQdX6xHrf5TFWbifxo438wGzawDuBZ4qPYAM8uaWTWGW4B7a85dZ2bVjH058HJwztnBnwZcBbwY4j1ErlBSKZ+IyGILJX2as/+UhZb4g5b6jcAjwCvA1939JTO73cw+Ghx2GbDTzF4FzgTuCM6do9LN/7iZvQAY8OfBOV8Ntr0AZIF/F9Y9RG1qdo69B8dVyicissim/h5aTC3+lWgL8+Lu/jDw8KJtt9W8fxB4cJlzHwMuWGL75ascZmztGR1n3jWiX0Rksc62Vjb19yjxr0DUg/vkBHZrcR4RkWXlsxl2q6TvlCnxx9jxUj61+EVEFsvnenl9tMz8vEcdSqIo8cdYoVgm19dJX1d71KGIiMROPpdhcmae/Ue0WM+pUOKPsUJxTAP7RESWUX0Mquf8p0aJP8YqpXx6vi8ishSt0rcySvwxdbA8zeHxmYUfbBER+XG5vk56O9s0de8pUuKPqWEN7BMROSEzI5/LaHneU6TEH1Mq5RMRObl8NqNn/KdIiT+mCsUy7a3GxvXdUYciIhJb+VwvI4cnmJieizqUxFDij6lCcYxzBzK0teorEhFZTvVxqLr766esElOFUlmlfCIiJ7FQ0qfFeuqmxB9Ds3PzvDGqUj4RkZMZzGp53lOlxB9D+w5NMDPnGtEvInIS3R2tbFjXrVr+U6DEH0MLc/Srq19E5KQGsxnV8p8CJf4YqnZZqatfROTk8rlKSZ+7FuuphxJ/DO0ullnX005/piPqUEREYi+fzTA2NUvx2FTUoSSCEn8MaXEeEZH6VXtHd2uAX12U+GNIi/OIiNSvOhBaJX31UeKPmWOTMxSPTWlEv4hInc5Z201Xe4tK+uqkxB8zBc3RLyJySlpajC0DGZX01UmJP2aqXVVajldEpH5DuV6V9NVJiT9mhotlWgw2D/REHYqISGLkcxn2HhxnenY+6lBiT4k/ZnaXymzq76GzrTXqUEREEiOfyzDvsOegWv0no8QfM4WiFucRETlV1XFRKuk7OSX+GJmfd4ZLYyrlExE5RQslfUr8J6XEHyMHjk4yOTOvUj4RkVPU19VOrq9TI/vroMQfI9UfWJXyiYicurwW66mLEn+MHF+cRy1+EZFTVVmsRy3+k1Hij5FCcYxMRytn9HVGHYqISOLks70cGp/hUHk66lBiLdTEb2bbzWynme0ys5uX2H+umT1uZs+b2ZNmtrFm32Yze9TMXjGzl81sS7B90Mx+GFzza2aWmiXsqnP0m1nUoYiIJI7m7K9PaInfzFqBu4ErgK3AdWa2ddFhdwH3ufsFwO3AnTX77gP+2N3fA1wCvB1s/xzweXc/DzgE/HZY99BohWJZ3fwiIiukVfrq0xbitS8Bdrl7AcDMHgCuBF6uOWYr8Ong/RPAt4JjtwJt7v4YgLuPBdsNuBz49eCcrwCfBb4Y4n38mD/97ms8+erbJz9wBUYOT/Cr2U2hXFtEJO02re+mvdX4j4+9yv3//56owzkllwz2c8sV72nIfyvMxL8B2FvzeR/w/kXHPAdcA/wpcDXQZ2YDwE8Ah83sG8Ag8F3gZmA9cNjdZ2uuuWGp/7iZ3QDcALB58+bVuB8AOttb6O0M53/b5e8+gyt+6qxQri0iknZtrS38q8vO45k9h6IO5ZR1tzduttYwE389bgL+zMw+DnwfGAHmqMT1QeAiYA/wNeDjwLfrvbC73wPcA7Bt2zZfrYA/dekQn7p0aLUuJyIiq+jTv/gTUYcQe2EO7hsBavutNwbbFrj7fne/xt0vAm4Nth2m0pJ/1t0LQev+W8DFwCiwzszalrumiIiILC/MxP80cH4wCr8DuBZ4qPYAM8uaWTWGW4B7a85dZ2a54PPlwMvu7lTGAnws2H49p9ALICIi0uxCS/xBS/1G4BHgFeDr7v6Smd1uZh8NDrsM2GlmrwJnAncE585ReQzwuJm9ABjw58E5nwE+bWa7gAHgL8K6BxERkbSxSiM63bZt2+Y7duyIOgwREZGGMLMfufu2pfZp5j4REZEmosQvIiLSRJT4RUREmogSv4iISBNR4hcREWkiSvwiIiJNRIlfRESkiSjxi4iINBElfhERkSbSFDP3mVkReCPqOEKQBUpRBxGitN8fpP8edX/Jl/Z7TOv9nevuuaV2NEXiTysz27HclIxpkPb7g/Tfo+4v+dJ+j2m/v6Woq19ERKSJKPGLiIg0ESX+ZLsn6gBClvb7g/Tfo+4v+dJ+j2m/v3fQM34REZEmoha/iIhIE1Hijzkz22RmT5jZy2b2kpn93hLHXGZmR8zs2eB1WxSxrpSZvW5mLwSx71hiv5nZF8xsl5k9b2YXRxHnSpjZu2q+l2fN7KiZ/etFxyTu+zOze83sbTN7sWZbv5k9ZmavBX+uX+bc64NjXjOz6xsXdf2Wub8/NrN/DH4Gv2lm65Y594Q/z3GxzD1+1sxGan4WP7LMudvNbGfwd/LmxkVdv2Xu72s19/a6mT27zLmJ+A5XzN31ivELOBu4OHjfB7wKbF10zGXAf4861tO4x9eB7An2fwT4DmDAB4AfRh3zCu+zFXiTSn1tor8/4J8AFwMv1mz7I+Dm4P3NwOeWOK8fKAR/rg/er4/6fuq8vw8DbcH7zy11f8G+E/48x+W1zD1+FrjpJOe1AruBPNABPLf436Q4vJa6v0X7/wNwW5K/w5W+1OKPOXc/4O7/ELw/BrwCbIg2qoa7ErjPK54C1pnZ2VEHtQK/AOx298RPJuXu3wcOLtp8JfCV4P1XgKuWOPWfAY+5+0F3PwQ8BmwPLdAVWur+3P1Rd58NPj4FbGx4YKtome+wHpcAu9y94O7TwANUvvtYOdH9mZkBvwrc39CgYkKJP0HMbAtwEfDDJXb/jJk9Z2bfMbP/raGBnT4HHjWzH5nZDUvs3wDsrfm8j2T+8nMty/9Dk+Tvr+pMdz8QvH8TOHOJY9LyXf5zKr1QSznZz3Pc3Rg8zrh3mcc1afgOPwi85e6vLbM/6d/hCSnxJ4SZ9QJ/Dfxrdz+6aPc/UOk+/mngPwHfanR8p+nn3f1i4Argd83sn0Qd0Gozsw7go8B/W2J30r+/d/BKf2kqS4bM7FZgFvjqMock+ef5i8AQcCFwgEp3eBpdx4lb+0n+Dk9KiT8BzKydStL/qrt/Y/F+dz/q7mPB+4eBdjPLNjjMFXP3keDPt4FvUulKrDUCbKr5vDHYliRXAP/g7m8t3pH076/GW9VHMMGfby9xTKK/SzP7OPDLwG8Ev9y8Qx0/z7Hl7m+5+5y7zwN/ztKxJ/07bAOuAb623DFJ/g7rocQfc8GzqL8AXnH3P1nmmLOC4zCzS6h8r6ONi3LlzCxjZn3V91QGUL246LCHgN8KRvd/ADhS06WcFMu2MJL8/S3yEFAdpX898O0ljnkE+LCZrQ+6kT8cbIs9M9sO/AHwUXcfX+aYen6eY2vR2JmrWTr2p4HzzWww6Mm6lsp3nxT/FPhHd9+31M6kf4d1iXp0oV4nfgE/T6XL9Hng2eD1EeBTwKeCY24EXqIyuvYp4GejjvsU7i8fxP1ccA+3Bttr78+Au6mMJH4B2BZ13Kd4jxkqiXxtzbZEf39Ufok5AMxQecb728AA8DjwGvBdoD84dhvwX2rO/efAruD1iajv5RTubxeVZ9vVv4f/OTj2HODh4P2SP89xfC1zj38Z/B17nkoyP3vxPQafP0Klwmh3XO9xqfsLtn+5+nev5thEfocrfWnmPhERkSairn4REZEmosQvIiLSRJT4RUREmogSv4iISBNR4hcREWkiSvwisurMbEvtqmgiEh9K/CIiIk1EiV9EQmVmeTN7xszeF3UsIgJtUQcgIullZu+ismzrx939uajjERElfhEJT47KfP3XuPvLUQcjIhXq6heRsBwB9lBZb0JEYkItfhEJyzSVFd4eMbMxd/+vUQckIkr8IhIidy+b2S8DjwXJP0nLt4qkklbnExERaSJ6xi8iItJElPhFRESaiBK/iIhIE1HiFxERaSJK/CIiIk1EiV9ERKSJKPGLiIg0ESV+ERGRJvK/AC1SV1RpBeQuAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GRRbHIDQt6NR"
      },
      "source": [
        "### Turns out the best k is around the range of 10, which is the square root of the size of the train data. You can verify that by checking the value of X_train.shape[0]."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZhZvYumai12b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8b7dc772-0a39-467b-8f70-946683b828cd"
      },
      "source": [
        "X_train.shape[0]"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "100"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    }
  ]
}