C1 - Custom Models, Layers, and Loss Functions with TensorFlow/Week1/C1W1_Assignment.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ZL_6GK8qX35J"
   },
   "source": [
    "\n",
    "\n",
    "# Week 1: Multiple Output Models using the Keras Functional API\n",
    "\n",
    "Welcome to the first programming assignment of the course! Your task will be to use the Keras functional API to train a model to predict two outputs. For this lab, you will use the **[Wine Quality Dataset](https://archive.ics.uci.edu/ml/datasets/Wine+Quality)** from the **UCI machine learning repository**. It has separate datasets for red wine and white wine.\n",
    "\n",
    "Normally, the wines are classified into one of the quality ratings specified in the attributes. In this exercise, you will combine the two datasets to predict the wine quality and whether the wine is red or white solely from the attributes. \n",
    "\n",
    "You will model wine quality estimations as a regression problem and wine type detection as a binary classification problem.\n",
    "\n",
    "#### Please complete sections that are marked **(TODO)**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "obdcD6urYBY9"
   },
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "t8N3pcTQ5oQI"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow.keras.models import Model\n",
    "from tensorflow.keras.layers import Dense, Input\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import confusion_matrix\n",
    "import itertools\n",
    "\n",
    "import utils"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gQMERzWQYpgm"
   },
   "source": [
    "## Load Dataset\n",
    "\n",
    "\n",
    "You will now download the dataset from the [UCI Machine Learning Repository](https://archive.ics.uci.edu/ml/index.php). \n",
    "\n",
    "### Pre-process the white wine dataset (TODO)\n",
    "You will add a new column named `is_red` in your dataframe to indicate if the wine is white or red. \n",
    "- In the white wine dataset, you will fill the column `is_red` with  zeros (0)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "2qYAjKXCd4RH",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "cb9f284ba6255170fe04a309a407d72d",
     "grade": false,
     "grade_id": "cell-e5bfa0f152d9a21f",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "# URL of the white wine dataset\n",
    "URL = 'http://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-white.csv'\n",
    "\n",
    "# load the dataset from the URL\n",
    "white_df = pd.read_csv(URL, sep=\";\")\n",
    "\n",
    "# fill the `is_red` column with zeros.\n",
    "white_df[\"is_red\"] = 0\n",
    "\n",
    "# keep only the first of duplicate items\n",
    "white_df = white_df.drop_duplicates(keep='first')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "defe38d6ec58fd31cd67b89e46c4373f",
     "grade": true,
     "grade_id": "cell-30575e713b55fc51",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "# You can click `File -> Open` in the menu above and open the `utils.py` file \n",
    "# in case you want to inspect the unit tests being used for each graded function.\n",
    "\n",
    "utils.test_white_df(white_df)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "OQHK0ohBQRCk"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8.8\n",
      "9.1\n"
     ]
    }
   ],
   "source": [
    "print(white_df.alcohol[0])\n",
    "print(white_df.alcohol[100])\n",
    "\n",
    "# EXPECTED OUTPUT\n",
    "# 8.8\n",
    "# 9.1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pre-process the red wine dataset (TODO)\n",
    "- In the red wine dataset, you will fill in the column `is_red` with ones (1)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "8y3QxKwBed8v",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "be72275f78e2c2d0038dde9aa63f8d4f",
     "grade": false,
     "grade_id": "cell-e47a40f306593274",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "# URL of the red wine dataset\n",
    "URL = 'http://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv'\n",
    "\n",
    "# load the dataset from the URL\n",
    "red_df = pd.read_csv(URL, sep=\";\")\n",
    "\n",
    "# fill the `is_red` column with ones.\n",
    "red_df[\"is_red\"] = 1\n",
    "\n",
    "# keep only the first of duplicate items\n",
    "red_df = red_df.drop_duplicates(keep='first')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "d8e0c91b0fd668b63ba74a8f2f958b59",
     "grade": true,
     "grade_id": "cell-2a75937adcc0c25b",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_red_df(red_df)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "zsB3LUzNQpo_"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.4\n",
      "10.2\n"
     ]
    }
   ],
   "source": [
    "print(red_df.alcohol[0])\n",
    "print(red_df.alcohol[100])\n",
    "\n",
    "# EXPECTED OUTPUT\n",
    "# 9.4\n",
    "# 10.2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2G8B-NYuM6-f"
   },
   "source": [
    "### Concatenate the datasets\n",
    "\n",
    "Next, concatenate the red and white wine dataframes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "YpQrOjJbfN3m"
   },
   "outputs": [],
   "source": [
    "df = pd.concat([red_df, white_df], ignore_index=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Se2dTmThQyjb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.4\n",
      "9.5\n"
     ]
    }
   ],
   "source": [
    "print(df.alcohol[0])\n",
    "print(df.alcohol[100])\n",
    "\n",
    "# EXPECTED OUTPUT\n",
    "# 9.4\n",
    "# 9.5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "wx6y3rPpQv4k"
   },
   "outputs": [],
   "source": [
    "# NOTE: In a real-world scenario, you should shuffle the data.\n",
    "# YOU ARE NOT going to do that here because we want to test\n",
    "# with deterministic data. But if you want the code to do it,\n",
    "# it's in the commented line below:\n",
    "\n",
    "#df = df.iloc[np.random.permutation(len(df))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-EqIcbg5M_n1"
   },
   "source": [
    "This will chart the quality of the wines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "IsvK0-Sgy17C"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAQk0lEQVR4nO3df6zddX3H8edL6rAUmShyw1q2sqQxAs1QbhgbCbmMTasYwWUmJUxgc6khuOjWZCn7xy1LE5aM/ZANsg4cNSJNh5KSIU7CdudMRCyKKT8kdFKxlFEdiJQZtPjeH/eLXtrT9vbcH+fc83k+kpNz7ud8P9/zfvec+7rf+7nfc5qqQpLUhtcMugBJ0sIx9CWpIYa+JDXE0Jekhhj6ktSQJYMu4EhOOumkWrlyZV9zX3zxRZYtWza3BQ3IqPQyKn2AvQyrUelltn088MAD36uqNx84PvShv3LlSrZv397X3MnJSSYmJua2oAEZlV5GpQ+wl2E1Kr3Mto8k3+417vKOJDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1ZOjfkSsNqx1PPc+VG+7qa+6uay+a42qkmfFIX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1JAjhn6SU5P8R5JHkzyc5CPd+BuT3JPk8e76xGlzrkmyM8ljSd45bfzsJDu6+z6eJPPTliSpl5kc6e8H1lfVW4FzgauTnA5sAO6tqlXAvd3XdPetBc4A1gA3JDmm29eNwDpgVXdZM4e9SJKO4IihX1VPV9XXutsvAI8Cy4GLgc3dZpuBS7rbFwNbquqlqnoC2Amck+QU4ISq+nJVFfDJaXMkSQvgqNb0k6wE3gZ8BRirqqdh6gcDcHK32XLgO9Om7e7Glne3DxyXJC2QJTPdMMnxwGeAj1bVDw6zHN/rjjrMeK/HWsfUMhBjY2NMTk7OtMxX2bdvX99zh82o9DIqfQCMLYX1q/f3NXfY/g1G6XkZlV7mq48ZhX6S1zIV+LdW1We74WeSnFJVT3dLN3u78d3AqdOmrwD2dOMreowfpKo2AZsAxsfHa2JiYmbdHGBycpJ+5w6bUellVPoAuP7WbVy3Y8bHTa+y67KJuS1mlkbpeRmVXuarj5mcvRPgZuDRqvrraXfdCVzR3b4C2DZtfG2SY5OcxtQfbO/vloBeSHJut8/Lp82RJC2AmRymnAd8ANiR5MFu7E+Ba4GtST4IPAm8H6CqHk6yFXiEqTN/rq6ql7t5VwG3AEuBu7uLJGmBHDH0q+pL9F6PB7jwEHM2Aht7jG8HzjyaAiVJc8d35EpSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JD+vtfnaUDrNxw14y2W796P1cesO2uay+aj5Ik9eCRviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhwx9JN8IsneJA9NG/uzJE8lebC7vHvafdck2ZnksSTvnDZ+dpId3X0fT5K5b0eSdDgzOdK/BVjTY/xvquqs7vI5gCSnA2uBM7o5NyQ5ptv+RmAdsKq79NqnJGkeHTH0q+qLwLMz3N/FwJaqeqmqngB2AuckOQU4oaq+XFUFfBK4pN+iJUn9WTKLuR9OcjmwHVhfVc8By4H7pm2zuxv7cXf7wPGekqxj6rcCxsbGmJyc7KvAffv29T132Ax7L+tX75/RdmNLD952mPs6nF69zNSw9Tzsr6+jMSq9zFcf/Yb+jcBfANVdXwf8PtBrnb4OM95TVW0CNgGMj4/XxMREX0VOTk7S79xhM+y9XLnhrhltt371fq7b8eqX3a7LJuahovl3/a3bDuplpoat52F/fR2NUellvvro6+ydqnqmql6uqp8A/wSc0921Gzh12qYrgD3d+Ioe45KkBdRX6Hdr9K94H/DKmT13AmuTHJvkNKb+YHt/VT0NvJDk3O6sncuBbbOoW5LUhyP+bprkNmACOCnJbuBjwESSs5haotkFfAigqh5OshV4BNgPXF1VL3e7uoqpM4GWAnd3F0nSAjpi6FfVpT2Gbz7M9huBjT3GtwNnHlV1kqQ55TtyJakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhwx9JN8IsneJA9NG3tjknuSPN5dnzjtvmuS7EzyWJJ3Ths/O8mO7r6PJ8nctyNJOpyZHOnfAqw5YGwDcG9VrQLu7b4myenAWuCMbs4NSY7p5twIrANWdZcD9ylJmmdHDP2q+iLw7AHDFwObu9ubgUumjW+pqpeq6glgJ3BOklOAE6rqy1VVwCenzZEkLZB+1/THquppgO765G58OfCdadvt7saWd7cPHJckLaAlc7y/Xuv0dZjx3jtJ1jG1FMTY2BiTk5N9FbNv376+5w6bYe9l/er9M9pubOnB2w5zX4fTq5eZGraeh/31dTRGpZf56qPf0H8mySlV9XS3dLO3G98NnDptuxXAnm58RY/xnqpqE7AJYHx8vCYmJvoqcnJykn7nDpth7+XKDXfNaLv1q/dz3Y5Xv+x2XTYxDxXNv+tv3XZQLzM1bD0P++vraIxKL/PVR7/LO3cCV3S3rwC2TRtfm+TYJKcx9Qfb+7sloBeSnNudtXP5tDmSpAVyxMOUJLcBE8BJSXYDHwOuBbYm+SDwJPB+gKp6OMlW4BFgP3B1Vb3c7eoqps4EWgrc3V0kSQvoiKFfVZce4q4LD7H9RmBjj/HtwJlHVZ0kaU75jlxJaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhoy15+9I2nIrezxkRnrV++f0Udp7Lr2ovkoSQvII31JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1ZFahn2RXkh1JHkyyvRt7Y5J7kjzeXZ84bftrkuxM8liSd862eEnS0ZmLI/0Lquqsqhrvvt4A3FtVq4B7u69JcjqwFjgDWAPckOSYOXh8SdIMzcfyzsXA5u72ZuCSaeNbquqlqnoC2AmcMw+PL0k6hFRV/5OTJ4DngAL+sao2Jfl+Vb1h2jbPVdWJSf4euK+qPtWN3wzcXVW399jvOmAdwNjY2Nlbtmzpq759+/Zx/PHH9zV32Ax7Lzueen5G240thWd++Oqx1ct/fh4qmn97n33+oF5mapA993quej0vvSyG52rYv1dmarZ9XHDBBQ9MW4H5qSWzqgrOq6o9SU4G7knyzcNsmx5jPX/iVNUmYBPA+Ph4TUxM9FXc5OQk/c4dNsPey5Ub7prRdutX7+e6Ha9+2e26bGIeKpp/19+67aBeZmqQPfd6rno9L70shudq2L9XZmq++pjV8k5V7emu9wJ3MLVc80ySUwC6673d5ruBU6dNXwHsmc3jS5KOTt+hn2RZkte/cht4B/AQcCdwRbfZFcC27vadwNokxyY5DVgF3N/v40uSjt5slnfGgDuSvLKfT1fV55N8Fdia5IPAk8D7Aarq4SRbgUeA/cDVVfXyrKqXJB2VvkO/qr4F/EqP8f8FLjzEnI3Axn4fU5I0O74jV5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDen7P0bX/Fm54a6Dxtav3s+VPcYPtOvai+ajJEkjwiN9SWqIoS9JDTH0Jakhhr4kNcTQl6SGePaOpKHX64y2Q+l1pptntf2MR/qS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktSQBX9HbpI1wN8BxwA3VdW18/VYO556fkafQd+L7+CTNIoW9Eg/yTHAPwDvAk4HLk1y+kLWIEktW+gj/XOAnVX1LYAkW4CLgUcWuA5JmndH85lBB7plzbI5rORnUlXzsuOeD5b8DrCmqv6g+/oDwK9W1YcP2G4dsK778i3AY30+5EnA9/qcO2xGpZdR6QPsZViNSi+z7eOXqurNBw4u9JF+eowd9FOnqjYBm2b9YMn2qhqf7X6Gwaj0Mip9gL0Mq1HpZb76WOizd3YDp077egWwZ4FrkKRmLXTofxVYleS0JD8HrAXuXOAaJKlZC7q8U1X7k3wY+DemTtn8RFU9PI8POesloiEyKr2MSh9gL8NqVHqZlz4W9A+5kqTB8h25ktQQQ1+SGjJyoZ/kdUnuT/KNJA8n+fNB1zRbSY5J8vUk/zroWmYjya4kO5I8mGT7oOuZjSRvSHJ7km8meTTJrw26pqOV5C3dc/HK5QdJPjrouvqV5I+67/mHktyW5HWDrqlfST7S9fHwXD8nI7emnyTAsqral+S1wJeAj1TVfQMurW9J/hgYB06oqvcMup5+JdkFjFfVon/jTJLNwH9V1U3dmWjHVdX3B11Xv7qPSHmKqTdLfnvQ9RytJMuZ+l4/vap+mGQr8LmqumWwlR29JGcCW5j6BIMfAZ8Hrqqqx+di/yN3pF9T9nVfvra7LNqfbElWABcBNw26Fk1JcgJwPnAzQFX9aDEHfudC4L8XY+BPswRYmmQJcByL9z1AbwXuq6r/q6r9wH8C75urnY9c6MNPl0MeBPYC91TVVwZd0yz8LfAnwE8GXcgcKOALSR7oPmpjsfpl4LvAP3fLbjclmZ8PSlk4a4HbBl1Ev6rqKeCvgCeBp4Hnq+oLg62qbw8B5yd5U5LjgHfz6je1zspIhn5VvVxVZzH1jt9zul+XFp0k7wH2VtUDg65ljpxXVW9n6lNWr05y/qAL6tMS4O3AjVX1NuBFYMNgS+pftzz1XuBfBl1Lv5KcyNSHN54G/AKwLMnvDraq/lTVo8BfAvcwtbTzDWD/XO1/JEP/Fd2v3JPAmgGX0q/zgPd2a+FbgN9I8qnBltS/qtrTXe8F7mBqzXIx2g3snvYb5O1M/RBYrN4FfK2qnhl0IbPwm8ATVfXdqvox8Fng1wdcU9+q6uaqentVnQ88C8zJej6MYOgneXOSN3S3lzL1YvjmYKvqT1VdU1UrqmolU79+/3tVLcqjlyTLkrz+ldvAO5j6NXbRqar/Ab6T5C3d0IUs7o8Hv5RFvLTTeRI4N8lx3ckcFwKPDrimviU5ubv+ReC3mcPnZ8H/56wFcAqwuTsb4TXA1qpa1Kc6jogx4I6p70eWAJ+uqs8PtqRZ+UPg1m5p5FvA7w24nr50a8a/BXxo0LXMRlV9JcntwNeYWgr5Oov74xg+k+RNwI+Bq6vqubna8cidsilJOrSRW96RJB2aoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5Ia8v9wPc08ifqwpQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df['quality'].hist(bins=20);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Nut1rmYLzf-p"
   },
   "source": [
    "### Imbalanced data (TODO)\n",
    "You can see from the plot above that the wine quality dataset is imbalanced. \n",
    "- Since there are very few observations with quality equal to 3, 4, 8 and 9, you can drop these observations from your dataset. \n",
    "- You can do this by removing data belonging to all classes except those > 4 and < 8."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "doH9_-gnf3sz",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "d9ba9fc3a3ca02ccc567be33652b80fe",
     "grade": false,
     "grade_id": "cell-6a3e9db696f6827b",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "# get data with wine quality greater than 4 and less than 8\n",
    "df = df[(df['quality'] > 4) & (df['quality'] < 8)] # YOUR CODE HERE) & (df['quality'] < YOUR CODE HERE )]\n",
    "\n",
    "# reset index and drop the old one\n",
    "df = df.reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "281e1d86a4803560ed5892cd7eda4c01",
     "grade": true,
     "grade_id": "cell-aed3da719d4682c7",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_df_drop(df)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "xNR1iAlMRPXO"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.4\n",
      "10.9\n"
     ]
    }
   ],
   "source": [
    "print(df.alcohol[0])\n",
    "print(df.alcohol[100])\n",
    "\n",
    "# EXPECTED OUTPUT\n",
    "# 9.4\n",
    "# 10.9"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "cwhuRpnVRTzG"
   },
   "source": [
    "You can plot again to see the new range of data and quality"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "857ygzZiLgGg"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAQaUlEQVR4nO3df6zddX3H8edrRQkW+ZXOO9IywaQx48dE2zDUzN2GRapuK/vDpIQIZCydBpOZkEWYyTRZmuAfbAk4yDpxQGQ2zB8rEdlGGDdmU8TiwPJDtEonpUinIFBiMLD3/jjfbsfLae85p/ec2/p5PpKT8z2f7/fz/b6/Xz687rmfe863qSokSW34laUuQJI0PYa+JDXE0Jekhhj6ktQQQ1+SGnLUUhewkBUrVtSpp546Vt8XX3yR5cuXL25Bi8C6RmNdo7Gu0fyy1nX//ff/uKp+9VUrquqwfqxZs6bGdc8994zdd5KsazTWNRrrGs0va13A9hqQqU7vSFJDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQw772zBIh6sdTz7HpVfeMVbfXVe/b5GrkYbjO31JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDFgz9JKckuSfJo0keTvKnXftJSe5K8r3u+cS+Plcl2ZnksSTn97WvSbKjW3dtkkzmtCRJgwzzTv9l4Iqq+g3gXODyJKcDVwJ3V9Vq4O7uNd26jcAZwHrg+iTLun3dAGwCVneP9Yt4LpKkBSwY+lX1VFV9q1t+AXgUWAlsAG7uNrsZuKBb3gBsraqXqupxYCdwTpKTgeOq6utVVcAtfX0kSVMw0px+klOBtwLfAGaq6ino/WAA3tBtthJ4oq/b7q5tZbc8v12SNCVHDbthkmOBLwAfqarnDzIdP2hFHaR90LE20ZsGYmZmhrm5uWHL/AX79u0bu+8kWddoDte6Zo6BK856eay+kzyfw/V6WddoJlXXUKGf5DX0Av/Wqvpi1/x0kpOr6qlu6mZv174bOKWv+ypgT9e+akD7q1TVFmALwNq1a2t2dna4s5lnbm6OcftOknWN5nCt67pbt3HNjqHfN/2CXRfNLm4xfQ7X62Vdo5lUXcN8eifAjcCjVfVXfatuBy7pli8BtvW1b0xydJLT6P3B9r5uCuiFJOd2+7y4r48kaQqGeZvyTuADwI4kD3Rtfw5cDdyW5DLgh8D7Aarq4SS3AY/Q++TP5VX1StfvQ8BNwDHAnd1DkjQlC4Z+Vf07g+fjAc47QJ/NwOYB7duBM0cpUJK0ePxGriQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDVkvH/V+Qix48nnuPTKO8bqu+vq9y1yNZK09HynL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIQuGfpLPJNmb5KG+tk8keTLJA93jvX3rrkqyM8ljSc7va1+TZEe37tokWfzTkSQdzDDv9G8C1g9o/+uqOrt7fAUgyenARuCMrs/1SZZ1298AbAJWd49B+5QkTdCCoV9VXwWeGXJ/G4CtVfVSVT0O7ATOSXIycFxVfb2qCrgFuGDcoiVJ40kvgxfYKDkV+HJVndm9/gRwKfA8sB24oqqeTfIp4N6q+my33Y3AncAu4Oqq+t2u/beBj1bV7x3geJvo/VbAzMzMmq1bt451cnufeY6nfzZWV85aefx4HYewb98+jj322Intf1zWNRrH12isazSHWte6devur6q189uPGnN/NwB/CVT3fA3wR8Cgefo6SPtAVbUF2AKwdu3amp2dHavI627dxjU7xjvFXReNd8xhzM3NMe45TZJ1jcbxNRrrGs2k6hrr0ztV9XRVvVJV/wP8HXBOt2o3cErfpquAPV37qgHtkqQpGiv0uzn6/f4Q2P/JntuBjUmOTnIavT/Y3ldVTwEvJDm3+9TOxcC2Q6hbkjSGBX83TfI5YBZYkWQ38HFgNsnZ9KZodgF/AlBVDye5DXgEeBm4vKpe6Xb1IXqfBDqG3jz/nYt5IpKkhS0Y+lV14YDmGw+y/WZg84D27cCZI1UnSVpUfiNXkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqyIKhn+QzSfYmeaiv7aQkdyX5Xvd8Yt+6q5LsTPJYkvP72tck2dGtuzZJFv90JEkHM8w7/ZuA9fPargTurqrVwN3da5KcDmwEzuj6XJ9kWdfnBmATsLp7zN+nJGnCFgz9qvoq8My85g3Azd3yzcAFfe1bq+qlqnoc2Amck+Rk4Liq+npVFXBLXx9J0pSMO6c/U1VPAXTPb+jaVwJP9G23u2tb2S3Pb5ckTdFRi7y/QfP0dZD2wTtJNtGbCmJmZoa5ubmxipk5Bq446+Wx+o57zGHs27dvovsfl3WNxvE1GusazaTqGjf0n05yclU91U3d7O3adwOn9G23CtjTta8a0D5QVW0BtgCsXbu2Zmdnxyryulu3cc2O8U5x10XjHXMYc3NzjHtOk2Rdo3F8jca6RjOpusad3rkduKRbvgTY1te+McnRSU6j9wfb+7opoBeSnNt9aufivj6SpClZ8G1Kks8Bs8CKJLuBjwNXA7cluQz4IfB+gKp6OMltwCPAy8DlVfVKt6sP0fsk0DHAnd1DkjRFC4Z+VV14gFXnHWD7zcDmAe3bgTNHqk6StKj8Rq4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ1Z7HvvSJI6p155x9h9b1q/fBEr+X++05ekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDin0k+xKsiPJA0m2d20nJbkryfe65xP7tr8qyc4kjyU5/1CLlySNZjHe6a+rqrOram33+krg7qpaDdzdvSbJ6cBG4AxgPXB9kmWLcHxJ0pAmMb2zAbi5W74ZuKCvfWtVvVRVjwM7gXMmcHxJ0gGkqsbvnDwOPAsU8LdVtSXJT6vqhL5tnq2qE5N8Cri3qj7btd8I3FlVnx+w303AJoCZmZk1W7duHau+vc88x9M/G6srZ608fryOQ9i3bx/HHnvsxPY/LusajeNrNC3WtePJ58bue9rxyw6prnXr1t3fNwPzf44ae48976yqPUneANyV5DsH2TYD2gb+xKmqLcAWgLVr19bs7OxYxV136zau2THeKe66aLxjDmNubo5xz2mSrGs0jq/RtFjXpVfeMXbfm9Yvn0hdhzS9U1V7uue9wJfoTdc8neRkgO55b7f5buCUvu6rgD2HcnxJ0mjGDv0ky5O8fv8y8G7gIeB24JJus0uAbd3y7cDGJEcnOQ1YDdw37vElSaM7lOmdGeBLSfbv5x+q6p+TfBO4LcllwA+B9wNU1cNJbgMeAV4GLq+qVw6peknSSMYO/ar6AfCWAe0/Ac47QJ/NwOZxjylJOjR+I1eSGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDZl66CdZn+SxJDuTXDnt40tSy6Ya+kmWAX8DvAc4HbgwyenTrEGSWjbtd/rnADur6gdV9XNgK7BhyjVIUrOOmvLxVgJP9L3eDfzW/I2SbAI2dS/3JXlszOOtAH48Tsd8cswjDmfsuibMukbj+BqNdY1g3ScPua43DmqcduhnQFu9qqFqC7DlkA+WbK+qtYe6n8VmXaOxrtFY12haq2va0zu7gVP6Xq8C9ky5Bklq1rRD/5vA6iSnJXktsBG4fco1SFKzpjq9U1UvJ/kw8C/AMuAzVfXwBA95yFNEE2Jdo7Gu0VjXaJqqK1WvmlKXJP2S8hu5ktQQQ1+SGnLEhn6SXUl2JHkgyfYB65Pk2u52D99O8ra+dRO7FcQQdV3U1fPtJF9L8pZh+064rtkkz3XrH0jyF33rlvJ6/VlfTQ8leSXJScP0PcS6Tkjy+STfSfJokrfPW79U42uhupZqfC1U11KNr4Xqmvr4SvLmvmM+kOT5JB+Zt83kxldVHZEPYBew4iDr3wvcSe+7AecC3+jalwHfB94EvBZ4EDh9inW9AzixW37P/rqG6TvhumaBLw9oX9LrNW/b3wf+bUrX62bgj7vl1wInHCbja6G6lmp8LVTXUo2vg9a1VONr3vn/CHjjtMbXEftOfwgbgFuq517ghCQns8S3gqiqr1XVs93Le+l9V+FwdjjdOuNC4HOTPkiS44B3ATcCVNXPq+qn8zab+vgapq6lGF9DXq8DWdLrNc9Uxtc85wHfr6r/mtc+sfF1JId+Af+a5P70btsw36BbPqw8SPu06up3Gb2f5uP0nURdb0/yYJI7k5zRtR0W1yvJ64D1wBdG7TuGNwH/Dfx9kv9M8ukky+dtsxTja5i6+k1rfA1b17TH19DXa8rjq99GBv+gmdj4OpJD/51V9TZ6v8JenuRd89Yf6JYPQ90KYoJ19YpL1tH7n/Kjo/adUF3fovcr5luA64B/2l/qgH1N/XrR+9X7P6rqmTH6juoo4G3ADVX1VuBFYP7c6VKMr2Hq6hU33fE1TF1LMb6Gvl5Md3wBkN4XVP8A+MdBqwe0Lcr4OmJDv6r2dM97gS/R+7Wn34Fu+TDRW0EMURdJfhP4NLChqn4ySt9J1VVVz1fVvm75K8BrkqzgMLhenVe9I5rg9doN7K6qb3SvP08vPOZvM+3xNUxdSzG+FqxricbXUNerM83xtd97gG9V1dMD1k1sfB2RoZ9keZLX718G3g08NG+z24GLu7+Cnws8V1VPMcFbQQxTV5JfB74IfKCqvjviOU2yrl9Lkm75HHpj4ycs8fXq1h0P/A6wbdS+46iqHwFPJHlz13Qe8Mi8zaY+voapaynG15B1TX18Dfnfcerjq8/B/oYwufG1WH+FnuaD3lzdg93jYeBjXfsHgQ92y6H3D7Z8H9gBrO3r/17gu926j025rk8DzwIPdI/tB+s7xbo+3K17kN4fAN9xOFyv7vWlwNZh+i5ibWcD24Fv05uKOHGpx9eQdU19fA1Z19TH1zB1LeH4eh29H3rH97VNZXx5GwZJasgROb0jSRqPoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5Ia8r889HO6f1QuqQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df['quality'].hist(bins=20);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "n3k0vqSsp84t"
   },
   "source": [
    "### Train Test Split (TODO)\n",
    "\n",
    "Next, you can split the datasets into training, test and validation datasets.\n",
    "- The data frame should be split 80:20 into `train` and `test` sets.\n",
    "- The resulting `train` should then be split 80:20 into `train` and `val` sets.\n",
    "- The `train_test_split` parameter `test_size` takes a float value that ranges between 0. and 1, and represents the proportion of the dataset that is allocated to the test set.  The rest of the data is allocated to the training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "PAVIf2-fgRVY",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "7f5738f4fb51d65adc9a8acbdf2b9970",
     "grade": false,
     "grade_id": "cell-91946cadf745206b",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "# Please do not change the random_state parameter. This is needed for grading.\n",
    "\n",
    "# split df into 80:20 train and test sets\n",
    "train, test = train_test_split(df, test_size = 0.2, random_state = 1) # YOUR CODE HERE, random_state = 1)\n",
    "                               \n",
    "# split train into 80:20 train and val sets\n",
    "train, val = train_test_split(train, test_size = 0.2, random_state = 1) # YOUR CODE HERE, random_state = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "editable": false,
    "id": "57h9LcEzRWpk",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "42adbe9e66efac7c7a5f8cd73ac92f22",
     "grade": true,
     "grade_id": "cell-64b8b38cd0b965f6",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_data_sizes(train.size, test.size, val.size)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "RwTNu4KFqG-K"
   },
   "source": [
    "Here's where you can explore the training stats. You can pop the labels 'is_red' and 'quality' from the data as these will be used as the labels\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Y_afyhhHM6WQ"
   },
   "outputs": [],
   "source": [
    "train_stats = train.describe()\n",
    "train_stats.pop('is_red')\n",
    "train_stats.pop('quality')\n",
    "train_stats = train_stats.transpose()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ahvbYm4fNqSt"
   },
   "source": [
    "Explore the training stats!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "n_gAtPjZ0otF"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>fixed acidity</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>7.221616</td>\n",
       "      <td>1.325297</td>\n",
       "      <td>3.80000</td>\n",
       "      <td>6.40000</td>\n",
       "      <td>7.00000</td>\n",
       "      <td>7.7000</td>\n",
       "      <td>15.60000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>volatile acidity</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>0.338929</td>\n",
       "      <td>0.162476</td>\n",
       "      <td>0.08000</td>\n",
       "      <td>0.23000</td>\n",
       "      <td>0.29000</td>\n",
       "      <td>0.4000</td>\n",
       "      <td>1.24000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>citric acid</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>0.321569</td>\n",
       "      <td>0.147970</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.25000</td>\n",
       "      <td>0.31000</td>\n",
       "      <td>0.4000</td>\n",
       "      <td>1.66000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>residual sugar</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>5.155911</td>\n",
       "      <td>4.639632</td>\n",
       "      <td>0.60000</td>\n",
       "      <td>1.80000</td>\n",
       "      <td>2.80000</td>\n",
       "      <td>7.6500</td>\n",
       "      <td>65.80000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>chlorides</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>0.056976</td>\n",
       "      <td>0.036802</td>\n",
       "      <td>0.01200</td>\n",
       "      <td>0.03800</td>\n",
       "      <td>0.04700</td>\n",
       "      <td>0.0660</td>\n",
       "      <td>0.61100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>30.388590</td>\n",
       "      <td>17.236784</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>17.00000</td>\n",
       "      <td>28.00000</td>\n",
       "      <td>41.0000</td>\n",
       "      <td>131.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>115.062282</td>\n",
       "      <td>56.706617</td>\n",
       "      <td>6.00000</td>\n",
       "      <td>75.00000</td>\n",
       "      <td>117.00000</td>\n",
       "      <td>156.0000</td>\n",
       "      <td>344.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>density</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>0.994633</td>\n",
       "      <td>0.003005</td>\n",
       "      <td>0.98711</td>\n",
       "      <td>0.99232</td>\n",
       "      <td>0.99481</td>\n",
       "      <td>0.9968</td>\n",
       "      <td>1.03898</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>pH</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>3.223201</td>\n",
       "      <td>0.161272</td>\n",
       "      <td>2.72000</td>\n",
       "      <td>3.11000</td>\n",
       "      <td>3.21000</td>\n",
       "      <td>3.3300</td>\n",
       "      <td>4.01000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sulphates</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>0.534051</td>\n",
       "      <td>0.149149</td>\n",
       "      <td>0.22000</td>\n",
       "      <td>0.43000</td>\n",
       "      <td>0.51000</td>\n",
       "      <td>0.6000</td>\n",
       "      <td>1.95000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>alcohol</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>10.504466</td>\n",
       "      <td>1.154654</td>\n",
       "      <td>8.50000</td>\n",
       "      <td>9.50000</td>\n",
       "      <td>10.30000</td>\n",
       "      <td>11.3000</td>\n",
       "      <td>14.00000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                       count        mean        std      min       25%  \\\n",
       "fixed acidity         3155.0    7.221616   1.325297  3.80000   6.40000   \n",
       "volatile acidity      3155.0    0.338929   0.162476  0.08000   0.23000   \n",
       "citric acid           3155.0    0.321569   0.147970  0.00000   0.25000   \n",
       "residual sugar        3155.0    5.155911   4.639632  0.60000   1.80000   \n",
       "chlorides             3155.0    0.056976   0.036802  0.01200   0.03800   \n",
       "free sulfur dioxide   3155.0   30.388590  17.236784  1.00000  17.00000   \n",
       "total sulfur dioxide  3155.0  115.062282  56.706617  6.00000  75.00000   \n",
       "density               3155.0    0.994633   0.003005  0.98711   0.99232   \n",
       "pH                    3155.0    3.223201   0.161272  2.72000   3.11000   \n",
       "sulphates             3155.0    0.534051   0.149149  0.22000   0.43000   \n",
       "alcohol               3155.0   10.504466   1.154654  8.50000   9.50000   \n",
       "\n",
       "                            50%       75%        max  \n",
       "fixed acidity           7.00000    7.7000   15.60000  \n",
       "volatile acidity        0.29000    0.4000    1.24000  \n",
       "citric acid             0.31000    0.4000    1.66000  \n",
       "residual sugar          2.80000    7.6500   65.80000  \n",
       "chlorides               0.04700    0.0660    0.61100  \n",
       "free sulfur dioxide    28.00000   41.0000  131.00000  \n",
       "total sulfur dioxide  117.00000  156.0000  344.00000  \n",
       "density                 0.99481    0.9968    1.03898  \n",
       "pH                      3.21000    3.3300    4.01000  \n",
       "sulphates               0.51000    0.6000    1.95000  \n",
       "alcohol                10.30000   11.3000   14.00000  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_stats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bGPvt9jir_HC"
   },
   "source": [
    "### Get the labels (TODO)\n",
    "\n",
    "The features and labels are currently in the same dataframe.\n",
    "- You will want to store the label columns `is_red` and `quality` separately from the feature columns.  \n",
    "- The following function, `format_output`, gets these two columns from the dataframe (it's given to you).\n",
    "- `format_output` also formats the data into numpy arrays. \n",
    "- Please use the `format_output` and apply it to the `train`, `val` and `test` sets to get dataframes for the labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Z_fs14XQqZVP"
   },
   "outputs": [],
   "source": [
    "def format_output(data):\n",
    "    is_red = data.pop('is_red')\n",
    "    is_red = np.array(is_red)\n",
    "    quality = data.pop('quality')\n",
    "    quality = np.array(quality)\n",
    "    return (quality, is_red)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "8L3ZZe1fQicm",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "7a86809e54895a816434c48dc903f55d",
     "grade": false,
     "grade_id": "cell-5c30fa2c2a354b0f",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "# format the output of the train set\n",
    "train_Y = format_output(train) # YOUR CODE HERE)\n",
    "\n",
    "# format the output of the val set\n",
    "val_Y = format_output(val)# YOUR CODE HERE)\n",
    "    \n",
    "# format the output of the test set\n",
    "test_Y = format_output(test)# YOUR CODE HERE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "359cabbafaed14ec9bbc1e57a7b6f32c",
     "grade": true,
     "grade_id": "cell-4977d8befb80f56b",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_format_output(df, train_Y, val_Y, test_Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that after you get the labels, the `train`, `val` and `test` dataframes no longer contain the label columns, and contain just the feature columns.\n",
    "- This is because you used `.pop` in the `format_output` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fixed acidity</th>\n",
       "      <th>volatile acidity</th>\n",
       "      <th>citric acid</th>\n",
       "      <th>residual sugar</th>\n",
       "      <th>chlorides</th>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <th>density</th>\n",
       "      <th>pH</th>\n",
       "      <th>sulphates</th>\n",
       "      <th>alcohol</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>225</th>\n",
       "      <td>7.5</td>\n",
       "      <td>0.65</td>\n",
       "      <td>0.18</td>\n",
       "      <td>7.0</td>\n",
       "      <td>0.088</td>\n",
       "      <td>27.0</td>\n",
       "      <td>94.0</td>\n",
       "      <td>0.99915</td>\n",
       "      <td>3.38</td>\n",
       "      <td>0.77</td>\n",
       "      <td>9.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3557</th>\n",
       "      <td>6.3</td>\n",
       "      <td>0.27</td>\n",
       "      <td>0.29</td>\n",
       "      <td>12.2</td>\n",
       "      <td>0.044</td>\n",
       "      <td>59.0</td>\n",
       "      <td>196.0</td>\n",
       "      <td>0.99782</td>\n",
       "      <td>3.14</td>\n",
       "      <td>0.40</td>\n",
       "      <td>8.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3825</th>\n",
       "      <td>8.8</td>\n",
       "      <td>0.27</td>\n",
       "      <td>0.25</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.024</td>\n",
       "      <td>52.0</td>\n",
       "      <td>99.0</td>\n",
       "      <td>0.99250</td>\n",
       "      <td>2.87</td>\n",
       "      <td>0.49</td>\n",
       "      <td>11.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1740</th>\n",
       "      <td>6.4</td>\n",
       "      <td>0.45</td>\n",
       "      <td>0.07</td>\n",
       "      <td>1.1</td>\n",
       "      <td>0.030</td>\n",
       "      <td>10.0</td>\n",
       "      <td>131.0</td>\n",
       "      <td>0.99050</td>\n",
       "      <td>2.97</td>\n",
       "      <td>0.28</td>\n",
       "      <td>10.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1221</th>\n",
       "      <td>7.2</td>\n",
       "      <td>0.53</td>\n",
       "      <td>0.13</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.058</td>\n",
       "      <td>18.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>0.99573</td>\n",
       "      <td>3.21</td>\n",
       "      <td>0.68</td>\n",
       "      <td>9.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      fixed acidity  volatile acidity  citric acid  residual sugar  chlorides  \\\n",
       "225             7.5              0.65         0.18             7.0      0.088   \n",
       "3557            6.3              0.27         0.29            12.2      0.044   \n",
       "3825            8.8              0.27         0.25             5.0      0.024   \n",
       "1740            6.4              0.45         0.07             1.1      0.030   \n",
       "1221            7.2              0.53         0.13             2.0      0.058   \n",
       "\n",
       "      free sulfur dioxide  total sulfur dioxide  density    pH  sulphates  \\\n",
       "225                  27.0                  94.0  0.99915  3.38       0.77   \n",
       "3557                 59.0                 196.0  0.99782  3.14       0.40   \n",
       "3825                 52.0                  99.0  0.99250  2.87       0.49   \n",
       "1740                 10.0                 131.0  0.99050  2.97       0.28   \n",
       "1221                 18.0                  22.0  0.99573  3.21       0.68   \n",
       "\n",
       "      alcohol  \n",
       "225       9.4  \n",
       "3557      8.8  \n",
       "3825     11.4  \n",
       "1740     10.8  \n",
       "1221      9.9  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hEdbrruAsN1D"
   },
   "source": [
    "### Normalize the data (TODO)\n",
    "\n",
    "Next, you can normalize the data, x, using the formula:\n",
    "$$x_{norm} = \\frac{x - \\mu}{\\sigma}$$\n",
    "- The `norm` function is defined for you.\n",
    "- Please apply the `norm` function to normalize the dataframes that contains the feature columns of `train`, `val` and `test` sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WWiZPAHCLjUs"
   },
   "outputs": [],
   "source": [
    "def norm(x):\n",
    "    return (x - train_stats['mean']) / train_stats['std']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "JEaOi2I2Lk69",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "6bc0cdcb563d192f271067aa3373ff32",
     "grade": false,
     "grade_id": "cell-d8416d975c371095",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "# normalize the train set\n",
    "norm_train_X = norm(train) # YOUR CODE HERE)\n",
    "    \n",
    "# normalize the val set\n",
    "norm_val_X = norm(val) # YOUR CODE HERE)\n",
    "    \n",
    "# normalize the test set\n",
    "norm_test_X = norm(test) # YOUR CODE HERE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "4f567db45bf40191601780379cc100b8",
     "grade": true,
     "grade_id": "cell-97fad979d157529b",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_norm(norm_train_X, norm_val_X, norm_test_X, train, val, test)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hzykDwQhsaPO"
   },
   "source": [
    "## Define the Model (TODO)\n",
    "\n",
    "Define the model using the functional API. The base model will be 2 `Dense` layers of 128 neurons each, and have the `'relu'` activation.\n",
    "- Check out the documentation for [tf.keras.layers.Dense](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "Rhcns3oTFkM6",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "74b031247e569526552bf13a034a1c07",
     "grade": false,
     "grade_id": "cell-73fceedad1fe351c",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "def base_model(inputs):\n",
    "    \n",
    "    # connect a Dense layer with 128 neurons and a relu activation\n",
    "    x = Dense(128, activation = 'relu', input_shape = (norm_train_X.shape[1], ))(inputs) # YOUR CODE HERE\n",
    "    \n",
    "    # connect another Dense layer with 128 neurons and a relu activation\n",
    "    x = Dense(128, activation = 'relu')(x) # YOUR CODE HERE\n",
    "    return x\n",
    "  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "9255924b3def80f679616e4c851a43e1",
     "grade": true,
     "grade_id": "cell-54f742a133353d75",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_base_model(base_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xem_fcVws6Kz"
   },
   "source": [
    "# Define output layers of the model (TODO)\n",
    "\n",
    "You will add output layers to the base model. \n",
    "- The model will need two outputs.\n",
    "\n",
    "One output layer will predict wine quality, which is a numeric value.\n",
    "- Define a `Dense` layer with 1 neuron.\n",
    "- Since this is a regression output, the activation can be left as its default value `None`.\n",
    "\n",
    "The other output layer will predict the wine type, which is either red `1` or not red `0` (white).\n",
    "- Define a `Dense` layer with 1 neuron.\n",
    "- Since there are two possible categories, you can use a sigmoid activation for binary classification.\n",
    "\n",
    "Define the `Model`\n",
    "- Define the `Model` object, and set the following parameters:\n",
    "  - `inputs`: pass in the inputs to the model as a list.\n",
    "  - `outputs`: pass in a list of the outputs that you just defined: wine quality, then wine type.\n",
    "  - **Note**: please list the wine quality before wine type in the outputs, as this will affect the calculated loss if you choose the other order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "n5UGF8PMVLPt",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "76d35b90d20cdcbb22986cd8211057de",
     "grade": false,
     "grade_id": "cell-19e285f482f021fb",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "def final_model(inputs):\n",
    "    \n",
    "    # get the base model\n",
    "    x = base_model(inputs)\n",
    "\n",
    "    # connect the output Dense layer for regression\n",
    "    wine_quality = Dense(units='1', name='wine_quality')(x)\n",
    "\n",
    "    # connect the output Dense layer for classification. this will use a sigmoid activation.\n",
    "    wine_type = Dense(units='1', activation='sigmoid', name='wine_type')(x) # YOUR CODE HERE, name='wine_type')(x)\n",
    "\n",
    "    # define the model using the input and output layers\n",
    "    model = Model(inputs=inputs, outputs=[wine_quality, wine_type])# YOUR CODE HERE, outputs=# YOUR CODE HERE)\n",
    "\n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "89cbf89d8ab5e2e59ecf7f63f517520a",
     "grade": true,
     "grade_id": "cell-40d050f855c817d1",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_final_model(final_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "5R0BMTsltZyu"
   },
   "source": [
    "## Compiling the Model\n",
    "\n",
    "Next, compile the model. When setting the loss parameter of `model.compile`, you're setting the loss for each of the two outputs (wine quality and wine type).\n",
    "\n",
    "To set more than one loss, use a dictionary of key-value pairs.\n",
    "- You can look at the docs for the losses [here](https://www.tensorflow.org/api_docs/python/tf/keras/losses#functions).\n",
    "    - **Note**: For the desired spelling, please look at the \"Functions\" section of the documentation and not the \"classes\" section on that same page.\n",
    "- wine_type: Since you will be performing binary classification on wine type, you should use the binary crossentropy loss function for it.  Please pass this in as a string.  \n",
    "  - **Hint**, this should be all lowercase.  In the documentation, you'll see this under the \"Functions\" section, not the \"Classes\" section.\n",
    "- wine_quality: since this is a regression output, use the mean squared error.  Please pass it in as a string, all lowercase.\n",
    "  - **Hint**: You may notice that there are two aliases for mean squared error.  Please use the shorter name.\n",
    "\n",
    "\n",
    "You will also set the metric for each of the two outputs.  Again, to set metrics for two or more outputs, use a dictionary with key value pairs.\n",
    "- The metrics documentation is linked [here](https://www.tensorflow.org/api_docs/python/tf/keras/metrics).\n",
    "- For the wine type, please set it to accuracy as a string, all lowercase.\n",
    "- For wine quality, please use the root mean squared error.  Instead of a string, you'll set it to an instance of the class [RootMeanSquaredError](https://www.tensorflow.org/api_docs/python/tf/keras/metrics/RootMeanSquaredError), which belongs to the tf.keras.metrics module.\n",
    "\n",
    "**Note**: If you see the error message \n",
    ">Exception: wine quality loss function is incorrect.\n",
    "\n",
    "- Please also check your other losses and metrics, as the error may be caused by the other three key-value pairs and not the wine quality loss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "LK11duUbUjmh",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "22f45067ca69eb2ccadb43874dbcc27b",
     "grade": false,
     "grade_id": "cell-81afdc4dcca51d5e",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "inputs = tf.keras.layers.Input(shape=(11,))\n",
    "rms = tf.keras.optimizers.RMSprop(lr=0.0001)\n",
    "model = final_model(inputs)\n",
    "\n",
    "model.compile(optimizer=rms, \n",
    "              loss = {'wine_type' : 'binary_crossentropy', # YOUR CODE HERE,\n",
    "                      'wine_quality' : 'mse' # YOUR CODE HERE\n",
    "                     },\n",
    "              metrics = {'wine_type' : 'accuracy', # YOUR CODE HERE,\n",
    "                         'wine_quality': tf.keras.metrics.RootMeanSquaredError() # YOUR CODE HERE\n",
    "                       }\n",
    "             )\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "88e02238ea5e456ff65e835cc8158054",
     "grade": true,
     "grade_id": "cell-2eeeba02391c4632",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_model_compile(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "90MpAMpWuKm-"
   },
   "source": [
    "## Training the Model\n",
    "\n",
    "Fit the model to the training inputs and outputs. \n",
    "- Check the documentation for [model.fit](https://www.tensorflow.org/api_docs/python/tf/keras/Model#fit).\n",
    "- Remember to use the normalized training set as inputs. \n",
    "- For the validation data, please use the normalized validation set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "_eiZkle4XwiY",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "2ca7664be03bfd6fd3651ae44d17b793",
     "grade": false,
     "grade_id": "cell-0bb56262896f6680",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 3155 samples, validate on 789 samples\n",
      "Epoch 1/180\n",
      "3155/3155 [==============================] - 1s 414us/sample - loss: 27.7838 - wine_quality_loss: 27.0298 - wine_type_loss: 0.7231 - wine_quality_root_mean_squared_error: 5.2020 - wine_type_accuracy: 0.3547 - val_loss: 20.9903 - val_wine_quality_loss: 20.2886 - val_wine_type_loss: 0.7086 - val_wine_quality_root_mean_squared_error: 4.5035 - val_wine_type_accuracy: 0.4233\n",
      "Epoch 2/180\n",
      "3155/3155 [==============================] - 0s 99us/sample - loss: 15.0427 - wine_quality_loss: 14.3328 - wine_type_loss: 0.6790 - wine_quality_root_mean_squared_error: 3.7899 - wine_type_accuracy: 0.5423 - val_loss: 9.5886 - val_wine_quality_loss: 8.9759 - val_wine_type_loss: 0.6402 - val_wine_quality_root_mean_squared_error: 2.9914 - val_wine_type_accuracy: 0.7022\n",
      "Epoch 3/180\n",
      "3155/3155 [==============================] - 0s 97us/sample - loss: 6.1585 - wine_quality_loss: 5.5631 - wine_type_loss: 0.5836 - wine_quality_root_mean_squared_error: 2.3610 - wine_type_accuracy: 0.8342 - val_loss: 3.6504 - val_wine_quality_loss: 3.1745 - val_wine_type_loss: 0.5262 - val_wine_quality_root_mean_squared_error: 1.7675 - val_wine_type_accuracy: 0.8682\n",
      "Epoch 4/180\n",
      "3155/3155 [==============================] - 0s 95us/sample - loss: 3.1691 - wine_quality_loss: 2.7150 - wine_type_loss: 0.4557 - wine_quality_root_mean_squared_error: 1.6472 - wine_type_accuracy: 0.8830 - val_loss: 2.7211 - val_wine_quality_loss: 2.3630 - val_wine_type_loss: 0.4004 - val_wine_quality_root_mean_squared_error: 1.5233 - val_wine_type_accuracy: 0.8986\n",
      "Epoch 5/180\n",
      "3155/3155 [==============================] - 0s 93us/sample - loss: 2.5557 - wine_quality_loss: 2.2072 - wine_type_loss: 0.3444 - wine_quality_root_mean_squared_error: 1.4869 - wine_type_accuracy: 0.9293 - val_loss: 2.3388 - val_wine_quality_loss: 2.0634 - val_wine_type_loss: 0.3062 - val_wine_quality_root_mean_squared_error: 1.4257 - val_wine_type_accuracy: 0.9632\n",
      "Epoch 6/180\n",
      "3155/3155 [==============================] - 0s 93us/sample - loss: 2.1920 - wine_quality_loss: 1.9259 - wine_type_loss: 0.2622 - wine_quality_root_mean_squared_error: 1.3891 - wine_type_accuracy: 0.9655 - val_loss: 2.0307 - val_wine_quality_loss: 1.8214 - val_wine_type_loss: 0.2344 - val_wine_quality_root_mean_squared_error: 1.3402 - val_wine_type_accuracy: 0.9772\n",
      "Epoch 7/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 1.9397 - wine_quality_loss: 1.7363 - wine_type_loss: 0.2010 - wine_quality_root_mean_squared_error: 1.3185 - wine_type_accuracy: 0.9775 - val_loss: 1.8270 - val_wine_quality_loss: 1.6638 - val_wine_type_loss: 0.1818 - val_wine_quality_root_mean_squared_error: 1.2826 - val_wine_type_accuracy: 0.9886\n",
      "Epoch 8/180\n",
      "3155/3155 [==============================] - 0s 77us/sample - loss: 1.7412 - wine_quality_loss: 1.5892 - wine_type_loss: 0.1569 - wine_quality_root_mean_squared_error: 1.2587 - wine_type_accuracy: 0.9851 - val_loss: 1.6510 - val_wine_quality_loss: 1.5229 - val_wine_type_loss: 0.1429 - val_wine_quality_root_mean_squared_error: 1.2279 - val_wine_type_accuracy: 0.9899\n",
      "Epoch 9/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 1.5966 - wine_quality_loss: 1.4682 - wine_type_loss: 0.1254 - wine_quality_root_mean_squared_error: 1.2128 - wine_type_accuracy: 0.9883 - val_loss: 1.5179 - val_wine_quality_loss: 1.4129 - val_wine_type_loss: 0.1163 - val_wine_quality_root_mean_squared_error: 1.1838 - val_wine_type_accuracy: 0.9899\n",
      "Epoch 10/180\n",
      "3155/3155 [==============================] - 0s 92us/sample - loss: 1.4682 - wine_quality_loss: 1.3656 - wine_type_loss: 0.1030 - wine_quality_root_mean_squared_error: 1.1685 - wine_type_accuracy: 0.9892 - val_loss: 1.3932 - val_wine_quality_loss: 1.3058 - val_wine_type_loss: 0.0963 - val_wine_quality_root_mean_squared_error: 1.1387 - val_wine_type_accuracy: 0.9899\n",
      "Epoch 11/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 1.3590 - wine_quality_loss: 1.2715 - wine_type_loss: 0.0866 - wine_quality_root_mean_squared_error: 1.1279 - wine_type_accuracy: 0.9895 - val_loss: 1.2973 - val_wine_quality_loss: 1.2217 - val_wine_type_loss: 0.0822 - val_wine_quality_root_mean_squared_error: 1.1022 - val_wine_type_accuracy: 0.9899\n",
      "Epoch 12/180\n",
      "3155/3155 [==============================] - 0s 76us/sample - loss: 1.2661 - wine_quality_loss: 1.1921 - wine_type_loss: 0.0749 - wine_quality_root_mean_squared_error: 1.0914 - wine_type_accuracy: 0.9902 - val_loss: 1.2168 - val_wine_quality_loss: 1.1507 - val_wine_type_loss: 0.0719 - val_wine_quality_root_mean_squared_error: 1.0699 - val_wine_type_accuracy: 0.9899\n",
      "Epoch 13/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 1.1904 - wine_quality_loss: 1.1222 - wine_type_loss: 0.0661 - wine_quality_root_mean_squared_error: 1.0603 - wine_type_accuracy: 0.9905 - val_loss: 1.1478 - val_wine_quality_loss: 1.0886 - val_wine_type_loss: 0.0639 - val_wine_quality_root_mean_squared_error: 1.0410 - val_wine_type_accuracy: 0.9899\n",
      "Epoch 14/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 1.1188 - wine_quality_loss: 1.0583 - wine_type_loss: 0.0593 - wine_quality_root_mean_squared_error: 1.0293 - wine_type_accuracy: 0.9911 - val_loss: 1.0737 - val_wine_quality_loss: 1.0192 - val_wine_type_loss: 0.0581 - val_wine_quality_root_mean_squared_error: 1.0076 - val_wine_type_accuracy: 0.9886\n",
      "Epoch 15/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 1.0518 - wine_quality_loss: 0.9956 - wine_type_loss: 0.0543 - wine_quality_root_mean_squared_error: 0.9987 - wine_type_accuracy: 0.9918 - val_loss: 1.0155 - val_wine_quality_loss: 0.9646 - val_wine_type_loss: 0.0533 - val_wine_quality_root_mean_squared_error: 0.9807 - val_wine_type_accuracy: 0.9886\n",
      "Epoch 16/180\n",
      "3155/3155 [==============================] - 0s 76us/sample - loss: 0.9921 - wine_quality_loss: 0.9403 - wine_type_loss: 0.0504 - wine_quality_root_mean_squared_error: 0.9704 - wine_type_accuracy: 0.9918 - val_loss: 0.9609 - val_wine_quality_loss: 0.9126 - val_wine_type_loss: 0.0498 - val_wine_quality_root_mean_squared_error: 0.9543 - val_wine_type_accuracy: 0.9886\n",
      "Epoch 17/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.9379 - wine_quality_loss: 0.8901 - wine_type_loss: 0.0472 - wine_quality_root_mean_squared_error: 0.9437 - wine_type_accuracy: 0.9921 - val_loss: 0.9088 - val_wine_quality_loss: 0.8630 - val_wine_type_loss: 0.0469 - val_wine_quality_root_mean_squared_error: 0.9282 - val_wine_type_accuracy: 0.9899\n",
      "Epoch 18/180\n",
      "3155/3155 [==============================] - 0s 92us/sample - loss: 0.8869 - wine_quality_loss: 0.8495 - wine_type_loss: 0.0450 - wine_quality_root_mean_squared_error: 0.9177 - wine_type_accuracy: 0.9924 - val_loss: 0.8508 - val_wine_quality_loss: 0.8072 - val_wine_type_loss: 0.0449 - val_wine_quality_root_mean_squared_error: 0.8975 - val_wine_type_accuracy: 0.9899\n",
      "Epoch 19/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.8439 - wine_quality_loss: 0.8010 - wine_type_loss: 0.0427 - wine_quality_root_mean_squared_error: 0.8950 - wine_type_accuracy: 0.9924 - val_loss: 0.8042 - val_wine_quality_loss: 0.7619 - val_wine_type_loss: 0.0430 - val_wine_quality_root_mean_squared_error: 0.8723 - val_wine_type_accuracy: 0.9899\n",
      "Epoch 20/180\n",
      "3155/3155 [==============================] - 0s 76us/sample - loss: 0.8006 - wine_quality_loss: 0.7588 - wine_type_loss: 0.0409 - wine_quality_root_mean_squared_error: 0.8715 - wine_type_accuracy: 0.9927 - val_loss: 0.7652 - val_wine_quality_loss: 0.7241 - val_wine_type_loss: 0.0416 - val_wine_quality_root_mean_squared_error: 0.8504 - val_wine_type_accuracy: 0.9899\n",
      "Epoch 21/180\n",
      "3155/3155 [==============================] - 0s 93us/sample - loss: 0.7621 - wine_quality_loss: 0.7217 - wine_type_loss: 0.0396 - wine_quality_root_mean_squared_error: 0.8500 - wine_type_accuracy: 0.9927 - val_loss: 0.7357 - val_wine_quality_loss: 0.6952 - val_wine_type_loss: 0.0403 - val_wine_quality_root_mean_squared_error: 0.8337 - val_wine_type_accuracy: 0.9899\n",
      "Epoch 22/180\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3155/3155 [==============================] - 0s 93us/sample - loss: 0.7253 - wine_quality_loss: 0.6874 - wine_type_loss: 0.0384 - wine_quality_root_mean_squared_error: 0.8287 - wine_type_accuracy: 0.9930 - val_loss: 0.6942 - val_wine_quality_loss: 0.6547 - val_wine_type_loss: 0.0391 - val_wine_quality_root_mean_squared_error: 0.8091 - val_wine_type_accuracy: 0.9911\n",
      "Epoch 23/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.6895 - wine_quality_loss: 0.6525 - wine_type_loss: 0.0374 - wine_quality_root_mean_squared_error: 0.8075 - wine_type_accuracy: 0.9930 - val_loss: 0.6639 - val_wine_quality_loss: 0.6252 - val_wine_type_loss: 0.0380 - val_wine_quality_root_mean_squared_error: 0.7909 - val_wine_type_accuracy: 0.9911\n",
      "Epoch 24/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.6587 - wine_quality_loss: 0.6218 - wine_type_loss: 0.0366 - wine_quality_root_mean_squared_error: 0.7887 - wine_type_accuracy: 0.9930 - val_loss: 0.6319 - val_wine_quality_loss: 0.5936 - val_wine_type_loss: 0.0373 - val_wine_quality_root_mean_squared_error: 0.7709 - val_wine_type_accuracy: 0.9911\n",
      "Epoch 25/180\n",
      "3155/3155 [==============================] - 0s 92us/sample - loss: 0.6279 - wine_quality_loss: 0.5910 - wine_type_loss: 0.0356 - wine_quality_root_mean_squared_error: 0.7695 - wine_type_accuracy: 0.9930 - val_loss: 0.6051 - val_wine_quality_loss: 0.5675 - val_wine_type_loss: 0.0368 - val_wine_quality_root_mean_squared_error: 0.7536 - val_wine_type_accuracy: 0.9911\n",
      "Epoch 26/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.6049 - wine_quality_loss: 0.5688 - wine_type_loss: 0.0350 - wine_quality_root_mean_squared_error: 0.7549 - wine_type_accuracy: 0.9930 - val_loss: 0.5829 - val_wine_quality_loss: 0.5459 - val_wine_type_loss: 0.0361 - val_wine_quality_root_mean_squared_error: 0.7392 - val_wine_type_accuracy: 0.9911\n",
      "Epoch 27/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.5810 - wine_quality_loss: 0.5455 - wine_type_loss: 0.0344 - wine_quality_root_mean_squared_error: 0.7393 - wine_type_accuracy: 0.9930 - val_loss: 0.5585 - val_wine_quality_loss: 0.5217 - val_wine_type_loss: 0.0356 - val_wine_quality_root_mean_squared_error: 0.7229 - val_wine_type_accuracy: 0.9911\n",
      "Epoch 28/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.5573 - wine_quality_loss: 0.5220 - wine_type_loss: 0.0339 - wine_quality_root_mean_squared_error: 0.7235 - wine_type_accuracy: 0.9933 - val_loss: 0.5378 - val_wine_quality_loss: 0.5012 - val_wine_type_loss: 0.0350 - val_wine_quality_root_mean_squared_error: 0.7088 - val_wine_type_accuracy: 0.9911\n",
      "Epoch 29/180\n",
      "3155/3155 [==============================] - 0s 93us/sample - loss: 0.5380 - wine_quality_loss: 0.5037 - wine_type_loss: 0.0334 - wine_quality_root_mean_squared_error: 0.7103 - wine_type_accuracy: 0.9933 - val_loss: 0.5137 - val_wine_quality_loss: 0.4778 - val_wine_type_loss: 0.0346 - val_wine_quality_root_mean_squared_error: 0.6919 - val_wine_type_accuracy: 0.9911\n",
      "Epoch 30/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.5187 - wine_quality_loss: 0.4851 - wine_type_loss: 0.0329 - wine_quality_root_mean_squared_error: 0.6969 - wine_type_accuracy: 0.9940 - val_loss: 0.5008 - val_wine_quality_loss: 0.4650 - val_wine_type_loss: 0.0343 - val_wine_quality_root_mean_squared_error: 0.6827 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 31/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.5020 - wine_quality_loss: 0.4693 - wine_type_loss: 0.0324 - wine_quality_root_mean_squared_error: 0.6851 - wine_type_accuracy: 0.9940 - val_loss: 0.4853 - val_wine_quality_loss: 0.4497 - val_wine_type_loss: 0.0340 - val_wine_quality_root_mean_squared_error: 0.6715 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 32/180\n",
      "3155/3155 [==============================] - 0s 76us/sample - loss: 0.4866 - wine_quality_loss: 0.4550 - wine_type_loss: 0.0321 - wine_quality_root_mean_squared_error: 0.6741 - wine_type_accuracy: 0.9943 - val_loss: 0.4697 - val_wine_quality_loss: 0.4344 - val_wine_type_loss: 0.0337 - val_wine_quality_root_mean_squared_error: 0.6601 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 33/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.4702 - wine_quality_loss: 0.4385 - wine_type_loss: 0.0317 - wine_quality_root_mean_squared_error: 0.6621 - wine_type_accuracy: 0.9943 - val_loss: 0.4652 - val_wine_quality_loss: 0.4303 - val_wine_type_loss: 0.0334 - val_wine_quality_root_mean_squared_error: 0.6568 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 34/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.4585 - wine_quality_loss: 0.4274 - wine_type_loss: 0.0315 - wine_quality_root_mean_squared_error: 0.6535 - wine_type_accuracy: 0.9940 - val_loss: 0.4436 - val_wine_quality_loss: 0.4090 - val_wine_type_loss: 0.0331 - val_wine_quality_root_mean_squared_error: 0.6404 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 35/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.4457 - wine_quality_loss: 0.4143 - wine_type_loss: 0.0311 - wine_quality_root_mean_squared_error: 0.6438 - wine_type_accuracy: 0.9943 - val_loss: 0.4326 - val_wine_quality_loss: 0.3985 - val_wine_type_loss: 0.0329 - val_wine_quality_root_mean_squared_error: 0.6319 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 36/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.4362 - wine_quality_loss: 0.4043 - wine_type_loss: 0.0308 - wine_quality_root_mean_squared_error: 0.6367 - wine_type_accuracy: 0.9943 - val_loss: 0.4232 - val_wine_quality_loss: 0.3888 - val_wine_type_loss: 0.0329 - val_wine_quality_root_mean_squared_error: 0.6244 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 37/180\n",
      "3155/3155 [==============================] - 0s 93us/sample - loss: 0.4267 - wine_quality_loss: 0.3954 - wine_type_loss: 0.0306 - wine_quality_root_mean_squared_error: 0.6293 - wine_type_accuracy: 0.9937 - val_loss: 0.4144 - val_wine_quality_loss: 0.3805 - val_wine_type_loss: 0.0325 - val_wine_quality_root_mean_squared_error: 0.6176 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 38/180\n",
      "3155/3155 [==============================] - 0s 92us/sample - loss: 0.4188 - wine_quality_loss: 0.3882 - wine_type_loss: 0.0303 - wine_quality_root_mean_squared_error: 0.6232 - wine_type_accuracy: 0.9937 - val_loss: 0.4053 - val_wine_quality_loss: 0.3717 - val_wine_type_loss: 0.0325 - val_wine_quality_root_mean_squared_error: 0.6102 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 39/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.4107 - wine_quality_loss: 0.3805 - wine_type_loss: 0.0300 - wine_quality_root_mean_squared_error: 0.6169 - wine_type_accuracy: 0.9940 - val_loss: 0.3996 - val_wine_quality_loss: 0.3657 - val_wine_type_loss: 0.0324 - val_wine_quality_root_mean_squared_error: 0.6057 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 40/180\n",
      "3155/3155 [==============================] - 0s 77us/sample - loss: 0.4033 - wine_quality_loss: 0.3736 - wine_type_loss: 0.0298 - wine_quality_root_mean_squared_error: 0.6111 - wine_type_accuracy: 0.9937 - val_loss: 0.4032 - val_wine_quality_loss: 0.3693 - val_wine_type_loss: 0.0321 - val_wine_quality_root_mean_squared_error: 0.6089 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 41/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.3961 - wine_quality_loss: 0.3664 - wine_type_loss: 0.0296 - wine_quality_root_mean_squared_error: 0.6053 - wine_type_accuracy: 0.9943 - val_loss: 0.3927 - val_wine_quality_loss: 0.3591 - val_wine_type_loss: 0.0321 - val_wine_quality_root_mean_squared_error: 0.6002 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 42/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3916 - wine_quality_loss: 0.3625 - wine_type_loss: 0.0294 - wine_quality_root_mean_squared_error: 0.6017 - wine_type_accuracy: 0.9937 - val_loss: 0.3845 - val_wine_quality_loss: 0.3512 - val_wine_type_loss: 0.0320 - val_wine_quality_root_mean_squared_error: 0.5934 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 43/180\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3155/3155 [==============================] - 0s 77us/sample - loss: 0.3862 - wine_quality_loss: 0.3578 - wine_type_loss: 0.0291 - wine_quality_root_mean_squared_error: 0.5976 - wine_type_accuracy: 0.9937 - val_loss: 0.3805 - val_wine_quality_loss: 0.3473 - val_wine_type_loss: 0.0320 - val_wine_quality_root_mean_squared_error: 0.5900 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 44/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3819 - wine_quality_loss: 0.3526 - wine_type_loss: 0.0290 - wine_quality_root_mean_squared_error: 0.5940 - wine_type_accuracy: 0.9937 - val_loss: 0.3797 - val_wine_quality_loss: 0.3466 - val_wine_type_loss: 0.0318 - val_wine_quality_root_mean_squared_error: 0.5895 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 45/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3776 - wine_quality_loss: 0.3495 - wine_type_loss: 0.0287 - wine_quality_root_mean_squared_error: 0.5906 - wine_type_accuracy: 0.9937 - val_loss: 0.3754 - val_wine_quality_loss: 0.3423 - val_wine_type_loss: 0.0317 - val_wine_quality_root_mean_squared_error: 0.5860 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 46/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.3725 - wine_quality_loss: 0.3446 - wine_type_loss: 0.0285 - wine_quality_root_mean_squared_error: 0.5864 - wine_type_accuracy: 0.9943 - val_loss: 0.3714 - val_wine_quality_loss: 0.3383 - val_wine_type_loss: 0.0316 - val_wine_quality_root_mean_squared_error: 0.5826 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 47/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.3697 - wine_quality_loss: 0.3418 - wine_type_loss: 0.0283 - wine_quality_root_mean_squared_error: 0.5842 - wine_type_accuracy: 0.9943 - val_loss: 0.3691 - val_wine_quality_loss: 0.3364 - val_wine_type_loss: 0.0316 - val_wine_quality_root_mean_squared_error: 0.5806 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 48/180\n",
      "3155/3155 [==============================] - 0s 92us/sample - loss: 0.3657 - wine_quality_loss: 0.3378 - wine_type_loss: 0.0281 - wine_quality_root_mean_squared_error: 0.5810 - wine_type_accuracy: 0.9943 - val_loss: 0.3668 - val_wine_quality_loss: 0.3343 - val_wine_type_loss: 0.0314 - val_wine_quality_root_mean_squared_error: 0.5788 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 49/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.3634 - wine_quality_loss: 0.3354 - wine_type_loss: 0.0278 - wine_quality_root_mean_squared_error: 0.5792 - wine_type_accuracy: 0.9949 - val_loss: 0.3645 - val_wine_quality_loss: 0.3318 - val_wine_type_loss: 0.0314 - val_wine_quality_root_mean_squared_error: 0.5768 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 50/180\n",
      "3155/3155 [==============================] - 0s 76us/sample - loss: 0.3602 - wine_quality_loss: 0.3329 - wine_type_loss: 0.0277 - wine_quality_root_mean_squared_error: 0.5766 - wine_type_accuracy: 0.9946 - val_loss: 0.3618 - val_wine_quality_loss: 0.3292 - val_wine_type_loss: 0.0314 - val_wine_quality_root_mean_squared_error: 0.5745 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 51/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3582 - wine_quality_loss: 0.3300 - wine_type_loss: 0.0275 - wine_quality_root_mean_squared_error: 0.5749 - wine_type_accuracy: 0.9949 - val_loss: 0.3611 - val_wine_quality_loss: 0.3286 - val_wine_type_loss: 0.0314 - val_wine_quality_root_mean_squared_error: 0.5739 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 52/180\n",
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.3552 - wine_quality_loss: 0.3275 - wine_type_loss: 0.0273 - wine_quality_root_mean_squared_error: 0.5725 - wine_type_accuracy: 0.9946 - val_loss: 0.3617 - val_wine_quality_loss: 0.3290 - val_wine_type_loss: 0.0314 - val_wine_quality_root_mean_squared_error: 0.5744 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 53/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.3525 - wine_quality_loss: 0.3251 - wine_type_loss: 0.0272 - wine_quality_root_mean_squared_error: 0.5703 - wine_type_accuracy: 0.9943 - val_loss: 0.3579 - val_wine_quality_loss: 0.3251 - val_wine_type_loss: 0.0314 - val_wine_quality_root_mean_squared_error: 0.5710 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 54/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3512 - wine_quality_loss: 0.3242 - wine_type_loss: 0.0270 - wine_quality_root_mean_squared_error: 0.5693 - wine_type_accuracy: 0.9943 - val_loss: 0.3541 - val_wine_quality_loss: 0.3218 - val_wine_type_loss: 0.0311 - val_wine_quality_root_mean_squared_error: 0.5680 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 55/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3500 - wine_quality_loss: 0.3228 - wine_type_loss: 0.0268 - wine_quality_root_mean_squared_error: 0.5684 - wine_type_accuracy: 0.9949 - val_loss: 0.3536 - val_wine_quality_loss: 0.3214 - val_wine_type_loss: 0.0310 - val_wine_quality_root_mean_squared_error: 0.5676 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 56/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.3466 - wine_quality_loss: 0.3200 - wine_type_loss: 0.0267 - wine_quality_root_mean_squared_error: 0.5656 - wine_type_accuracy: 0.9946 - val_loss: 0.3605 - val_wine_quality_loss: 0.3283 - val_wine_type_loss: 0.0309 - val_wine_quality_root_mean_squared_error: 0.5738 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 57/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3456 - wine_quality_loss: 0.3183 - wine_type_loss: 0.0265 - wine_quality_root_mean_squared_error: 0.5648 - wine_type_accuracy: 0.9949 - val_loss: 0.3543 - val_wine_quality_loss: 0.3226 - val_wine_type_loss: 0.0307 - val_wine_quality_root_mean_squared_error: 0.5685 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 58/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.3441 - wine_quality_loss: 0.3177 - wine_type_loss: 0.0264 - wine_quality_root_mean_squared_error: 0.5637 - wine_type_accuracy: 0.9946 - val_loss: 0.3476 - val_wine_quality_loss: 0.3158 - val_wine_type_loss: 0.0307 - val_wine_quality_root_mean_squared_error: 0.5626 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 59/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.3426 - wine_quality_loss: 0.3164 - wine_type_loss: 0.0261 - wine_quality_root_mean_squared_error: 0.5625 - wine_type_accuracy: 0.9949 - val_loss: 0.3528 - val_wine_quality_loss: 0.3211 - val_wine_type_loss: 0.0306 - val_wine_quality_root_mean_squared_error: 0.5674 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 60/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3410 - wine_quality_loss: 0.3144 - wine_type_loss: 0.0260 - wine_quality_root_mean_squared_error: 0.5612 - wine_type_accuracy: 0.9949 - val_loss: 0.3492 - val_wine_quality_loss: 0.3173 - val_wine_type_loss: 0.0306 - val_wine_quality_root_mean_squared_error: 0.5641 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 61/180\n",
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.3396 - wine_quality_loss: 0.3136 - wine_type_loss: 0.0258 - wine_quality_root_mean_squared_error: 0.5600 - wine_type_accuracy: 0.9949 - val_loss: 0.3477 - val_wine_quality_loss: 0.3162 - val_wine_type_loss: 0.0305 - val_wine_quality_root_mean_squared_error: 0.5629 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 62/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3384 - wine_quality_loss: 0.3134 - wine_type_loss: 0.0257 - wine_quality_root_mean_squared_error: 0.5592 - wine_type_accuracy: 0.9949 - val_loss: 0.3491 - val_wine_quality_loss: 0.3174 - val_wine_type_loss: 0.0304 - val_wine_quality_root_mean_squared_error: 0.5642 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 63/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.3370 - wine_quality_loss: 0.3113 - wine_type_loss: 0.0255 - wine_quality_root_mean_squared_error: 0.5580 - wine_type_accuracy: 0.9949 - val_loss: 0.3509 - val_wine_quality_loss: 0.3192 - val_wine_type_loss: 0.0304 - val_wine_quality_root_mean_squared_error: 0.5658 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 64/180\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.3358 - wine_quality_loss: 0.3111 - wine_type_loss: 0.0255 - wine_quality_root_mean_squared_error: 0.5571 - wine_type_accuracy: 0.9949 - val_loss: 0.3581 - val_wine_quality_loss: 0.3265 - val_wine_type_loss: 0.0303 - val_wine_quality_root_mean_squared_error: 0.5722 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 65/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3340 - wine_quality_loss: 0.3096 - wine_type_loss: 0.0252 - wine_quality_root_mean_squared_error: 0.5556 - wine_type_accuracy: 0.9949 - val_loss: 0.3505 - val_wine_quality_loss: 0.3191 - val_wine_type_loss: 0.0302 - val_wine_quality_root_mean_squared_error: 0.5656 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 66/180\n",
      "3155/3155 [==============================] - 0s 86us/sample - loss: 0.3339 - wine_quality_loss: 0.3082 - wine_type_loss: 0.0251 - wine_quality_root_mean_squared_error: 0.5557 - wine_type_accuracy: 0.9949 - val_loss: 0.3444 - val_wine_quality_loss: 0.3130 - val_wine_type_loss: 0.0301 - val_wine_quality_root_mean_squared_error: 0.5603 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 67/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.3311 - wine_quality_loss: 0.3056 - wine_type_loss: 0.0249 - wine_quality_root_mean_squared_error: 0.5533 - wine_type_accuracy: 0.9952 - val_loss: 0.3465 - val_wine_quality_loss: 0.3151 - val_wine_type_loss: 0.0300 - val_wine_quality_root_mean_squared_error: 0.5622 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 68/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3322 - wine_quality_loss: 0.3068 - wine_type_loss: 0.0248 - wine_quality_root_mean_squared_error: 0.5544 - wine_type_accuracy: 0.9949 - val_loss: 0.3474 - val_wine_quality_loss: 0.3165 - val_wine_type_loss: 0.0301 - val_wine_quality_root_mean_squared_error: 0.5630 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 69/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.3309 - wine_quality_loss: 0.3063 - wine_type_loss: 0.0247 - wine_quality_root_mean_squared_error: 0.5533 - wine_type_accuracy: 0.9949 - val_loss: 0.3410 - val_wine_quality_loss: 0.3099 - val_wine_type_loss: 0.0299 - val_wine_quality_root_mean_squared_error: 0.5573 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 70/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.3295 - wine_quality_loss: 0.3054 - wine_type_loss: 0.0246 - wine_quality_root_mean_squared_error: 0.5521 - wine_type_accuracy: 0.9949 - val_loss: 0.3415 - val_wine_quality_loss: 0.3106 - val_wine_type_loss: 0.0300 - val_wine_quality_root_mean_squared_error: 0.5578 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 71/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3286 - wine_quality_loss: 0.3044 - wine_type_loss: 0.0249 - wine_quality_root_mean_squared_error: 0.5515 - wine_type_accuracy: 0.9949 - val_loss: 0.3422 - val_wine_quality_loss: 0.3112 - val_wine_type_loss: 0.0300 - val_wine_quality_root_mean_squared_error: 0.5584 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 72/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.3281 - wine_quality_loss: 0.3036 - wine_type_loss: 0.0242 - wine_quality_root_mean_squared_error: 0.5511 - wine_type_accuracy: 0.9949 - val_loss: 0.3431 - val_wine_quality_loss: 0.3121 - val_wine_type_loss: 0.0298 - val_wine_quality_root_mean_squared_error: 0.5594 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 73/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.3274 - wine_quality_loss: 0.3030 - wine_type_loss: 0.0244 - wine_quality_root_mean_squared_error: 0.5507 - wine_type_accuracy: 0.9949 - val_loss: 0.3423 - val_wine_quality_loss: 0.3114 - val_wine_type_loss: 0.0301 - val_wine_quality_root_mean_squared_error: 0.5584 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 74/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.3265 - wine_quality_loss: 0.3022 - wine_type_loss: 0.0241 - wine_quality_root_mean_squared_error: 0.5499 - wine_type_accuracy: 0.9952 - val_loss: 0.3419 - val_wine_quality_loss: 0.3113 - val_wine_type_loss: 0.0298 - val_wine_quality_root_mean_squared_error: 0.5584 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 75/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.3250 - wine_quality_loss: 0.3007 - wine_type_loss: 0.0239 - wine_quality_root_mean_squared_error: 0.5487 - wine_type_accuracy: 0.9952 - val_loss: 0.3511 - val_wine_quality_loss: 0.3201 - val_wine_type_loss: 0.0297 - val_wine_quality_root_mean_squared_error: 0.5666 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 76/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3244 - wine_quality_loss: 0.3007 - wine_type_loss: 0.0238 - wine_quality_root_mean_squared_error: 0.5482 - wine_type_accuracy: 0.9956 - val_loss: 0.3408 - val_wine_quality_loss: 0.3102 - val_wine_type_loss: 0.0297 - val_wine_quality_root_mean_squared_error: 0.5575 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 77/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.3229 - wine_quality_loss: 0.2993 - wine_type_loss: 0.0237 - wine_quality_root_mean_squared_error: 0.5469 - wine_type_accuracy: 0.9952 - val_loss: 0.3421 - val_wine_quality_loss: 0.3115 - val_wine_type_loss: 0.0295 - val_wine_quality_root_mean_squared_error: 0.5588 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 78/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.3230 - wine_quality_loss: 0.2991 - wine_type_loss: 0.0235 - wine_quality_root_mean_squared_error: 0.5471 - wine_type_accuracy: 0.9952 - val_loss: 0.3450 - val_wine_quality_loss: 0.3146 - val_wine_type_loss: 0.0295 - val_wine_quality_root_mean_squared_error: 0.5613 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 79/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3224 - wine_quality_loss: 0.2985 - wine_type_loss: 0.0234 - wine_quality_root_mean_squared_error: 0.5467 - wine_type_accuracy: 0.9956 - val_loss: 0.3413 - val_wine_quality_loss: 0.3107 - val_wine_type_loss: 0.0295 - val_wine_quality_root_mean_squared_error: 0.5581 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 80/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.3211 - wine_quality_loss: 0.2977 - wine_type_loss: 0.0233 - wine_quality_root_mean_squared_error: 0.5456 - wine_type_accuracy: 0.9956 - val_loss: 0.3452 - val_wine_quality_loss: 0.3145 - val_wine_type_loss: 0.0295 - val_wine_quality_root_mean_squared_error: 0.5615 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 81/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.3215 - wine_quality_loss: 0.2982 - wine_type_loss: 0.0233 - wine_quality_root_mean_squared_error: 0.5460 - wine_type_accuracy: 0.9952 - val_loss: 0.3482 - val_wine_quality_loss: 0.3175 - val_wine_type_loss: 0.0294 - val_wine_quality_root_mean_squared_error: 0.5643 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 82/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.3195 - wine_quality_loss: 0.2971 - wine_type_loss: 0.0232 - wine_quality_root_mean_squared_error: 0.5443 - wine_type_accuracy: 0.9956 - val_loss: 0.3444 - val_wine_quality_loss: 0.3139 - val_wine_type_loss: 0.0292 - val_wine_quality_root_mean_squared_error: 0.5610 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 83/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.3193 - wine_quality_loss: 0.2968 - wine_type_loss: 0.0231 - wine_quality_root_mean_squared_error: 0.5443 - wine_type_accuracy: 0.9956 - val_loss: 0.3413 - val_wine_quality_loss: 0.3112 - val_wine_type_loss: 0.0291 - val_wine_quality_root_mean_squared_error: 0.5584 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 84/180\n",
      "3155/3155 [==============================] - 0s 92us/sample - loss: 0.3191 - wine_quality_loss: 0.2960 - wine_type_loss: 0.0235 - wine_quality_root_mean_squared_error: 0.5442 - wine_type_accuracy: 0.9956 - val_loss: 0.3411 - val_wine_quality_loss: 0.3115 - val_wine_type_loss: 0.0290 - val_wine_quality_root_mean_squared_error: 0.5583 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 85/180\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3155/3155 [==============================] - 0s 94us/sample - loss: 0.3186 - wine_quality_loss: 0.2956 - wine_type_loss: 0.0228 - wine_quality_root_mean_squared_error: 0.5437 - wine_type_accuracy: 0.9956 - val_loss: 0.3387 - val_wine_quality_loss: 0.3089 - val_wine_type_loss: 0.0289 - val_wine_quality_root_mean_squared_error: 0.5563 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 86/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.3183 - wine_quality_loss: 0.2957 - wine_type_loss: 0.0227 - wine_quality_root_mean_squared_error: 0.5436 - wine_type_accuracy: 0.9956 - val_loss: 0.3368 - val_wine_quality_loss: 0.3069 - val_wine_type_loss: 0.0289 - val_wine_quality_root_mean_squared_error: 0.5546 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 87/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.3164 - wine_quality_loss: 0.2939 - wine_type_loss: 0.0226 - wine_quality_root_mean_squared_error: 0.5420 - wine_type_accuracy: 0.9956 - val_loss: 0.3409 - val_wine_quality_loss: 0.3110 - val_wine_type_loss: 0.0290 - val_wine_quality_root_mean_squared_error: 0.5582 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 88/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3164 - wine_quality_loss: 0.2938 - wine_type_loss: 0.0227 - wine_quality_root_mean_squared_error: 0.5420 - wine_type_accuracy: 0.9952 - val_loss: 0.3397 - val_wine_quality_loss: 0.3099 - val_wine_type_loss: 0.0288 - val_wine_quality_root_mean_squared_error: 0.5573 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 89/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3153 - wine_quality_loss: 0.2928 - wine_type_loss: 0.0224 - wine_quality_root_mean_squared_error: 0.5412 - wine_type_accuracy: 0.9956 - val_loss: 0.3430 - val_wine_quality_loss: 0.3135 - val_wine_type_loss: 0.0286 - val_wine_quality_root_mean_squared_error: 0.5604 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 90/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.3153 - wine_quality_loss: 0.2925 - wine_type_loss: 0.0223 - wine_quality_root_mean_squared_error: 0.5413 - wine_type_accuracy: 0.9956 - val_loss: 0.3433 - val_wine_quality_loss: 0.3141 - val_wine_type_loss: 0.0286 - val_wine_quality_root_mean_squared_error: 0.5607 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 91/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3145 - wine_quality_loss: 0.2926 - wine_type_loss: 0.0222 - wine_quality_root_mean_squared_error: 0.5406 - wine_type_accuracy: 0.9956 - val_loss: 0.3368 - val_wine_quality_loss: 0.3073 - val_wine_type_loss: 0.0286 - val_wine_quality_root_mean_squared_error: 0.5548 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 92/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3127 - wine_quality_loss: 0.2902 - wine_type_loss: 0.0221 - wine_quality_root_mean_squared_error: 0.5390 - wine_type_accuracy: 0.9956 - val_loss: 0.3482 - val_wine_quality_loss: 0.3190 - val_wine_type_loss: 0.0285 - val_wine_quality_root_mean_squared_error: 0.5651 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 93/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.3122 - wine_quality_loss: 0.2900 - wine_type_loss: 0.0220 - wine_quality_root_mean_squared_error: 0.5387 - wine_type_accuracy: 0.9956 - val_loss: 0.3374 - val_wine_quality_loss: 0.3081 - val_wine_type_loss: 0.0285 - val_wine_quality_root_mean_squared_error: 0.5555 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 94/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.3119 - wine_quality_loss: 0.2898 - wine_type_loss: 0.0219 - wine_quality_root_mean_squared_error: 0.5384 - wine_type_accuracy: 0.9956 - val_loss: 0.3374 - val_wine_quality_loss: 0.3081 - val_wine_type_loss: 0.0285 - val_wine_quality_root_mean_squared_error: 0.5554 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 95/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3120 - wine_quality_loss: 0.2894 - wine_type_loss: 0.0218 - wine_quality_root_mean_squared_error: 0.5386 - wine_type_accuracy: 0.9956 - val_loss: 0.3377 - val_wine_quality_loss: 0.3083 - val_wine_type_loss: 0.0285 - val_wine_quality_root_mean_squared_error: 0.5557 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 96/180\n",
      "3155/3155 [==============================] - 0s 71us/sample - loss: 0.3111 - wine_quality_loss: 0.2902 - wine_type_loss: 0.0217 - wine_quality_root_mean_squared_error: 0.5378 - wine_type_accuracy: 0.9956 - val_loss: 0.3366 - val_wine_quality_loss: 0.3072 - val_wine_type_loss: 0.0284 - val_wine_quality_root_mean_squared_error: 0.5547 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 97/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.3103 - wine_quality_loss: 0.2884 - wine_type_loss: 0.0216 - wine_quality_root_mean_squared_error: 0.5372 - wine_type_accuracy: 0.9956 - val_loss: 0.3411 - val_wine_quality_loss: 0.3119 - val_wine_type_loss: 0.0284 - val_wine_quality_root_mean_squared_error: 0.5589 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 98/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3098 - wine_quality_loss: 0.2880 - wine_type_loss: 0.0215 - wine_quality_root_mean_squared_error: 0.5368 - wine_type_accuracy: 0.9956 - val_loss: 0.3349 - val_wine_quality_loss: 0.3056 - val_wine_type_loss: 0.0283 - val_wine_quality_root_mean_squared_error: 0.5534 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 99/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.3098 - wine_quality_loss: 0.2889 - wine_type_loss: 0.0215 - wine_quality_root_mean_squared_error: 0.5370 - wine_type_accuracy: 0.9956 - val_loss: 0.3399 - val_wine_quality_loss: 0.3106 - val_wine_type_loss: 0.0282 - val_wine_quality_root_mean_squared_error: 0.5580 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 100/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3091 - wine_quality_loss: 0.2878 - wine_type_loss: 0.0214 - wine_quality_root_mean_squared_error: 0.5363 - wine_type_accuracy: 0.9956 - val_loss: 0.3422 - val_wine_quality_loss: 0.3130 - val_wine_type_loss: 0.0282 - val_wine_quality_root_mean_squared_error: 0.5600 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 101/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.3080 - wine_quality_loss: 0.2860 - wine_type_loss: 0.0214 - wine_quality_root_mean_squared_error: 0.5353 - wine_type_accuracy: 0.9956 - val_loss: 0.3348 - val_wine_quality_loss: 0.3058 - val_wine_type_loss: 0.0282 - val_wine_quality_root_mean_squared_error: 0.5533 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 102/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3075 - wine_quality_loss: 0.2865 - wine_type_loss: 0.0213 - wine_quality_root_mean_squared_error: 0.5349 - wine_type_accuracy: 0.9956 - val_loss: 0.3388 - val_wine_quality_loss: 0.3095 - val_wine_type_loss: 0.0282 - val_wine_quality_root_mean_squared_error: 0.5570 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 103/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3082 - wine_quality_loss: 0.2866 - wine_type_loss: 0.0212 - wine_quality_root_mean_squared_error: 0.5357 - wine_type_accuracy: 0.9956 - val_loss: 0.3420 - val_wine_quality_loss: 0.3127 - val_wine_type_loss: 0.0282 - val_wine_quality_root_mean_squared_error: 0.5598 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 104/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.3071 - wine_quality_loss: 0.2856 - wine_type_loss: 0.0211 - wine_quality_root_mean_squared_error: 0.5347 - wine_type_accuracy: 0.9956 - val_loss: 0.3384 - val_wine_quality_loss: 0.3091 - val_wine_type_loss: 0.0282 - val_wine_quality_root_mean_squared_error: 0.5566 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 105/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.3069 - wine_quality_loss: 0.2855 - wine_type_loss: 0.0210 - wine_quality_root_mean_squared_error: 0.5346 - wine_type_accuracy: 0.9956 - val_loss: 0.3349 - val_wine_quality_loss: 0.3059 - val_wine_type_loss: 0.0282 - val_wine_quality_root_mean_squared_error: 0.5535 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 106/180\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3058 - wine_quality_loss: 0.2845 - wine_type_loss: 0.0209 - wine_quality_root_mean_squared_error: 0.5337 - wine_type_accuracy: 0.9956 - val_loss: 0.3410 - val_wine_quality_loss: 0.3117 - val_wine_type_loss: 0.0281 - val_wine_quality_root_mean_squared_error: 0.5590 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 107/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.3046 - wine_quality_loss: 0.2835 - wine_type_loss: 0.0209 - wine_quality_root_mean_squared_error: 0.5326 - wine_type_accuracy: 0.9956 - val_loss: 0.3389 - val_wine_quality_loss: 0.3097 - val_wine_type_loss: 0.0281 - val_wine_quality_root_mean_squared_error: 0.5572 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 108/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3042 - wine_quality_loss: 0.2833 - wine_type_loss: 0.0208 - wine_quality_root_mean_squared_error: 0.5323 - wine_type_accuracy: 0.9959 - val_loss: 0.3390 - val_wine_quality_loss: 0.3097 - val_wine_type_loss: 0.0281 - val_wine_quality_root_mean_squared_error: 0.5572 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 109/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.3047 - wine_quality_loss: 0.2841 - wine_type_loss: 0.0207 - wine_quality_root_mean_squared_error: 0.5328 - wine_type_accuracy: 0.9959 - val_loss: 0.3338 - val_wine_quality_loss: 0.3049 - val_wine_type_loss: 0.0280 - val_wine_quality_root_mean_squared_error: 0.5527 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 110/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.3028 - wine_quality_loss: 0.2826 - wine_type_loss: 0.0206 - wine_quality_root_mean_squared_error: 0.5312 - wine_type_accuracy: 0.9959 - val_loss: 0.3411 - val_wine_quality_loss: 0.3125 - val_wine_type_loss: 0.0279 - val_wine_quality_root_mean_squared_error: 0.5593 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 111/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3034 - wine_quality_loss: 0.2824 - wine_type_loss: 0.0205 - wine_quality_root_mean_squared_error: 0.5318 - wine_type_accuracy: 0.9959 - val_loss: 0.3397 - val_wine_quality_loss: 0.3106 - val_wine_type_loss: 0.0280 - val_wine_quality_root_mean_squared_error: 0.5580 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 112/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.3026 - wine_quality_loss: 0.2820 - wine_type_loss: 0.0204 - wine_quality_root_mean_squared_error: 0.5311 - wine_type_accuracy: 0.9956 - val_loss: 0.3355 - val_wine_quality_loss: 0.3066 - val_wine_type_loss: 0.0278 - val_wine_quality_root_mean_squared_error: 0.5544 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 113/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3014 - wine_quality_loss: 0.2806 - wine_type_loss: 0.0203 - wine_quality_root_mean_squared_error: 0.5301 - wine_type_accuracy: 0.9959 - val_loss: 0.3340 - val_wine_quality_loss: 0.3055 - val_wine_type_loss: 0.0278 - val_wine_quality_root_mean_squared_error: 0.5531 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 114/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.3014 - wine_quality_loss: 0.2808 - wine_type_loss: 0.0202 - wine_quality_root_mean_squared_error: 0.5302 - wine_type_accuracy: 0.9959 - val_loss: 0.3388 - val_wine_quality_loss: 0.3101 - val_wine_type_loss: 0.0278 - val_wine_quality_root_mean_squared_error: 0.5573 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 115/180\n",
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.3017 - wine_quality_loss: 0.2814 - wine_type_loss: 0.0201 - wine_quality_root_mean_squared_error: 0.5306 - wine_type_accuracy: 0.9959 - val_loss: 0.3343 - val_wine_quality_loss: 0.3057 - val_wine_type_loss: 0.0278 - val_wine_quality_root_mean_squared_error: 0.5533 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 116/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3000 - wine_quality_loss: 0.2798 - wine_type_loss: 0.0201 - wine_quality_root_mean_squared_error: 0.5290 - wine_type_accuracy: 0.9959 - val_loss: 0.3422 - val_wine_quality_loss: 0.3133 - val_wine_type_loss: 0.0277 - val_wine_quality_root_mean_squared_error: 0.5605 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 117/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.3001 - wine_quality_loss: 0.2801 - wine_type_loss: 0.0203 - wine_quality_root_mean_squared_error: 0.5292 - wine_type_accuracy: 0.9959 - val_loss: 0.3332 - val_wine_quality_loss: 0.3048 - val_wine_type_loss: 0.0277 - val_wine_quality_root_mean_squared_error: 0.5524 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 118/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.2987 - wine_quality_loss: 0.2789 - wine_type_loss: 0.0199 - wine_quality_root_mean_squared_error: 0.5280 - wine_type_accuracy: 0.9959 - val_loss: 0.3376 - val_wine_quality_loss: 0.3087 - val_wine_type_loss: 0.0278 - val_wine_quality_root_mean_squared_error: 0.5562 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 119/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2990 - wine_quality_loss: 0.2791 - wine_type_loss: 0.0198 - wine_quality_root_mean_squared_error: 0.5282 - wine_type_accuracy: 0.9959 - val_loss: 0.3378 - val_wine_quality_loss: 0.3089 - val_wine_type_loss: 0.0278 - val_wine_quality_root_mean_squared_error: 0.5565 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 120/180\n",
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.2996 - wine_quality_loss: 0.2799 - wine_type_loss: 0.0198 - wine_quality_root_mean_squared_error: 0.5289 - wine_type_accuracy: 0.9959 - val_loss: 0.3403 - val_wine_quality_loss: 0.3116 - val_wine_type_loss: 0.0276 - val_wine_quality_root_mean_squared_error: 0.5589 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 121/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2981 - wine_quality_loss: 0.2786 - wine_type_loss: 0.0198 - wine_quality_root_mean_squared_error: 0.5275 - wine_type_accuracy: 0.9959 - val_loss: 0.3342 - val_wine_quality_loss: 0.3056 - val_wine_type_loss: 0.0276 - val_wine_quality_root_mean_squared_error: 0.5534 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 122/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.2983 - wine_quality_loss: 0.2789 - wine_type_loss: 0.0197 - wine_quality_root_mean_squared_error: 0.5278 - wine_type_accuracy: 0.9959 - val_loss: 0.3413 - val_wine_quality_loss: 0.3126 - val_wine_type_loss: 0.0276 - val_wine_quality_root_mean_squared_error: 0.5597 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 123/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.2976 - wine_quality_loss: 0.2774 - wine_type_loss: 0.0196 - wine_quality_root_mean_squared_error: 0.5272 - wine_type_accuracy: 0.9959 - val_loss: 0.3368 - val_wine_quality_loss: 0.3084 - val_wine_type_loss: 0.0275 - val_wine_quality_root_mean_squared_error: 0.5558 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 124/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.2967 - wine_quality_loss: 0.2773 - wine_type_loss: 0.0195 - wine_quality_root_mean_squared_error: 0.5265 - wine_type_accuracy: 0.9959 - val_loss: 0.3370 - val_wine_quality_loss: 0.3084 - val_wine_type_loss: 0.0275 - val_wine_quality_root_mean_squared_error: 0.5560 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 125/180\n",
      "3155/3155 [==============================] - 0s 86us/sample - loss: 0.2965 - wine_quality_loss: 0.2768 - wine_type_loss: 0.0196 - wine_quality_root_mean_squared_error: 0.5263 - wine_type_accuracy: 0.9962 - val_loss: 0.3350 - val_wine_quality_loss: 0.3069 - val_wine_type_loss: 0.0275 - val_wine_quality_root_mean_squared_error: 0.5542 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 126/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.2967 - wine_quality_loss: 0.2769 - wine_type_loss: 0.0193 - wine_quality_root_mean_squared_error: 0.5266 - wine_type_accuracy: 0.9962 - val_loss: 0.3363 - val_wine_quality_loss: 0.3081 - val_wine_type_loss: 0.0275 - val_wine_quality_root_mean_squared_error: 0.5554 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 127/180\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2959 - wine_quality_loss: 0.2768 - wine_type_loss: 0.0193 - wine_quality_root_mean_squared_error: 0.5259 - wine_type_accuracy: 0.9962 - val_loss: 0.3333 - val_wine_quality_loss: 0.3050 - val_wine_type_loss: 0.0274 - val_wine_quality_root_mean_squared_error: 0.5528 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 128/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.2949 - wine_quality_loss: 0.2754 - wine_type_loss: 0.0192 - wine_quality_root_mean_squared_error: 0.5250 - wine_type_accuracy: 0.9962 - val_loss: 0.3350 - val_wine_quality_loss: 0.3067 - val_wine_type_loss: 0.0275 - val_wine_quality_root_mean_squared_error: 0.5542 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 129/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.2946 - wine_quality_loss: 0.2752 - wine_type_loss: 0.0191 - wine_quality_root_mean_squared_error: 0.5248 - wine_type_accuracy: 0.9962 - val_loss: 0.3414 - val_wine_quality_loss: 0.3135 - val_wine_type_loss: 0.0274 - val_wine_quality_root_mean_squared_error: 0.5600 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 130/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2951 - wine_quality_loss: 0.2763 - wine_type_loss: 0.0192 - wine_quality_root_mean_squared_error: 0.5253 - wine_type_accuracy: 0.9962 - val_loss: 0.3386 - val_wine_quality_loss: 0.3104 - val_wine_type_loss: 0.0273 - val_wine_quality_root_mean_squared_error: 0.5576 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 131/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.2943 - wine_quality_loss: 0.2750 - wine_type_loss: 0.0190 - wine_quality_root_mean_squared_error: 0.5246 - wine_type_accuracy: 0.9962 - val_loss: 0.3354 - val_wine_quality_loss: 0.3073 - val_wine_type_loss: 0.0272 - val_wine_quality_root_mean_squared_error: 0.5549 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 132/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.2943 - wine_quality_loss: 0.2748 - wine_type_loss: 0.0191 - wine_quality_root_mean_squared_error: 0.5247 - wine_type_accuracy: 0.9962 - val_loss: 0.3386 - val_wine_quality_loss: 0.3108 - val_wine_type_loss: 0.0272 - val_wine_quality_root_mean_squared_error: 0.5577 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 133/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.2933 - wine_quality_loss: 0.2739 - wine_type_loss: 0.0189 - wine_quality_root_mean_squared_error: 0.5238 - wine_type_accuracy: 0.9962 - val_loss: 0.3389 - val_wine_quality_loss: 0.3108 - val_wine_type_loss: 0.0272 - val_wine_quality_root_mean_squared_error: 0.5580 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 134/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.2936 - wine_quality_loss: 0.2747 - wine_type_loss: 0.0188 - wine_quality_root_mean_squared_error: 0.5241 - wine_type_accuracy: 0.9962 - val_loss: 0.3368 - val_wine_quality_loss: 0.3087 - val_wine_type_loss: 0.0271 - val_wine_quality_root_mean_squared_error: 0.5561 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 135/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2929 - wine_quality_loss: 0.2742 - wine_type_loss: 0.0187 - wine_quality_root_mean_squared_error: 0.5235 - wine_type_accuracy: 0.9962 - val_loss: 0.3413 - val_wine_quality_loss: 0.3133 - val_wine_type_loss: 0.0270 - val_wine_quality_root_mean_squared_error: 0.5603 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 136/180\n",
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.2925 - wine_quality_loss: 0.2736 - wine_type_loss: 0.0187 - wine_quality_root_mean_squared_error: 0.5232 - wine_type_accuracy: 0.9962 - val_loss: 0.3365 - val_wine_quality_loss: 0.3087 - val_wine_type_loss: 0.0270 - val_wine_quality_root_mean_squared_error: 0.5560 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 137/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2920 - wine_quality_loss: 0.2735 - wine_type_loss: 0.0186 - wine_quality_root_mean_squared_error: 0.5228 - wine_type_accuracy: 0.9965 - val_loss: 0.3351 - val_wine_quality_loss: 0.3074 - val_wine_type_loss: 0.0270 - val_wine_quality_root_mean_squared_error: 0.5548 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 138/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2912 - wine_quality_loss: 0.2723 - wine_type_loss: 0.0185 - wine_quality_root_mean_squared_error: 0.5221 - wine_type_accuracy: 0.9962 - val_loss: 0.3344 - val_wine_quality_loss: 0.3070 - val_wine_type_loss: 0.0270 - val_wine_quality_root_mean_squared_error: 0.5542 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 139/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.2906 - wine_quality_loss: 0.2718 - wine_type_loss: 0.0185 - wine_quality_root_mean_squared_error: 0.5216 - wine_type_accuracy: 0.9965 - val_loss: 0.3422 - val_wine_quality_loss: 0.3149 - val_wine_type_loss: 0.0269 - val_wine_quality_root_mean_squared_error: 0.5612 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 140/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2918 - wine_quality_loss: 0.2729 - wine_type_loss: 0.0184 - wine_quality_root_mean_squared_error: 0.5229 - wine_type_accuracy: 0.9962 - val_loss: 0.3357 - val_wine_quality_loss: 0.3079 - val_wine_type_loss: 0.0269 - val_wine_quality_root_mean_squared_error: 0.5554 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 141/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2888 - wine_quality_loss: 0.2706 - wine_type_loss: 0.0184 - wine_quality_root_mean_squared_error: 0.5201 - wine_type_accuracy: 0.9962 - val_loss: 0.3352 - val_wine_quality_loss: 0.3077 - val_wine_type_loss: 0.0270 - val_wine_quality_root_mean_squared_error: 0.5548 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 142/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.2898 - wine_quality_loss: 0.2714 - wine_type_loss: 0.0183 - wine_quality_root_mean_squared_error: 0.5210 - wine_type_accuracy: 0.9962 - val_loss: 0.3349 - val_wine_quality_loss: 0.3075 - val_wine_type_loss: 0.0269 - val_wine_quality_root_mean_squared_error: 0.5547 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 143/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2898 - wine_quality_loss: 0.2716 - wine_type_loss: 0.0182 - wine_quality_root_mean_squared_error: 0.5211 - wine_type_accuracy: 0.9962 - val_loss: 0.3330 - val_wine_quality_loss: 0.3054 - val_wine_type_loss: 0.0269 - val_wine_quality_root_mean_squared_error: 0.5529 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 144/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2889 - wine_quality_loss: 0.2708 - wine_type_loss: 0.0182 - wine_quality_root_mean_squared_error: 0.5202 - wine_type_accuracy: 0.9962 - val_loss: 0.3341 - val_wine_quality_loss: 0.3066 - val_wine_type_loss: 0.0268 - val_wine_quality_root_mean_squared_error: 0.5540 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 145/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.2884 - wine_quality_loss: 0.2709 - wine_type_loss: 0.0181 - wine_quality_root_mean_squared_error: 0.5199 - wine_type_accuracy: 0.9962 - val_loss: 0.3352 - val_wine_quality_loss: 0.3076 - val_wine_type_loss: 0.0268 - val_wine_quality_root_mean_squared_error: 0.5550 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 146/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.2883 - wine_quality_loss: 0.2703 - wine_type_loss: 0.0180 - wine_quality_root_mean_squared_error: 0.5198 - wine_type_accuracy: 0.9962 - val_loss: 0.3309 - val_wine_quality_loss: 0.3034 - val_wine_type_loss: 0.0267 - val_wine_quality_root_mean_squared_error: 0.5511 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 147/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.2878 - wine_quality_loss: 0.2698 - wine_type_loss: 0.0179 - wine_quality_root_mean_squared_error: 0.5194 - wine_type_accuracy: 0.9965 - val_loss: 0.3348 - val_wine_quality_loss: 0.3072 - val_wine_type_loss: 0.0267 - val_wine_quality_root_mean_squared_error: 0.5547 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 148/180\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.2871 - wine_quality_loss: 0.2688 - wine_type_loss: 0.0179 - wine_quality_root_mean_squared_error: 0.5188 - wine_type_accuracy: 0.9962 - val_loss: 0.3295 - val_wine_quality_loss: 0.3022 - val_wine_type_loss: 0.0266 - val_wine_quality_root_mean_squared_error: 0.5500 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 149/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2882 - wine_quality_loss: 0.2698 - wine_type_loss: 0.0179 - wine_quality_root_mean_squared_error: 0.5199 - wine_type_accuracy: 0.9965 - val_loss: 0.3345 - val_wine_quality_loss: 0.3069 - val_wine_type_loss: 0.0267 - val_wine_quality_root_mean_squared_error: 0.5544 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 150/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.2868 - wine_quality_loss: 0.2685 - wine_type_loss: 0.0178 - wine_quality_root_mean_squared_error: 0.5186 - wine_type_accuracy: 0.9965 - val_loss: 0.3325 - val_wine_quality_loss: 0.3054 - val_wine_type_loss: 0.0267 - val_wine_quality_root_mean_squared_error: 0.5527 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 151/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.2869 - wine_quality_loss: 0.2695 - wine_type_loss: 0.0177 - wine_quality_root_mean_squared_error: 0.5188 - wine_type_accuracy: 0.9965 - val_loss: 0.3402 - val_wine_quality_loss: 0.3124 - val_wine_type_loss: 0.0267 - val_wine_quality_root_mean_squared_error: 0.5596 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 152/180\n",
      "3155/3155 [==============================] - 0s 85us/sample - loss: 0.2861 - wine_quality_loss: 0.2685 - wine_type_loss: 0.0176 - wine_quality_root_mean_squared_error: 0.5181 - wine_type_accuracy: 0.9965 - val_loss: 0.3357 - val_wine_quality_loss: 0.3081 - val_wine_type_loss: 0.0266 - val_wine_quality_root_mean_squared_error: 0.5556 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 153/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.2857 - wine_quality_loss: 0.2687 - wine_type_loss: 0.0176 - wine_quality_root_mean_squared_error: 0.5177 - wine_type_accuracy: 0.9968 - val_loss: 0.3405 - val_wine_quality_loss: 0.3129 - val_wine_type_loss: 0.0266 - val_wine_quality_root_mean_squared_error: 0.5600 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 154/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.2853 - wine_quality_loss: 0.2680 - wine_type_loss: 0.0176 - wine_quality_root_mean_squared_error: 0.5174 - wine_type_accuracy: 0.9968 - val_loss: 0.3330 - val_wine_quality_loss: 0.3057 - val_wine_type_loss: 0.0265 - val_wine_quality_root_mean_squared_error: 0.5533 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 155/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.2845 - wine_quality_loss: 0.2667 - wine_type_loss: 0.0175 - wine_quality_root_mean_squared_error: 0.5167 - wine_type_accuracy: 0.9968 - val_loss: 0.3389 - val_wine_quality_loss: 0.3114 - val_wine_type_loss: 0.0265 - val_wine_quality_root_mean_squared_error: 0.5586 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 156/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2847 - wine_quality_loss: 0.2672 - wine_type_loss: 0.0174 - wine_quality_root_mean_squared_error: 0.5169 - wine_type_accuracy: 0.9965 - val_loss: 0.3340 - val_wine_quality_loss: 0.3066 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5543 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 157/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2847 - wine_quality_loss: 0.2670 - wine_type_loss: 0.0174 - wine_quality_root_mean_squared_error: 0.5170 - wine_type_accuracy: 0.9968 - val_loss: 0.3348 - val_wine_quality_loss: 0.3079 - val_wine_type_loss: 0.0265 - val_wine_quality_root_mean_squared_error: 0.5550 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 158/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.2836 - wine_quality_loss: 0.2666 - wine_type_loss: 0.0173 - wine_quality_root_mean_squared_error: 0.5160 - wine_type_accuracy: 0.9968 - val_loss: 0.3409 - val_wine_quality_loss: 0.3135 - val_wine_type_loss: 0.0265 - val_wine_quality_root_mean_squared_error: 0.5604 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 159/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2838 - wine_quality_loss: 0.2664 - wine_type_loss: 0.0179 - wine_quality_root_mean_squared_error: 0.5163 - wine_type_accuracy: 0.9968 - val_loss: 0.3344 - val_wine_quality_loss: 0.3072 - val_wine_type_loss: 0.0265 - val_wine_quality_root_mean_squared_error: 0.5546 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 160/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.2836 - wine_quality_loss: 0.2661 - wine_type_loss: 0.0172 - wine_quality_root_mean_squared_error: 0.5161 - wine_type_accuracy: 0.9968 - val_loss: 0.3327 - val_wine_quality_loss: 0.3055 - val_wine_type_loss: 0.0265 - val_wine_quality_root_mean_squared_error: 0.5530 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 161/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.2835 - wine_quality_loss: 0.2662 - wine_type_loss: 0.0172 - wine_quality_root_mean_squared_error: 0.5160 - wine_type_accuracy: 0.9965 - val_loss: 0.3367 - val_wine_quality_loss: 0.3095 - val_wine_type_loss: 0.0266 - val_wine_quality_root_mean_squared_error: 0.5566 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 162/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.2830 - wine_quality_loss: 0.2658 - wine_type_loss: 0.0171 - wine_quality_root_mean_squared_error: 0.5156 - wine_type_accuracy: 0.9965 - val_loss: 0.3368 - val_wine_quality_loss: 0.3093 - val_wine_type_loss: 0.0266 - val_wine_quality_root_mean_squared_error: 0.5567 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 163/180\n",
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.2825 - wine_quality_loss: 0.2654 - wine_type_loss: 0.0171 - wine_quality_root_mean_squared_error: 0.5152 - wine_type_accuracy: 0.9968 - val_loss: 0.3332 - val_wine_quality_loss: 0.3061 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5536 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 164/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2818 - wine_quality_loss: 0.2647 - wine_type_loss: 0.0170 - wine_quality_root_mean_squared_error: 0.5146 - wine_type_accuracy: 0.9965 - val_loss: 0.3366 - val_wine_quality_loss: 0.3092 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5566 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 165/180\n",
      "3155/3155 [==============================] - 0s 71us/sample - loss: 0.2806 - wine_quality_loss: 0.2635 - wine_type_loss: 0.0170 - wine_quality_root_mean_squared_error: 0.5134 - wine_type_accuracy: 0.9965 - val_loss: 0.3385 - val_wine_quality_loss: 0.3113 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5583 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 166/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2817 - wine_quality_loss: 0.2646 - wine_type_loss: 0.0169 - wine_quality_root_mean_squared_error: 0.5146 - wine_type_accuracy: 0.9968 - val_loss: 0.3336 - val_wine_quality_loss: 0.3063 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5540 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 167/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2811 - wine_quality_loss: 0.2638 - wine_type_loss: 0.0168 - wine_quality_root_mean_squared_error: 0.5141 - wine_type_accuracy: 0.9968 - val_loss: 0.3314 - val_wine_quality_loss: 0.3045 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5521 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 168/180\n",
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.2813 - wine_quality_loss: 0.2645 - wine_type_loss: 0.0167 - wine_quality_root_mean_squared_error: 0.5143 - wine_type_accuracy: 0.9968 - val_loss: 0.3321 - val_wine_quality_loss: 0.3051 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5526 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 169/180\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2804 - wine_quality_loss: 0.2638 - wine_type_loss: 0.0167 - wine_quality_root_mean_squared_error: 0.5135 - wine_type_accuracy: 0.9965 - val_loss: 0.3359 - val_wine_quality_loss: 0.3089 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5561 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 170/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.2790 - wine_quality_loss: 0.2624 - wine_type_loss: 0.0166 - wine_quality_root_mean_squared_error: 0.5122 - wine_type_accuracy: 0.9965 - val_loss: 0.3311 - val_wine_quality_loss: 0.3039 - val_wine_type_loss: 0.0262 - val_wine_quality_root_mean_squared_error: 0.5519 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 171/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.2786 - wine_quality_loss: 0.2617 - wine_type_loss: 0.0166 - wine_quality_root_mean_squared_error: 0.5118 - wine_type_accuracy: 0.9968 - val_loss: 0.3333 - val_wine_quality_loss: 0.3065 - val_wine_type_loss: 0.0262 - val_wine_quality_root_mean_squared_error: 0.5539 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 172/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.2790 - wine_quality_loss: 0.2624 - wine_type_loss: 0.0166 - wine_quality_root_mean_squared_error: 0.5122 - wine_type_accuracy: 0.9968 - val_loss: 0.3337 - val_wine_quality_loss: 0.3068 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5541 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 173/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.2789 - wine_quality_loss: 0.2624 - wine_type_loss: 0.0165 - wine_quality_root_mean_squared_error: 0.5122 - wine_type_accuracy: 0.9968 - val_loss: 0.3328 - val_wine_quality_loss: 0.3057 - val_wine_type_loss: 0.0262 - val_wine_quality_root_mean_squared_error: 0.5533 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 174/180\n",
      "3155/3155 [==============================] - 0s 86us/sample - loss: 0.2779 - wine_quality_loss: 0.2610 - wine_type_loss: 0.0164 - wine_quality_root_mean_squared_error: 0.5113 - wine_type_accuracy: 0.9968 - val_loss: 0.3324 - val_wine_quality_loss: 0.3053 - val_wine_type_loss: 0.0262 - val_wine_quality_root_mean_squared_error: 0.5530 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 175/180\n",
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.2783 - wine_quality_loss: 0.2616 - wine_type_loss: 0.0163 - wine_quality_root_mean_squared_error: 0.5118 - wine_type_accuracy: 0.9968 - val_loss: 0.3387 - val_wine_quality_loss: 0.3117 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5587 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 176/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2777 - wine_quality_loss: 0.2612 - wine_type_loss: 0.0163 - wine_quality_root_mean_squared_error: 0.5112 - wine_type_accuracy: 0.9968 - val_loss: 0.3340 - val_wine_quality_loss: 0.3068 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5544 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 177/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.2773 - wine_quality_loss: 0.2608 - wine_type_loss: 0.0163 - wine_quality_root_mean_squared_error: 0.5109 - wine_type_accuracy: 0.9968 - val_loss: 0.3342 - val_wine_quality_loss: 0.3073 - val_wine_type_loss: 0.0262 - val_wine_quality_root_mean_squared_error: 0.5547 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 178/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2774 - wine_quality_loss: 0.2611 - wine_type_loss: 0.0162 - wine_quality_root_mean_squared_error: 0.5110 - wine_type_accuracy: 0.9968 - val_loss: 0.3417 - val_wine_quality_loss: 0.3144 - val_wine_type_loss: 0.0262 - val_wine_quality_root_mean_squared_error: 0.5615 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 179/180\n",
      "3155/3155 [==============================] - 0s 86us/sample - loss: 0.2772 - wine_quality_loss: 0.2613 - wine_type_loss: 0.0162 - wine_quality_root_mean_squared_error: 0.5108 - wine_type_accuracy: 0.9968 - val_loss: 0.3312 - val_wine_quality_loss: 0.3043 - val_wine_type_loss: 0.0262 - val_wine_quality_root_mean_squared_error: 0.5520 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 180/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.2767 - wine_quality_loss: 0.2607 - wine_type_loss: 0.0161 - wine_quality_root_mean_squared_error: 0.5104 - wine_type_accuracy: 0.9968 - val_loss: 0.3468 - val_wine_quality_loss: 0.3197 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5659 - val_wine_type_accuracy: 0.9949\n"
     ]
    }
   ],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "history = model.fit(norm_train_X, train_Y, # YOUR CODE HERE, # YOUR CODE HERE,\n",
    "                    epochs = 180, validation_data=(norm_val_X, val_Y))# YOUR CODE HERE, # YOUR CODE HERE))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "fadad8896eda9c8c2115970724b15508",
     "grade": true,
     "grade_id": "cell-eb4d5b41bef8f0ab",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_history(history)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "CubF2J2gSf6q"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "789/789 [==============================] - 0s 22us/sample - loss: 0.3468 - wine_quality_loss: 0.3197 - wine_type_loss: 0.0263 - wine_quality_root_mean_squared_error: 0.5659 - wine_type_accuracy: 0.9949\n",
      "\n",
      "loss: 0.3468224025497872\n",
      "wine_quality_loss: 0.31971216201782227\n",
      "wine_type_loss: 0.02625664696097374\n",
      "wine_quality_rmse: 0.5658793449401855\n",
      "wine_type_accuracy: 0.9949302673339844\n"
     ]
    }
   ],
   "source": [
    "# Gather the training metrics\n",
    "loss, wine_quality_loss, wine_type_loss, wine_quality_rmse, wine_type_accuracy = model.evaluate(x=norm_val_X, y=val_Y)\n",
    "\n",
    "print()\n",
    "print(f'loss: {loss}')\n",
    "print(f'wine_quality_loss: {wine_quality_loss}')\n",
    "print(f'wine_type_loss: {wine_type_loss}')\n",
    "print(f'wine_quality_rmse: {wine_quality_rmse}')\n",
    "print(f'wine_type_accuracy: {wine_type_accuracy}')\n",
    "\n",
    "# EXPECTED VALUES\n",
    "# ~ 0.30 - 0.38\n",
    "# ~ 0.30 - 0.38\n",
    "# ~ 0.018 - 0.030\n",
    "# ~ 0.50 - 0.62\n",
    "# ~ 0.97 - 1.0\n",
    "\n",
    "# Example:\n",
    "#0.3657050132751465\n",
    "#0.3463745415210724\n",
    "#0.019330406561493874\n",
    "#0.5885359048843384\n",
    "#0.9974651336669922"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gPtTGAP4usnm"
   },
   "source": [
    "## Analyze the Model Performance\n",
    "\n",
    "Note that the model has two outputs. The output at index 0 is quality and index 1 is wine type\n",
    "\n",
    "So, round the quality predictions to the nearest integer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "tBq9PEeAaW-Y"
   },
   "outputs": [],
   "source": [
    "predictions = model.predict(norm_test_X)\n",
    "quality_pred = predictions[0]\n",
    "type_pred = predictions[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "YLhgTR4xTIxj"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[5.7742214]\n"
     ]
    }
   ],
   "source": [
    "print(quality_pred[0])\n",
    "\n",
    "# EXPECTED OUTPUT\n",
    "# 5.6 - 6.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "MPi-eYfGTUXi"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.00044387]\n",
      "[0.99999917]\n"
     ]
    }
   ],
   "source": [
    "print(type_pred[0])\n",
    "print(type_pred[944])\n",
    "\n",
    "# EXPECTED OUTPUT\n",
    "# A number close to zero\n",
    "# A number close to or equal to 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Kohk-9C6vt_s"
   },
   "source": [
    "### Plot Utilities\n",
    "\n",
    "We define a few utilities to visualize the model performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "62gEOFUhn6aQ"
   },
   "outputs": [],
   "source": [
    "def plot_metrics(metric_name, title, ylim=5):\n",
    "    plt.title(title)\n",
    "    plt.ylim(0,ylim)\n",
    "    plt.plot(history.history[metric_name],color='blue',label=metric_name)\n",
    "    plt.plot(history.history['val_' + metric_name],color='green',label='val_' + metric_name)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6rfgSx7uz5dj"
   },
   "outputs": [],
   "source": [
    "def plot_confusion_matrix(y_true, y_pred, title='', labels=[0,1]):\n",
    "    cm = confusion_matrix(y_true, y_pred)\n",
    "    fig = plt.figure()\n",
    "    ax = fig.add_subplot(111)\n",
    "    cax = ax.matshow(cm)\n",
    "    plt.title('Confusion matrix of the classifier')\n",
    "    fig.colorbar(cax)\n",
    "    ax.set_xticklabels([''] + labels)\n",
    "    ax.set_yticklabels([''] + labels)\n",
    "    plt.xlabel('Predicted')\n",
    "    plt.ylabel('True')\n",
    "    fmt = 'd'\n",
    "    thresh = cm.max() / 2.\n",
    "    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
    "          plt.text(j, i, format(cm[i, j], fmt),\n",
    "                  horizontalalignment=\"center\",\n",
    "                  color=\"black\" if cm[i, j] > thresh else \"white\")\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "dfVLIqi017Vf"
   },
   "outputs": [],
   "source": [
    "def plot_diff(y_true, y_pred, title = '' ):\n",
    "    plt.scatter(y_true, y_pred)\n",
    "    plt.title(title)\n",
    "    plt.xlabel('True Values')\n",
    "    plt.ylabel('Predictions')\n",
    "    plt.axis('equal')\n",
    "    plt.axis('square')\n",
    "    plt.plot([-100, 100], [-100, 100])\n",
    "    return plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8sd1jdFbwE0I"
   },
   "source": [
    "### Plots for Metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "f3MwZ5J1pOfj"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deZxU9Znv8c9TVb3QG93QTbPvyKIImg6KKETjgruJGS/GRLNMiEmca7aZmGTiJHPnzsTRLDeJicMkxpgJahyXIaNRjHEkRomAsimCLaA0zU7T0HtX93P/ONVQtt1QQHVXd9X3/XqdV1X9zu+ceurQfM+pX506Ze6OiIikr1CqCxARkZ6loBcRSXMKehGRNKegFxFJcwp6EZE0p6AXEUlzCnoRkTSnoJeMYmZbzazRzOrMbKeZ3WdmBbF595mZm9lVnZb5Yaz9E7HH2Wb2PTOriq1ni5n9oJvn6Jh+0qsvVCSOgl4y0ZXuXgDMBM4Avh43bxNwU8cDM4sAfwW8Fdfn60AFMAsoBM4HXu3qOeKmW5L/MkQSE0l1ASKp4u47zexpgsDv8DvgY2ZW4u41wHxgLUGgd3g/8Ji7V8ceb41NIn2SjuglY5nZSOBSoDKuuQlYAiyIPb4RuL/TosuBL5vZ581suplZjxcrchIU9JKJHjezQ8A2YDfwD53m3w/caGYDgXnA453m/wtwB3ADsBLYbmY3derzuJkdiJs+k/RXIZIgBb1komvcvRD4ADAFKI2f6e4vAGXA3wP/7e6Nnea3ufvd7j4HKAb+L3CvmU3t9BzFcdO/9+DrETkqBb1kLHd/HrgPuKuL2f8BfIX3Dtt0Xkeju98N1ADTkl2jSDLow1jJdD8EtprZzE7tPwL+BCzrvICZfRFYDfwFaCUYwinkvWfeiPQJCnrJaO6+x8zuB74FHIpr3w88281ijcD3gImAE5ySea27b47r8zsza4t7/Iy7fyipxYskyPTDIyIi6U1j9CIiae6YQW9mo8zsOTPbYGavmdmtXfQxM/uRmVWa2VozOzNu3nwz2xibd1uyX4CIiBxdIkf0UeAr7j4VOBv4gpl1PrvgUmBSbFoI/AzAzMLA3bH504Dru1hWRER60DGD3t13uPsrsfuHgA3AiE7drgbu98ByoNjMhhFcC6TS3Te7ewvwYKyviIj0kuM668bMxhJcBOovnWaNIPiWYYeqWFtX7Wd1s+6FBO8GyM/Pf9+UKVOOp7T3WLdrHd5cSKRuLNP0HkJE0tyqVav2untZV/MSDvrYpVwfAb7o7gc7z+5iET9K+3sb3RcBiwAqKip85cqViZbWpfH/bzxtW+ZQ+Idfc5KrEhHp88zs7e7mJRT0ZpZFEPK/cfdHu+hSBYyKezwSqAayu2nvceFQmLZQGy0tvfFsIiJ9VyJn3RjwC2CDu3+/m25LCC4CZWZ2NlDr7juAFcAkMxtnZtkEVwRckqTajyoSimDhNlpbe+PZRET6rkSO6OcAHwfWmdnqWNs3gNEA7n4P8CRwGcHlXhuAT8bmRc3sFuBpIAzc6+6vJfUVdCNsYSwU1RG9iGS8YwZ97Ep+R73etgdfr/1CN/OeJNgR9KpwKAwauhERSd9vxgZH9Bq6ERFJ26CPhCI6ohcRIY2DPhi6ieqIXkQyXvoGvYXB2ohGob091dWIiKRO+gZ97MNYQEf1IpLR0jboI6EIblFAQS8imS1tgz5sYTz2Az/6QFZEMln6Bn0oGKMHHdGLSGZL26APhm50RC8ikrZBHwzdBGP0CnoRyWTpG/ShMI6GbkRE0jfo9WGsiAiQxkEfCUVoR0M3IiJpG/QauhERCaRv0FuYdjR0IyKS1kGvI3oRkTQO+kgoQptrjF5EJG2DPhzS0I2ICCTwU4Jmdi9wBbDb3U/rYv7fAjfErW8qUObu+81sK3AIaAOi7l6RrMKPJWxh2l1DNyIiiRzR3wfM726mu9/p7jPdfSbwdeB5d98f1+X82PxeC3kIhm6iHiS8juhFJJMdM+jdfRmw/1j9Yq4HHjipipKkKKeIxmg9WLuO6EUkoyVtjN7M8giO/B+Ja3ZgqZmtMrOFyXquRBTnFuM45NTqiF5EMtoxx+iPw5XAnzsN28xx92ozGwI8Y2ZvxN4hvEdsR7AQYPTo0SddTMmAkuBO7gFaWkpOen0iIv1VMs+6WUCnYRt3r47d7gYeA2Z1t7C7L3L3CnevKCsrO+liinOLgzu5BzR0IyIZLSlBb2YDgXnAf8W15ZtZYcd94GJgfTKeLxGHg35AjYZuRCSjJXJ65QPAB4BSM6sC/gHIAnD3e2LdPgQsdff6uEXLgcfMrON5Frv7U8kr/ehKco8M3eiIXkQy2TGD3t2vT6DPfQSnYca3bQZmnGhhJ+vI0I2O6EUks6XtN2M7PowN5x9Q0ItIRkvboC/ILiBkIUJ5GroRkcyWtkEfshADcwYSytfQjYhktrQNegiGb2yAhm5EJLOlddAX5xZDbo2GbkQko6V90HvuAZqbU12JiEjqpHXQl+QGQzd79qS6EhGR1EnroC/OLcZzatixI9WViIikTloHfUluCW1ZBxT0IpLR0jroi3OLiVoj+w4068wbEclYaR/0AOQeYOfO1NYiIpIqaR308dek1/CNiGSqtA76+EsVK+hFJFNlRtDriF5EMlhaB338NekV9CKSqdI66DuO6AuH1OjDWBHJWGkd9B0fxhYM1hG9iGSutA763EguOeEcckv0YayIZK60DnoIhm+yinRELyKZ65hBb2b3mtluM1vfzfwPmFmtma2OTbfHzZtvZhvNrNLMbktm4YkqLyinLW87u3ZBW1sqKhARSa1EjujvA+Yfo8+f3H1mbPpHADMLA3cDlwLTgOvNbNrJFHsippROoTbyBm1tsHdvbz+7iEjqHTPo3X0ZsP8E1j0LqHT3ze7eAjwIXH0C6zkpU0unsq99C0QaNXwjIhkpWWP0s81sjZn93sxOjbWNALbF9amKtXXJzBaa2UozW7kniReQn1o6Fcdh8CYFvYhkpGQE/SvAGHefAfwYeDzWbl309e5W4u6L3L3C3SvKysqSUFZgatnU4E7ZBrZsSdpqRUT6jZMOenc/6O51sftPAllmVkpwBD8qrutIoPpkn+94nTL4FEIWInvEBt54o7efXUQk9U466M1sqJlZ7P6s2Dr3ASuASWY2zsyygQXAkpN9vuOVG8llXPE48sdsYMOG3n52EZHUixyrg5k9AHwAKDWzKuAfgCwAd78H+AjwOTOLAo3AAnd3IGpmtwBPA2HgXnd/rUdexTFMLZvKC6Ub2PBEKp5dRCS1jhn07n79Meb/BPhJN/OeBJ48sdKSZ2rpVH6fvZQDO6IcPBihqCjVFYmI9J60/2YsBEHfRgsUb9E4vYhknMwI+rgzbzROLyKZJiOC/tSy4NT+0LC1CnoRyTgZEfSFOYVMKJlA/oQ1CnoRyTgZEfQAM4bOwIes1Ri9iGScjAn604ecTl32m1S+U09TU6qrERHpPRkT9DOGzgBz2gev59VXU12NiEjvyZygL58R3Bm6huXLU1uLiEhvypigH1s8lqKcIgomruGll1JdjYhI78mYoDczTi8/ndwxCnoRySwZE/QQDN8cyltL1fZ2qqpSXY2ISO/IqKCfNWIWzRyC8rUapxeRjJFRQX/R+IsAiEx+WsM3IpIxMirohxUOY0b5DPJnPM0LL6S6GhGR3pFRQQ9wyYRLOFTyAivW1FFbm+pqRER6XuYF/cRLaLdWfMxzPP98qqsREel5GRf0c0bNIS8rj/Dkp3n22VRXIyLS8zIu6HMiOVww7gKypz2loBeRjHDMoDeze81st5mt72b+DWa2Nja9aGYz4uZtNbN1ZrbazFYms/CTccmES2gc8BavVb/Fzp2prkZEpGclckR/HzD/KPO3APPc/XTg/wCLOs0/391nunvFiZWYfJdMuCS4M1HDNyKS/o4Z9O6+DNh/lPkvuntN7OFyYGSSausxEwdNZHzxeHKmPc0jj6S6GhGRnpXsMfpPA7+Pe+zAUjNbZWYLj7agmS00s5VmtnLPnj1JLus9zxWcfTP2jzzxVItOsxSRtJa0oDez8wmC/mtxzXPc/UzgUuALZja3u+XdfZG7V7h7RVlZWbLK6tYlEy6h1epoKX+Rxx/v8acTEUmZpAS9mZ0O/By42t33dbS7e3XsdjfwGDArGc+XDBeMu4DscDZFsx7ngQdSXY2ISM856aA3s9HAo8DH3X1TXHu+mRV23AcuBro8cycVCnMKuXzS5bRNfYhnnm2jh0eLRERSJpHTKx8AXgImm1mVmX3azG42s5tjXW4HBgM/7XQaZTnwgpmtAV4GnnD3p3rgNZywj07/KPW2k/ZRz/Pww6muRkSkZ5i7p7qG96ioqPCVK3v+tPvG1kbK7yon/MZ1nLb55/zpTz3+lCIiPcLMVnV3GnvGfTM23oCsAXxo6odoGv+fvPBSM9u2pboiEZHky+igB/joaR+liVqY9HseeijV1YiIJF/GB/0Hx3+QsrwySuYuZvHiVFcjIpJ8GR/0kVCE6069jrrhv+PV1w6xenWqKxIRSa6MD3oIzr5ppYnwaY9z772prkZEJLkU9MDskbMZWzyWsgt+w3/8BzQ1pboiEZHkUdATXPvmxtNvZFfBUmp4S5dEEJG0oqCP+WzFZwmHwhRdeDf33JPqakREkkdBHzO8cDgfmfYRWk69l+dfqmPNmlRXJCKSHAr6OH8z629oopasil/x4x+nuhoRkeRQ0MeZPXI2s0bMIveCu/iPB1rZuzfVFYmInDwFfRwz4+/P+3sORbbSPOkBfvjDVFckInLyFPSdXHHKFcwon0HhZf/MD3/UpqN6Een3FPSdmBnfPO+bHMrZSP24B/je91JdkYjIyVHQd+HaaddyxtAzyL/y7/nRT5vZvTvVFYmInDgFfRdCFuKOC++gPuttGk/9GXfemeqKREROnIK+GxdNuIiLxl9E9oX/xE8W1bNrV6orEhE5MQr6o/jOB75Dc3gfzaf+nDvuSHU1IiInJpHfjL3XzHabWZc/7G2BH5lZpZmtNbMz4+bNN7ONsXm3JbPw3jB71GzmjplL3ge/x0//rYUdO1JdkYjI8UvkiP4+YP5R5l8KTIpNC4GfAZhZGLg7Nn8acL2ZTTuZYlPhtjm3UR/ZRsvkxXz3u6muRkTk+B0z6N19GbD/KF2uBu73wHKg2MyGAbOASnff7O4twIOxvv3K/InzOWPoGeRd/g/c84sGqqpSXZGIyPFJxhj9CCD+Z7WrYm3dtXfJzBaa2UozW7lnz54klJUcZsYPLvkB9ZF3iM7+LrffnuqKRESOTzKC3rpo86O0d8ndF7l7hbtXlJWVJaGs5Jk3dh4LTluAzflXfvn4ZlatSnVFIiKJS0bQVwGj4h6PBKqP0t4v3XnRneRkhcm+8st86Uvg3e6yRET6lmQE/RLgxtjZN2cDte6+A1gBTDKzcWaWDSyI9e2XRhaN5FvzvkXL+P/iT9VP8+ijqa5IRCQxiZxe+QDwEjDZzKrM7NNmdrOZ3Rzr8iSwGagE/h34PIC7R4FbgKeBDcBv3f21HngNveZLZ3+JSYMmkX3N/+arX2vRb8uKSL9g3gfHICoqKnzlypWpLqNLT775JJcvvhyW/ivfvepv+drXUl2RiAiY2Sp3r+hqnr4Ze5wum3QZV5xyBeEP/iPf+X61TrcUkT5PQX8CfnDJDwhntdB83tf56ldTXY2IyNEp6E/AxEETufXsW/Hpv+ahZzfwzDOprkhEpHsK+hP0d3P+jvzsfAqu/Daf+hTU1KS6IhGRrinoT1BpXim3nnUrdWN+y472tXzuczq3XkT6JgX9SfjK7K9QklvCqIX/m4cechYvTnVFIiLvpaA/CSUDSrjjwjvYyvNM+sj9fP7z8Pbbqa5KROTdFPQn6dNnfppzRp3D3vd9hbbc3dx4I7S1pboqEZEjFPQnKWQh/u2Kf6MhWsfEr3yaZcucu+5KdVUiIkco6JPgtCGncedFd7Km8b+ZufBuvvUteOWVVFclIhJQ0CfJLbNu4fJJl7N+5JcoPPMJPvYxaGxMdVUiIgr6pDEzFl+7mBnlM6i7/CNsqPuzroMjIn2Cgj6JinKKeOpjTzFq4AgGfuJGfnxPA088keqqRCTTKeiTrDSvlF9c9Qtqw5sp+6tv84lPwPbtqa5KRDKZgr4HzBs7j78+46/ZP/n7HCp5gRtugNbWVFclIplKQd9D7rz4TiYMmkDWx6/h+XWVfPnLqa5IRDKVgr6HFOcW88RHnyAnB0q+cAU/+fdaFi1KdVUikokU9D1o4qCJPPq/HuVgpJLyz36Cz3+hneefT3VVIpJpEgp6M5tvZhvNrNLMbuti/t+a2erYtN7M2sxsUGzeVjNbF5vXN38fsAfNHTOXuy6+i12DHqfkqn/m2mvhzTdTXZWIZJJEfhw8DNwNXApMA643s2nxfdz9Tnef6e4zga8Dz7v7/rgu58fmd/l7hunu1rNu5eOnf5y9p3+L5mm/ZN482LAh1VWJSKZI5Ih+FlDp7pvdvQV4ELj6KP2vBx5IRnHpwsz4+VU/5+IJF9N44WdoGP9b5s2DtWtTXZmIZIJEgn4EsC3ucVWs7T3MLA+YDzwS1+zAUjNbZWYLu3sSM1toZivNbOWePXsSKKt/yQ5n88h1jzB71GwOXrSAlhk/4/zzYdWqVFcmIukukaC3Ltq6+y2lK4E/dxq2mePuZxIM/XzBzOZ2taC7L3L3CnevKCsrS6Cs/qcgu4CnP/Y0l59yObXnfh7e/zMuvFAXQBORnpVI0FcBo+IejwSqu+m7gE7DNu5eHbvdDTxGMBSUsfKy8njkuke44pQr2D/784Tf90suvBBefjnVlYlIukok6FcAk8xsnJllE4T5ks6dzGwgMA/4r7i2fDMr7LgPXAysT0bh/Vl2OJuH/+phLp5wMfvO+xSccxdz58JvfpPqykQkHR0z6N09CtwCPA1sAH7r7q+Z2c1mdnNc1w8BS929Pq6tHHjBzNYALwNPuPtTySu//8qN5LJkwRKuO/U6at7/twxd8G0+9jH4u7/TL1SJSHKZe3fD7alTUVHhK1dmxin37d7OXy/5a365+pfMOfRD/vy9W7n0Uli8GIqLU12diPQXZraqu1PY9c3YFAtZiEVXLuKaKdfw58IvUvHd61m6vIqzz4Y33kh1dSKSDhT0fUAkFOHBax/k9rm3s671MXK/fCrVgx6kogJ+9atUVyci/Z2Cvo/IieTwnfO/w+tfeJ3pw6Zx6JLrKfjYJ/nEwjpuvBEOHUp1hSLSXyno+5jxJeNZ9ollfPO8b7J72K8Y9I2Z/HrHN5h80Qv89rfQBz9SEZE+TkHfB2WFs/inC/6JP970RyYMH0T4vDvZMX8u/+ufFnPWWfA//5PqCkWkP1HQ92EfGPsBXv7Myxy4rYZ5Y+cRuvZGKgcu4vwL2rjsMl0rR0QSo6DvBwqyC/jd9Us4d8wcas79LGXfPpXn9z3IjJntXHMNLFumIR0R6Z6Cvp8ozCnkuZue45HrHqG8LIuGy66n7PbpPLPnPuZd0MLkyfAv/wLV3V2cQkQyloK+HwlZiA9P/TBrbl7DA9c+wNDyMA0Xf5Lib48jetYdfONfqhk1CuZfVcdvHq6jpSXVFYtIX6BvxvZj7s7St5Zy10t38YfNfwCgyEdxkCpoySfvldu4qvyLzD07n9mz4bTTIBJJcdEi0iOO9s1YBX2aeH3P6yzZuITVO1czZfA0lq5dzUsHHsOaSvBXPgVvXcyAQ6dRcXo+584q4pzZxtlnQ2lpqisXkWRQ0Geo5VXL+f5L3+fRDY/S5nFXSmssgaqzYdtshvN+Zk2cwAcrxnDeOdlMngy5uamrWUROjII+w9U21bKiegWb9m2iobWB9Tvf4Pm3lvN2w+t4x2/IuMHBEXBgHAOax1EaHsewwmEMK81nyJB2hgxtY+7Uacw9ZQa5kSN7gp11O9mwZwPnjDqHnEhOwjUdbD5IViiLAVkDkv1yRTKSgl66dKDpAGt3rWVzzRZe3bKFV7du4Z2DW9gT3UJDeDtYF38bbRGyaqYzoGU04ZxGDgz6I25R8kLFnDfoOuaPuo7a0BbealjFGSOmMWP4qZTmldLY2sjOup3sqt/FsreX8fDrDzMgMoBPzvwkN828iRnlM2htb2Vfwz7qW+sZWzyWSOjYHyhUH6rmofUPceH4C5lePv2or3XNzjUU5RQxrWzace2UOmuKNvFU5VNcNP4i8rPzT3g9IvFe2fEKL257kVtm3XJCyyvo5bg1R5vZ27CXmvp6dmwPs/VteLFyLa/VrGQ7K6hjN9FoO+2bLqGpcjZMeQymPgrZDcEKWvIhu77LdWe3FzHdP057zn7WRh+mjSj5oRIa2g8cfocxKGcIF4++ioH5eVi4lTaP0treSrQ9SmtbK+3eTrQ9ylOVT9EYbSRkIa6dei0AWw9sZdO+TYwoGsHEQRN5fc/rVO6vPPz8kVCEaWXTmDRoEpFQhJLcEsYUj2H0wNHkRnL5S9Vf2Nuwl4LsAiprKtmwZwNN0SaGFgzlw1M/zOJ1i9mwdwOjB47mG+d+g9K8UiKhyOGppqmG/9n6P0RCET489cO8U/sOz219js01mynILuAzZ36GopwiVmxfwYrqFexp2MOM8hlUDK9g8uDJ/HHLH3m5+mWG5g+lZEAJhjFz6EymlE7h/jX3s273Osryy5hWOo2zR57N6IGjaWht4Nktz5IdzmZG+QweXP8gK6pX8Nn3fZY5o+ewvGo5NY01NLQ2sGn/JiIW1LZx30Z+t+l3TC2dypnDzgSgMLuQ4YXDaYw20hxt5pTBp1CUU8S2g9t45PVHeG7rc4wpHsO00mlMKZ3CgaYDbNi7gYbWBrJCWYwaOIpttdtYvn05pww+hbmj51KWX8bAnIFEQhEWr1vM0s1LuXD8hcyfMJ8BWQPIDmeTFcoKbsPBbVO0ibqWOk4tO5XhhcPZuG8jhnHK4FNobmvm9T2vM3jAYDbu28iiVYvIDmfzvmHvY+WOleyu380nZ36S7HA2T1U+xcRBEzln1DlE26O8uO1Ffrfpd0wcNJEbpt/A5MGTaYo28VLVS+xr2Ec4FGZq6VTKC8pZs3MNb9W8xa76XUwfMp0Lxl3AxEETyc/KpynaxJpda9i0bxOzRsxiQskEqg5WcajlEG3tbeREchgQGUBuJJcXt73If7/534wrHsecUXMIh8L8YfMf+PXaX3PWiLO4avJVfO6Jz1GaV8q6z62jILvguP/PKuilx7hDTQ3s3w/b99Tx3NvPkt8yjoL66Wzdv53NtW+yp24fDQcH0Lx3KHW7hlJbPYSavVm0twN5e2HqIzB8VTB0VF8ObVkw8SmY8EzwJG1ZGBFCnoV5hBBZhCyEGQxuPIvTGv6G7WW/ZHPuf5IXGsigyCjKI6dwoH0be9oqGZ03jSlF72N09hmEcg+xk9VsrF3N9vqttHkbNU17qWk+8jPH2eFsSvNKOdR8iLHFY5lePp28SB5rd6/l5e0vM6poFF8/9+v8dOVPWb+76x9MK8opItoepaE12PENyR/ClNIpbD2wlXdq3zncb9KgSQzJH8KaXWuoa6k73D6ldAp7G/ZysPng4Z0aBDupGeUz2N+4ny0HtnT775IdzmZ8yXje2Pvea12PKBxBXUsdtc21AJxadipv1779rufvTtjCzB41m111u3ir5i3avR0Idg6FOYU0R5vZ17iPguwCZo2YxYY9G9hRt+Nd6xgQGcAF4y5g2dvLONSS2NX68rPyqW8NDhzGDBzD7vrdNEYbD88fkj+EkIXYWbeT8vxyCnMKD+/cB+YMPPxaAQzjrJFnsWnfJvY37udYskJZDM4bzM66nYfbQhY6/Nrj1+vd/pw2FOcWU9tUe7iPYZw/7nxe3PYiTdEmZpTP4MkbnmR44fAEtsh7Keilz2lvh4MHgx1Eezs0NMCmTdDYCCUl0NICBw4cmWpqgj6Nje+e6uqOzK+tDdZ1QrLroGgb5ByiqPF0BhXlEolAOByckhqJQE4OhAe9Q3F2KcX5eeQVRGktrCQ3r5VITpRQJEooHGVAdg7j808nnNPEptZnGZY3mkmFM8nKMsJZUdY3PkNBXoT3D68gq62E1lbIy29nV3QT7zS+xjlj38+Y4tGHS2trb+Pl7S+zbvc6rjzlSoYVDgNgf+N+VlWvYmfdTsyM88eeT0tbCyurVzJn9ByGFQzj95W/p/pQNXNGzWFY4TBywjkMyBpAc7SZ57Y+x4jCEUwvn05rWyvVh6oJWYja5lqqD1WTn5VPJBThjb1vUN9aT3l+OeeOPpfygnIgGMKq3F/JwJyBjCwaiZkBUN9Sf/jI3N2pPlTNgaYD1DbXcqj5EBXDKxicN5iG1gbe2v8Wre2ttLS10NoW3HZMHUfEr+58lcr9lZw57Eyaok08s/kZRhWN4tzR53Kw+SCF2YVcM+UassPZ7KrfxZD8IQA8v/V5Qhbi3NHnsrdhL2t2rWFAZAATBk1geOFwmqPNvLjtRaoOVhGyEGePPJsRRSNoaWth3a517K7fzenlpzOuZBwhC/FO7Tv8+Z0/H94p5mXlMbV0KhMHTeSlqpeoOljFmIFjKBlQQshCNEebaYw20tjayIRBE7hg3AWHh0sBJpRMYEzxGLbUbOE/X/9PFr5vIQNzB57gH7CCXjJEe3twOeempuB+ezu0tgY7gqYmyMoK7u/YEexI2tuDn22Mnxobg28X19YGj6PRYGprC9ZRXx/sXOrq3n0/mf+NwmEoKIDCQsjLC+rOzk78tnNbQUFwGq1Z8Bqam4Md15AhwfpDoWAKh4OdWW5ucNuxgwuHg3UNGBBMOTlHtl9urr6b0VccLegT+icys/nA/wPCwM/d/bud5n+A4EfBO95LPuru/5jIsiLJEgrBwIHB1Jvcgx1Kx9TSEoRpU1Ow44hGg/Zo9MjOor4+mNcR5HV1wU6q47bjfkPDkXXG39bXd93e+ba1tedffzgcBH5ubrBDCIeDqWMHcrSpuBimTw92ao2NwfaJRoMdSl5ecJuVdWSd8VPHTqhj6thJdeyQOu53nC7csTPPygreNWZldf1asrKCdcfeoKSFYwa9mYWBu4GLgKXUTB0AAAmHSURBVCpghZktcffXO3X9k7tfcYLLivRbZkeOpPsa92CHsXdvUGdHGLa2wq5d7373E40e2UE1Nx95JxONBjuOjuGy5ubgtYZCQd/4qbn5yPra2oLn73jc1bR7N/ziF8F6OwI6HA4eNzQEz50qkciRd0Xx75Dip44+hYVQVBRM4fCRnYpZ0BYKBdsmFDryTqljil9/URF89rM98FoS6DMLqHT3zQBm9iBwNZBIWJ/MsiJyksyCECosfO+8IUN6v56uuHd/9NzxTqjzEFvHFD8v/l1Ux7uDjls4cuTf3BwM4bW1vff52tre/W4o/n7nxx3v1Fpagndg27cHnzu1tR15ro7hxPb2YCfrfmQ4sGP5+DqGDk1d0I8AtsU9rgLO6qLfbDNbA1QDX3X3145jWcxsIbAQYPTo0V11EZE0dLQhko6j3XTW8W6qY+fRExK5emVX/wydP3p6BRjj7jOAHwOPH8eyQaP7InevcPeKsrKyBMoSEen/QqFg2CY/v+c+X0ok6KuAUXGPRxIctR/m7gfdvS52/0kgy8xKE1lWRER6ViJBvwKYZGbjzCwbWAAsie9gZkMtdhKtmc2KrXdfIsuKiEjPOuYYvbtHzewW4GmCUyTvdffXzOzm2Px7gI8AnzOzKNAILPDgBP0ul+2h1yIiIl3QF6ZERNLA0b4wpZ8SFBFJcwp6EZE0p6AXEUlzCnoRkTSnoBcRSXMKehGRNKegFxFJcwp6EZE0p6AXEUlzCnoRkTSnoBcRSXMKehGRNKegFxFJcwp6EZE0p6AXEUlzCnoRkTSnoBcRSXMKehGRNJdQ0JvZfDPbaGaVZnZbF/NvMLO1selFM5sRN2+rma0zs9Vmpt8HFBHpZcf8cXAzCwN3AxcBVcAKM1vi7q/HddsCzHP3GjO7FFgEnBU3/3x335vEukVEJEGJHNHPAirdfbO7twAPAlfHd3D3F929JvZwOTAyuWWKiMiJSiToRwDb4h5Xxdq682ng93GPHVhqZqvMbOHxlygiIifjmEM3gHXR5l12NDufIOjPjWue4+7VZjYEeMbM3nD3ZV0suxBYCDB69OgEyhIRkUQkckRfBYyKezwSqO7cycxOB34OXO3u+zra3b06drsbeIxgKOg93H2Ru1e4e0VZWVnir0BERI4qkaBfAUwys3Fmlg0sAJbEdzCz0cCjwMfdfVNce76ZFXbcBy4G1iereBERObZjDt24e9TMbgGeBsLAve7+mpndHJt/D3A7MBj4qZkBRN29AigHHou1RYDF7v5Uj7wSERHpkrl3OdyeUhUVFb5ypU65FxFJlJmtih1gv4e+GSsikuYU9CIiaU5BLyKS5hT0IiJpTkEvIpLmFPQiImlOQS8ikuYU9CIiaU5BLyKS5hT0IiJpTkEvIpLmFPQiImlOQS8ikuYU9CIiaU5BLyKS5hT0IiJpTkEvIpLmFPQiImlOQS8ikuYSCnozm29mG82s0sxu62K+mdmPYvPXmtmZiS4rIiI965hBb2Zh4G7gUmAacL2ZTevU7VJgUmxaCPzsOJYVEZEelMgR/Syg0t03u3sL8CBwdac+VwP3e2A5UGxmwxJcVkREelAkgT4jgG1xj6uAsxLoMyLBZQEws4UE7wYA6sxsYwK1daUU2HuCy/am/lIn9J9a+0ud0H9q7S91Qv+ptafqHNPdjESC3rpo8wT7JLJs0Oi+CFiUQD1HZWYr3b3iZNfT0/pLndB/au0vdUL/qbW/1An9p9ZU1JlI0FcBo+IejwSqE+yTncCyIiLSgxIZo18BTDKzcWaWDSwAlnTqswS4MXb2zdlArbvvSHBZERHpQcc8onf3qJndAjwNhIF73f01M7s5Nv8e4EngMqASaAA+ebRle+SVHHHSwz+9pL/UCf2n1v5SJ/SfWvtLndB/au31Os29yyFzERFJE/pmrIhImlPQi4ikubQJ+r58qQUzG2Vmz5nZBjN7zcxujbV/28y2m9nq2HRZH6h1q5mti9WzMtY2yMyeMbM3Y7clfaDOyXHbbbWZHTSzL/aFbWpm95rZbjNbH9fW7TY0s6/H/m43mtklfaDWO83sjdjlTB4zs+JY+1gza4zbtvekuM5u/6374DZ9KK7OrWa2OtbeO9vU3fv9RPBB71vAeIJTOtcA01JdV1x9w4AzY/cLgU0El4T4NvDVVNfXqdatQGmntn8Fbovdvw24I9V1dvHvv5PgCyMp36bAXOBMYP2xtmHs72ANkAOMi/0dh1Nc68VAJHb/jrhax8b36wPbtMt/6764TTvN/x5we29u03Q5ou/Tl1pw9x3u/krs/iFgA8G3hvuLq4Ffxe7/CrgmhbV05YPAW+7+dqoLAXD3ZcD+Ts3dbcOrgQfdvdndtxCcuTarVwql61rdfam7R2MPlxN8/yWlutmm3elz27SDmRlwHfBAb9UD6TN0090lGPocMxsLnAH8JdZ0S+wt8r19YUiE4JvLS81sVeyyFADlHnwvgtjtkJRV17UFvPs/Tl/bptD9Nuzrf7ufAn4f93icmb1qZs+b2XmpKipOV//WfXmbngfscvc349p6fJumS9AnfKmFVDKzAuAR4IvufpDgKp8TgJnADoK3dKk2x93PJLji6BfMbG6qCzqa2BfxrgIejjX1xW16NH32b9fMvglEgd/EmnYAo939DODLwGIzK0pVfXT/b91ntylwPe8+KOmVbZouQZ/IZRpSysyyCEL+N+7+KIC773L3NndvB/6dXnx72R13r47d7gYeI6hplwVXIyV2uzt1Fb7HpcAr7r4L+uY2jeluG/bJv10zuwm4ArjBY4PJsaGQfbH7qwjGvk9JVY1H+bfuq9s0AnwYeKijrbe2aboEfZ++1EJsXO4XwAZ3/35c+7C4bh8C1ndetjeZWb6ZFXbcJ/hQbj3Btrwp1u0m4L9SU2GX3nWE1Ne2aZzutuESYIGZ5ZjZOILfdHg5BfUdZmbzga8BV7l7Q1x7mQW/MYGZjSeodXNqqjzqv3Wf26YxFwJvuHtVR0OvbdPe+iS6pyeCSzBsItgjfjPV9XSq7VyCt45rgdWx6TLg18C6WPsSYFiK6xxPcLbCGuC1ju0IDAaeBd6M3Q5K9TaN1ZUH7AMGxrWlfJsS7Hh2AK0ER5efPto2BL4Z+7vdCFzaB2qtJBjj7vhbvSfW99rY38Ua4BXgyhTX2e2/dV/bprH2+4CbO/XtlW2qSyCIiKS5dBm6ERGRbijoRUTSnIJeRCTNKehFRNKcgl5EJM0p6EVE0pyCXkQkzf1/9lJSjpuFEf8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_metrics('wine_quality_root_mean_squared_error', 'RMSE', ylim=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "QIAxEezCppnd"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEICAYAAABWJCMKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deZRc9X3n/fenq6tXSd1Cai1IAgksIDLYGGTw8Zbx4AWIY3mJZ8Q4hiTOEJIQDzNxjknyJMM5ceaxGS+x/XiMsU0ME9t4xSgTeQDjxMa7GswiFoEQElpa+9qLev0+f9xbrVJR3V29Vnf153XOPXXrrt+6iPr073eXUkRgZmazT1W5CzAzs/JwAJiZzVIOADOzWcoBYGY2SzkAzMxmKQeAmdks5QCwGU3SGyRtKXcdZjORA8CmFUl/KWljwbTnhpi2PiIeiojzJ7iGsyS15w0hqSPv/Rsmcn9D1HCLpH+a7P3Y7FZd7gLMCvwYuFlSJiL6JS0BssAlBdNeli474SLiRWBO7r2kAF4ZEVsnY39m5eIWgE03m0i+8C9O378R+FdgS8G05yNij6R/J2lXbmVJ2yV9SNLjko5J+oakurz5b5f0qKSjkn4m6RWlFiZpiaROSQvypl0q6YCkrKTfk/RTSZ9N9/2MpCvylm2S9GVJbZJ2S/qIpMxoD5Ckd0h6Mv0M/ybpN/LmfTjd9glJW3L7l3SZpFZJxyXtk/TJ0e7XKo8DwKaViOgBfknyJU/6+hDwk4Jpw/31/x+AK4FVwCuA3wOQdAlwB/BHwALgC8AGSbUl1rYX+Ld0+zm/C9wdEb3p+8uBbcBC4L8D35V0RjrvTqCPpPXyKuCtwB+Wsu8cSecBXwduAlqAjcA/S6qRdD5wI/DqiJgLvA3Ynq76aeDTETEPOBf45mj2a5XJAWDT0Y849WX/BpIAeKhg2o+GWf8zEbEnIg4D/8yplsN/Br4QEb+MiP6IuBPoBl4zitruJPnSJ/3r/Rrgf+fN3w/8Q0T0RsQ3SFouvyVpMXAVcFNEdETEfuBTwPpR7BvgPwL/EhEPpKHzcaAeeC3QD9QCayRlI2J7RDyfrtcLvEzSwohoj4hfjHK/VoEcADYd/Rh4vaT5QEtEPAf8DHhtOu1Chm8B7M0b7+RUf/7ZwJ+nXSdHJR0FVgBnjqK2e0m+YM8B3gIci4hf5c3fHac/YXFHuv2zSbq22vL2/QVg0Sj2TbqtHbk3ETEA7ASWpecobgJuAfZLultS7rN9ADgPeEbSJklvH+V+rQI5AGw6+jnQBFwP/BQgIo4De9JpeyLihTFsdyfw9xHRnDc0RMTXS91ARJwk6T55H/B+Tv/rH2CZJOW9PyuteydJa2Nh3r7nRcTLR/kZ9pCECQDpvlYAu9P6vhYRr0+XCeBj6fTnIuIaksD5GPBtSY2j3LdVGAeATTsR0QW0Av+NpOsn5yfptLFe/fNF4AZJlyvRKOm3JM0d5XbuIjmv8A6g8FLNRcAH05PC7wV+A9gYEW3A/cAnJM2TVCXpXEm/Ocx+qiTV5Q21JOHzW5KukJQF/pwkWH4m6XxJ/z5d7iTQRdIthKTfldSSthiOptvvH+XntgrjALDp6kckX6Y/yZv2UDptTAEQEa0k5wH+P+AIsJX0BPEot/NTYAB4JCK2F8z+JbAaOAj8PfA7EXEonXctUAM8le7/28DSYXZ1DcmXeG54PiK2kJyD+Gy6j98Gfjs9eV4LfDSdvpfkWP1Vuq0rgScltZOcEF6ftmZsFpN/EMZs9CT9EPhaRHwpb9rvAX+YdsGYTXu+EcxslCS9GrgEWFfuWszGo6QuIElXpjeVbJV0c5H570tvvHk8vbnmlSOtK+kMSQ8ouaX/gfTqDrNpTdKdwA9ILuc8Ue56zMZjxC6g9FrnZ0kuedtFcqfmNRHxVN4yrwWejogjkq4CbomIy4dbV9KtwOGI+GgaDPMj4sOT8BnNzKyIUloAlwFbI2JbeqLpbgqavhHxs4g4kr79BbC8hHXXkdxUQ/r6zrF/DDMzG61SzgEsI7mGOWcXye3uQ/kA8P0S1l2cXhpHRLRJKnpDjKTrSa79prGx8dILLrighJLH5pG2R4gTi3j5Wcupqxt5eTOzmeDhhx8+GBEthdNLCQAVmVa030jSm0gCIHcVRMnrDiUibgduB1i7dm20traOZvVRmfORZjp+up47P/IPXHrppO3GzGxKSdpRbHopXUC7SO40zFlOcjdi4Q5eAXwJWJd33fNw6+6TtDRddynJM1TKKltVC9Un6eoqdyVmZpOvlADYBKyWtEpSDcnDqzbkLyDpLOC7wPsj4tkS190AXJeOX0fyjJWyqs3UQabbAWBms8KIXUAR0SfpRuA+IAPcERFPSrohnX8b8Lckj9f9X+ljUPoiYu1Q66ab/ijwTUkfAF4E3jvBn23UajN1bgGY2axR0o1gEbGR5Lnj+dNuyxv/Q4Z4rnmxddPph4ArXrpG+dRVuwvIzGYPPwsoT111HVS7C8jMZgcHQJ66rFsAZjZ7OADyNNT4HICZzR4OgDwNNb4KyMxmDwdAnnp3AZnZLOIAyFNXXYdqTtLZWe5KzMwmnwMgT111HfJVQGY2SzgA8tRm3AVkZrOHAyBPXXUd4ZPAZjZLOADy1FbXEhm3AMxsdnAA5KmrroOqPjq7+stdipnZpHMA5KmrTn4FpqO7u8yVmJlNPgdAntpMLQCdPSfLXImZ2eRzAOTJtQAcAGY2GzgA8uQCoKvPZ4HNrPI5API01jQC0NXrW4HNrPI5API0ZBsAONnvADCzyucAyDMYAAMOADOrfCUFgKQrJW2RtFXSzUXmXyDp55K6JX0ob/r5kh7NG45Luimdd4uk3Xnzrp64jzU2jdmkC2igqpO+vjIXY2Y2yUb8TWBJGeBzwFuAXcAmSRsi4qm8xQ4DHwTemb9uRGwBLs7bzm7gnrxFPhURHx/XJ5hAuRYANR10dcHcueWtx8xsMpXSArgM2BoR2yKiB7gbWJe/QETsj4hNQO8w27kCeD4idoy52kk2GADZTj8OwswqXikBsAzYmfd+VzpttNYDXy+YdqOkxyXdIWn+GLY5oRwAZjablBIAKjItRrMTSTXAO4Bv5U3+PHAuSRdRG/CJIda9XlKrpNYDBw6MZrejlrsMlGyHA8DMKl4pAbALWJH3fjmwZ5T7uQp4JCL25SZExL6I6I+IAeCLJF1NLxERt0fE2ohY29LSMsrdjk59dX0y4haAmc0CpQTAJmC1pFXpX/LrgQ2j3M81FHT/SFqa9/ZdwOZRbnPCZaoyZFXrADCzWWHEq4Aiok/SjcB9QAa4IyKelHRDOv82SUuAVmAeMJBe6rkmIo5LaiC5guiPCjZ9q6SLSbqTtheZXxb1mUZ6azr8u8BmVvFGDACAiNgIbCyYdlve+F6SrqFi63YCC4pMf/+oKp0iddUNHHcLwMxmAd8JXKChusFdQGY2KzgACjTWNPoqIDObFRwABRqybgGY2ezgACgwp9YBYGazgwOgwNzaRgeAmc0KDoACc2obBh8GZ2ZWyRwABRqyDajGLQAzq3wOgAI+CWxms4UDoEBjTSOR9Z3AZlb5HAAFGrINUNVHR9dwP21gZjbzOQAK5H4ToL3bTQAzq2wOgAK53wVu7+0ocyVmZpPLAVAg1wLo7HELwMwqmwOgwGAA9DoAzKyyOQAK5H4WstNdQGZW4RwABXItgA63AMyswjkACrgLyMxmCwdAgVwAdPV1ElHmYszMJpEDoEDuMtCo7uDkyTIXY2Y2iUoKAElXStoiaaukm4vMv0DSzyV1S/pQwbztkp6Q9Kik1rzpZ0h6QNJz6ev88X+c8cu1AMh2cuJEeWsxM5tMIwaApAzwOeAqYA1wjaQ1BYsdBj4IfHyIzbwpIi6OiLV5024GHoyI1cCD6fuyyw+A9vby1mJmNplKaQFcBmyNiG0R0QPcDazLXyAi9kfEJmA0D9BZB9yZjt8JvHMU606a+mx9MpLtcAvAzCpaKQGwDNiZ935XOq1UAdwv6WFJ1+dNXxwRbQDp66JiK0u6XlKrpNYDBw6MYrdjU6Uqaqvq3QVkZhWvlABQkWmjuT7mdRFxCUkX0p9KeuMo1iUibo+ItRGxtqWlZTSrjlldpsFdQGZW8UoJgF3Airz3y4E9pe4gIvakr/uBe0i6lAD2SVoKkL7uL3Wbk60h2wg17gIys8pWSgBsAlZLWiWpBlgPbChl45IaJc3NjQNvBTanszcA16Xj1wH3jqbwyZT7VTAHgJlVsuqRFoiIPkk3AvcBGeCOiHhS0g3p/NskLQFagXnAgKSbSK4YWgjcIym3r69FxP9NN/1R4JuSPgC8CLx3Yj/a2DXWOADMrPKNGAAAEbER2Fgw7ba88b0kXUOFjgOvHGKbh4ArSq50Cs2ra4Rsh88BmFlF853ARTTWNKAatwDMrLI5AIporGmkqq7dAWBmFc0BUMS82nlQd9xdQGZW0RwARTTVNhE1x9wCMLOK5gAooqm2iYHsCY6f6C93KWZmk8YBUERTXRMAx066CWBmlcsBUERTbRoA3cfKXImZ2eRxABSRawGc6HEAmFnlcgAUkWsBdPQ7AMyscjkAimiuawag0wFgZhXMAVBErguoP3uU7u4yF2NmNkkcAEXkuoCoPeabwcysYjkAisi1AKjzzWBmVrkcAEXUVddRrRqodQCYWeVyAAyhMdPkFoCZVTQHwBDmZpt8DsDMKpoDYAjzat0CMLPK5gAYQlNdk88BmFlFKykAJF0paYukrZJuLjL/Akk/l9Qt6UN501dI+ldJT0t6UtJ/yZt3i6Tdkh5Nh6sn5iNNjPn1SQvAXUBmVqlG/E1gSRngc8BbgF3AJkkbIuKpvMUOAx8E3lmweh/w5xHxiKS5wMOSHshb91MR8fFxf4pJsKDRLQAzq2yltAAuA7ZGxLaI6AHuBtblLxAR+yNiE9BbML0tIh5Jx08ATwPLJqTySTa/MWkBHD9e7krMzCZHKQGwDNiZ934XY/gSl7QSeBXwy7zJN0p6XNIdkuYPsd71kloltR44cGC0ux2z5tomqD3BoSP+URgzq0ylBICKTIvR7ETSHOA7wE0Rkfub+vPAucDFQBvwiWLrRsTtEbE2Ita2tLSMZrfjkrsbeP9R9wGZWWUqJQB2ASvy3i8H9pS6A0lZki//r0bEd3PTI2JfRPRHxADwRZKupmkj9zygg+1+IqiZVaZSAmATsFrSKkk1wHpgQykblyTgy8DTEfHJgnlL896+C9hcWslTI9cCONThADCzyjTiVUAR0SfpRuA+IAPcERFPSrohnX+bpCVAKzAPGJB0E7AGeAXwfuAJSY+mm/yriNgI3CrpYpLupO3AH03sRxufXAvgaKcDwMwq04gBAJB+YW8smHZb3vhekq6hQj+h+DkEIuL9pZc59QZ/GN6/C2xmFcp3Ag8h1wLoGjhGb+8IC5uZzUAOgCHkfhaSumMcPVreWszMJoMDYAiDPwpTe4zDh8tbi5nZZHAADKGuuo6saqH+iAPAzCqSA2AYzTULof6QA8DMKpIDYBgLGhZCw0EHgJlVJAfAMBY1JgFw5Ei5KzEzm3gOgGEsmecWgJlVLgfAMFoaF6JGB4CZVSYHwDAWNiwk6o5w8HBfuUsxM5twDoBhLGxYCAr2HfdJADOrPA6AYSxsWAjAwY6DZa7EzGziOQCGkQuAwycdAGZWeRwAw8gFwNFeB4CZVR4HwDByAdAxcJCBgTIXY2Y2wRwAw1hQvwCAqD/I8eMjLGxmNsM4AIZRn62nVo2+GczMKpIDYARNWd8NbGaVqaQAkHSlpC2Stkq6ucj8CyT9XFK3pA+Vsq6kMyQ9IOm59HX++D/OxDujLgmA/fvLXYmZ2cQaMQAkZYDPAVeR/ND7NZLWFCx2GPgg8PFRrHsz8GBErAYeTN9PO4vmJAHQ1lbuSszMJlYpLYDLgK0RsS0ieoC7gXX5C0TE/ojYBBT+eu5w664D7kzH7wTeOcbPMKnObE4CYO/ecldiZjaxSgmAZcDOvPe70mmlGG7dxRHRBpC+Liq2AUnXS2qV1HrgwIESdztxFs9dCI1uAZhZ5SklAFRkWpS4/fGsmywccXtErI2ItS0tLaNZdUIsbFgItcfZs69nyvdtZjaZSgmAXcCKvPfLgT0lbn+4dfdJWgqQvk7L06y5m8F2HTpU5krMzCZWKQGwCVgtaZWkGmA9sKHE7Q+37gbgunT8OuDe0sueOrkAaDvmx0GYWWWpHmmBiOiTdCNwH5AB7oiIJyXdkM6/TdISoBWYBwxIuglYExHHi62bbvqjwDclfQB4EXjvRH+4ibC4cTEAB062EXERKtapZWY2A40YAAARsRHYWDDttrzxvSTdOyWtm04/BFwxmmLLYfm85GN11+ymvR3mzi1zQWZmE8R3Ao/gzLlnJiNNO30lkJlVFAfACGqra5mfXQzzdvleADOrKA6AEixtXA7zdjoAzKyiOABKsHL+Cpi3y11AZlZRHAAlWLVwOTS5BWBmlcUBUIKzmlZA3TFe3Hui3KWYmU0YB0AJcpeC7jiyu8yVmJlNHAdACVbMS55m0daxc4QlzcxmDgdACXItgAPdu8pciZnZxHEAlCB3M9iJqp10dJS5GDOzCeIAKEFtdS1NmeRmsBdeKHc1ZmYTwwFQouRmsF08/3y5KzEzmxgOgBKds3AFzNvJtm3lrsTMbGI4AEp07sKzoHkHz28b1Q+amZlNWw6AEp23YDXUnuCZnfvKXYqZ2YRwAJTo/AXnA/DsoS1lrsTMbGI4AEp03oLzANjTs4WBgTIXY2Y2ARwAJVrRtIIs9fQ3P8uePSMvb2Y23ZUUAJKulLRF0lZJNxeZL0mfSec/LumSdPr5kh7NG46nvxeMpFsk7c6bd/XEfrSJVaUqljeshgVbfCWQmVWEEQNAUgb4HHAVsAa4RtKagsWuAlanw/XA5wEiYktEXBwRFwOXAp3APXnrfSo3P/3t4Gnt/IXnwcItvhfAzCpCKS2Ay4CtEbEtInqAu4F1BcusA+6KxC+AZklLC5a5Ang+InaMu+oyuXj5+TB/G89t6yl3KWZm41ZKACwD8h+DuSudNtpl1gNfL5h2Y9pldIek+cV2Lul6Sa2SWg8cOFBCuZNnzaLzoaqfX2/38yDMbOYrJQBUZFrh3VDDLiOpBngH8K28+Z8HzgUuBtqATxTbeUTcHhFrI2JtS0tLCeVOntyVQJvbfCmomc18pQTALmBF3vvlQOF1MCMtcxXwSEQM3kUVEfsioj8iBoAvknQ1TWvnL0zuBdjdvYXu7jIXY2Y2TqUEwCZgtaRV6V/y64ENBctsAK5NrwZ6DXAsIvJ/Qv0aCrp/Cs4RvAvYPOrqp1hzXTNNmUXEGc/w9NPlrsbMbHxGDICI6ANuBO4Dnga+GRFPSrpB0g3pYhuBbcBWkr/m/yS3vqQG4C3Adws2faukJyQ9DrwJ+K/j/TBT4eULXwlLHuXxx8tdiZnZ+FSXslB6iebGgmm35Y0H8KdDrNsJLCgy/f2jqnSaeN05l/CzPZ/k0Sd6uJaacpdjZjZmvhN4lNYuuwQyvfzi+SfLXYqZ2bg4AEbpkqWXAPDU0YfLXImZ2fg4AEbpnPnnUMc8jjU8wsGD5a7GzGzsHACjVKUqVs99FSx9hCeeKHc1ZmZj5wAYg9eecwksfoyf/aKv3KWYmY2ZA2AMXn/OJZA9yf2PPFPuUszMxswBMAZrz1wLQOu+n/vHYcxsxnIAjMH5C86nuepMOhf/gC1+LJCZzVAOgDGQxG+e9WY450Ee+ombAGY2MzkAxug9F78FGg7xz62/LncpZmZj4gAYo7ec+2YAfr7vgTJXYmY2Ng6AMVoyZwlLqy7iUNMPePHFcldjZjZ6DoBxuHL1W+Csn/Cte9vLXYqZ2ag5AMbhute8A6q7+cpP/6XcpZiZjZoDYBxef9braYzFbI5vc+RIuasxMxsdB8A4ZKoyXHn2e2D1v/Cdf+4odzlmZqPiABinP/7N34FsF1/44caRFzYzm0YcAOP071a+kYaBRTx88m727i13NWZmpSspACRdKWmLpK2Sbi4yX5I+k85/XNIlefO2p7/9+6ik1rzpZ0h6QNJz6ev8iflIUytTlWH9mmuJ8+/lH/5xZ7nLMTMr2YgBICkDfA64ClgDXCNpTcFiVwGr0+F64PMF898UERdHxNq8aTcDD0bEauDB9P2M9LdvuxEU3PbI5/xwODObMUppAVwGbI2IbRHRA9wNrCtYZh1wVyR+ATRLWjrCdtcBd6bjdwLvHEXd08rZzWdz2dx3c+zc2/k/9/lksJnNDKUEwDIgv29jVzqt1GUCuF/Sw5Kuz1tmcUS0AaSvi4rtXNL1kloltR44cKCEcsvj/113E9Qf4UNf+3K5SzEzK0kpAaAi02IUy7wuIi4h6Sb6U0lvHEV9RMTtEbE2Ita2tLSMZtUp9aZzX8vLMm/iuaV/x8YfHit3OWZmIyolAHYBK/LeLwf2lLpMRORe9wP3kHQpAezLdROlr/tHW/x0IomvvO9/QuNB/uSrHyt3OWZmIyolADYBqyWtklQDrAc2FCyzAbg2vRroNcCxiGiT1ChpLoCkRuCtwOa8da5Lx68D7h3nZym71626lEuq/xM7ln6Kz3zt6XKXY2Y2rBEDICL6gBuB+4CngW9GxJOSbpB0Q7rYRmAbsBX4IvAn6fTFwE8kPQb8CviXiPi/6byPAm+R9BzwlvT9jPedP76V6oF5/NdfvYtnd7gryMymL0UUdudPX2vXro3W1taRFyyzu370Y6578AoWn7ia3Z+4h0yV77czs/KR9HDBZfiA7wSeFNf+5ht5V+Mn2de8gfd8+iPlLsfMrCgHwCT51oduZMnea7n32C189offLHc5ZmYv4QCYJJmM+NFf3EZmz2v54I/X85EffpyZ1N1mZpXPATCJzjunnu+9+wH09O/wNw/9Bddv+GN6+3vLXZaZGeAAmHRvv7KeL115Nzz0l3zp0S9wxT9ezf6OGX3Lg5lVCAfAFPiD36/i23/8P6j5/pd5aMdDvPyzF/PDF35Y7rLMbJZzAEyR97wHHv7SH7Ds+7/k4O55vPmuN/P/PPi39PT3lLs0M5ulHABT6MIL4fH7X8nbdrQSj17L3//k71h26znc+tNb2XF0R7nLM7NZxgEwxc44A75/7xy+e+1XWHTffRzcch4f/sGHWfnplbz+jtfzjc3foLuvu9xlmtks4AAoAwne9S7Y/uBb+e8rf0jNbc+R/beP8uSOvaz/znoWfXwRv/vd3+V7z3yPrt6ucpdrZhXKj4KYBrZvh7/4C/jOdweIVT9gyRXf5MTye+gYOExjtpG3n/d23nj2G7lw0YVcuOhCzqg/o9wlm9kMMtSjIBwA08ju3XDXXfCP/wjPPd9L5mX/xtIrvs2xpfdwYuDUj+EsnbN0MAxyw5qWNcypmVPG6s1sunIAzCAR8KtfwXe+kwzbtgVV83dx0b/fzPJLN1O1eDO7ezfz1MGnONl3cnC9Vc2rWNG0gsZsIxcuupDLl11Oc10zDdkGGmsaWdW8irm1c8v4ycysHBwAM1QEPPpoEgTf+x48+WQyvbERLl3bz3mXv8AZF2xmYOFmXjz5BPs793Ps5DE2799M78Dpdx0LsWr+KubWzKW2upaaTA01mRrqquuYXzefhQ0LaWloYWHDQhY0LEiCI9vIsnnLmFc7j87eTmoyNcytmUsQDMQA/QP99Ec/QixsWIhU7MfhzKycHAAV4sUX4cc/TloIv/xlEg496a0ES5fCZZfBq18Ny1d20Tv/aZavaodsJye6T/DMwWd46uBTdPV20d3fTU9/D9193ZzsO8nhrsMc7DzIiZ4TY66tpaGFixZfRF11HU21TSyds5Slc5fS0tBCpirDnJo5LJu7jOqqavoG+phTM4d5tfOYVzuPhmwDmarMBB0lM8vnAKhQ3d3w2GOnAuFXv4Jnnz01X4Lf+I0kGC64AM4++9SwZAkU/lRBd183h7oOcbDzIJ29nbT3tLPr+C46ejpoyDbQ3d9Ne087QmSqMmSUoUpV9A308di+x3jm4DN093dz9ORR2k600dVX+lVM1VXV1FfX05BtSFojjUlrpDHbSG2mltrqWuqq66jN1DIQA/QO9NLb30s2k2X5vOW097Sz89hOVjavZE3LGprqmphTM4c5NXNozDbSWHNqO1XyBXA2ezgAZpHjx5MTyi+8AK2tsGlTMuzbd/pyNTXw8pfD5ZfDokWwYAG86lVw0UXQ3Dz+OiKC493HOdh5kCAZ3318N/3RT3VVNR09HRzvPs7x7uN09nbS1dfFyb6TnOg+MRhCuSDq7k9aKrkWS5WqyGay1GRq6O7rprs/uXdift18jpw8MmJtGWUGu8FqM7U0ZBs4o/4MFjQsYEF9MuTe11XX0TfQN/ggv5bGFhY1LmJx42IWz1nMgvoF9PT30DfQ55aMTUsOAOPEiaQLaft22LEjeX3kEXj4YTh69PRlm5th5UpYtSoZzj33VAti6VJoaCjDBxhCRHCw8yD12Xrm1MzhcNdhth7eSntP+0uGXLdXT3/PYDdYT38P7T3tHO46zKGuQxzqPMShrkMcPXl05J0XUVddR0O2YXDbuVbNUEN1VXXR7UgiW5UdPFeTrcqSzWRf8lqTqaGpton59fPpG+ijSlUsmbPkJVeFDcQAjdlGlsxZQqYqc9qx6OnvoamuiQX1C3wepwKNKwAkXQl8GsgAX4qIjxbMVzr/aqAT+L2IeETSCuAuYAkwANweEZ9O17kF+M9A7vrGv4qIjcPV4QCYPBFJC6G1FZ55JgmHF15Ihu3boaugJ6e5Gc48E5YvhxUrkiE3nnudO8MvOOob6ONI1xF6+nvIZrJUV1UTERzoPMC+9n3s69jHvvZ9HO46TF11HZmqzGC3WVdvV/KlncnS1dtFZ28nnX2dyWvB0D/QX3T/AzFA30DfYFj19vcOdnv1R/F1xivXpRck3wvZqiz12frBEKvP1pOtyiIJkQRFLjCEqKuuo7GmcbDLrTHbeFr3Wy64csdmosbdpTe8MQeApAzwLMkPt+8CNgHXRMRTectcDfwZSQBcDnw6Ii6XtBRYmobBXOBh4J0R8VQaAO0R8Z8gDacAAAwhSURBVPFSP4QDoDwiYM+eJBh27UrG9+xJxnPD3r0vXa+pKQmDM89MwmDJkuR5SKtWJeNLlkBLC2TcYzJquXDo7e+lp7+HY93HONJ1ZPAE+972vaedf4kIqlRFe087e9v3EsRgy6I2U0t1VTXHuo+xr30fAzEAJF/svf29g91zXX1JkPUN9A3+uFEuKCKCIDjZd5KOng46ejvo6Omgvaedjt6OSX/oYXVVNQ3ZBqpURUdPB8BgOORaT/nvc2Fen60fbPWc7DvJyb6T9Pb3MhADg1e6RcTg+5pMzeB5pYZsA9mq5A+D3GtuyP3BMPi+KksQHOk6wkAMUFddR322nppMzWlX0w33ffy+V7yPl53xsjEdn6ECoHjb83SXAVsjYlu6obuBdcBTecusA+6KpPpfSGqWtDQi2oA2gIg4IelpYFnBujbNSbBsWTIMpacnCYWdO5NA2Lnz1HhbWzLvBz9Izk/kq6pKQmDx4uQ8RG1tEhaXXpp0OS1YcGpobnZY5FSpavALvJFG5tfPZ2XzynKXNaRcSybXipno8e7+brp6u+iPfhqzjUga7NrKheTg8mkrShKdvZ3sPrGbKlUNXmCQC5IqVSGUvKYtnlx34e7ju+no7aBvoO8lQ29/76nxgkux66rryChDV1/XYNCW6vLll485AIZSSgAsA3bmvd9F8lf+SMssI/3yB5C0EngV8Mu85W6UdC3QCvx5RIx89s6mpZqa5JzBypVDLxNxKhT27j192LcvGY4cSe51uPvul64vwfz5pwKhvh62bIH+frjuuuQ8RVsbrFmTnNg+80yoLuVfuE263F/CZMtdydTrH+gfDIK66rrB6blgqlLVaVfUTaVS/vcodkaosJ0y7DKS5gDfAW6KiNzfgJ8H/i5d7u+ATwB/8JKdS9cD1wOcddZZJZRr05UEZ52VDCM5cCA5/3Do0NBDeztccUXSqvjEJ5IgKNzf4sXJSeuWluRJrAsWJCEyf37yft48GBhIWh5nn51MHxiAX/86qeHd707WMRurTFWm6JVh2UxyEr+cSgmAXcCKvPfLgT2lLiMpS/Ll/9WI+G5ugYgYvChR0heB/1Ns5xFxO3A7JOcASqjXKkBLSzKUav/+5J6IlhZ4/PHk6qZc11NbGxw8eCpQjhxJWiOl+LM/gze8ITmp3dSUtCiOHUuugrr00lOh0dycvDY2JsFjNhOUEgCbgNWSVgG7gfXAfypYZgNJd87dJN1DxyKiLb066MvA0xHxyfwV8s4RALwL2DyOz2Gz3KJFp8YvuywZhjIwkLQajhxJvswzmeQqpx07kvcRyf0R9fVwxx3JzXX33Ze0OHp7kyA4fhw+/emXbru6+lQYNDcnw9y5MGfO6a9NTcm8pqakFTJ37ulDQ4ODxCbfiAEQEX2SbgTuI7kM9I6IeFLSDen824CNJFcAbSW5DPT309VfB7wfeELSo+m03OWet0q6mKQLaDvwRxP2qcyGUVV16ss5X7HQ+Oxni2+jvz+5KqqtLQmSo0eTITeeP62tLbkHo709ee0p4YKYqqqXhkLhUCw4ctObmk4NQ4VJf79Pqs92vhHMbIp1dyctiFxAHD+eBEMpQ+Gyvb0j7y+TORUGjY1JK2X3bjh8OOnGeuUrk5bJnDnJ/Pwhf1o2m7SQGhuTE+319ZN/rGxijOcyUDObQLW1oz/HMZTu7uIhcezY6cPRo8lrZ2cSGq9+ddJN9bOfwb33QkdHMq/UvwelJAhqapLPU1tbfLymJumeO+ecZH8NDcnQ2Dj8eG2tu8CmggPAbAbLfeEuXDj+bUUk50I6OpLuqo6O04fc+Y+jR2Hz5iRouruToaen+Hh7e3JZ7z/9U+nhAkkXWC4UCkOilAApZTybLR4yXV1w//1JaF100fiP63TmADAzIPkyzH1JjtQ6ec97Rrftnp5TrYzc63Djw807cCA5YV84b7S92ZnMqUCorz81bN2atJYA3vrW5N6WTCYZamuTZRoaTi2fGx/pdTq2ahwAZjbpamqSYf78ydl+RNLiGGu4dHUl411dyV/969cnV3995SvJ49b7+5Ohp2dsYQPJl/9oAqOuLjlm2Wzyes018LKJvRHYAWBmM5+UfGHW1U3cjXtvexv8zd+8dHrEqSDID47xvh46lLzmpp08mXS79fQk+7zsMgeAmVlZSafOvUxWiyZfRNL6KPzxpongADAzm8akyXumlR+ibWY2SzkAzMxmKQeAmdks5QAwM5ulHABmZrOUA8DMbJZyAJiZzVIOADOzWcoBYGY2SzkAzMxmKQeAmdks5QAwM5ulSgoASVdK2iJpq6Sbi8yXpM+k8x+XdMlI60o6Q9IDkp5LX6fguXpmZpYzYgBIygCfA64C1gDXSFpTsNhVwOp0uB74fAnr3gw8GBGrgQfT92ZmNkVKaQFcBmyNiG0R0QPcDawrWGYdcFckfgE0S1o6wrrrgDvT8TuBd47zs5iZ2SiU8pTpZcDOvPe7gMtLWGbZCOsujog2gIhok7So2M4lXU/SqgBol7SlhJqLWQgcHOO6U22m1DpT6oSZU+tMqRNmTq0zpU6YvFrPLjaxlAAo9jPGhb+IOdQypaw7rIi4Hbh9NOsUI6k1ItaOdztTYabUOlPqhJlT60ypE2ZOrTOlTpj6WkvpAtoFrMh7vxzYU+Iyw627L+0mIn3dX3rZZmY2XqUEwCZgtaRVkmqA9cCGgmU2ANemVwO9BjiWdu8Mt+4G4Lp0/Drg3nF+FjMzG4URu4Aiok/SjcB9QAa4IyKelHRDOv82YCNwNbAV6AR+f7h1001/FPimpA8ALwLvndBP9lLj7kaaQjOl1plSJ8ycWmdKnTBzap0pdcIU16qIUXXJm5lZhfCdwGZms5QDwMxslpoVATDSoyzKRdIKSf8q6WlJT0r6L+n0WyTtlvRoOlxd7loBJG2X9ERaU2s6bVo90kPS+XnH7VFJxyXdNF2OqaQ7JO2XtDlv2pDHUNJfpv9ut0h6W5nr/J+Snkkf93KPpOZ0+kpJXXnH9rapqnOYWof87z3Njuk38mrcLunRdPrUHNOIqOiB5OTz88A5QA3wGLCm3HWltS0FLknH5wLPkjwy4xbgQ+Wur0i924GFBdNuBW5Ox28GPlbuOgv+2+8luQlmWhxT4I3AJcDmkY5h+m/hMaAWWJX+O86Usc63AtXp+Mfy6lyZv9w0OaZF/3tPt2NaMP8TwN9O5TGdDS2AUh5lURYR0RYRj6TjJ4CnSe6enkmm8yM9rgCej4gd5S4kJyJ+DBwumDzUMVwH3B0R3RHxAslVdpeVq86IuD8i+tK3vyC5r6fshjimQ5lWxzRHkoD/AHx9KmrJmQ0BMNRjKqYVSSuBVwG/TCfdmDa17yh3t0qeAO6X9HD6iA4oeKQHUPSRHmWyntP/h5qOxxSGPobT+d/uHwDfz3u/StKvJf1I0hvKVVSBYv+9p+sxfQOwLyKey5s26cd0NgTAuB9HMdkkzQG+A9wUEcdJnqZ6LnAx0EbSNJwOXhcRl5A83fVPJb2x3AUNJb3x8B3At9JJ0/WYDmda/tuV9NdAH/DVdFIbcFZEvAr4b8DXJM0rV32pof57T8tjClzD6X+sTMkxnQ0BUMqjLMpGUpbky/+rEfFdgIjYFxH9ETEAfJEpaqKOJCL2pK/7gXtI6pquj/S4CngkIvbB9D2mqaGO4bT7tyvpOuDtwPsi7axOu1MOpeMPk/Srn1e+Kof97z0dj2k18G7gG7lpU3VMZ0MAlPIoi7JI+/2+DDwdEZ/Mm740b7F3AZsL151qkholzc2Nk5wQ3Mz0faTHaX9RTcdjmmeoY7gBWC+pVtIqkt/b+FUZ6gOSq+mADwPviIjOvOktSn77A0nnkNS5rTxVDtY01H/vaXVMU28GnomIXbkJU3ZMp+Lsd7kHksdUPEuSon9d7nry6no9SfPzceDRdLga+N/AE+n0DcDSaVDrOSRXTzwGPJk7jsACkh/0eS59PWMa1NoAHAKa8qZNi2NKEkptQC/JX6MfGO4YAn+d/rvdAlxV5jq3kvSf5/6t3pYu+57038RjwCPAb0+DYzrkf+/pdEzT6V8BbihYdkqOqR8FYWY2S82GLiAzMyvCAWBmNks5AMzMZikHgJnZLOUAMDObpRwAZmazlAPAzGyW+v8BMfKKra/4cD4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_metrics('wine_type_loss', 'Wine Type Loss', ylim=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "uYV9AOAMwI9p"
   },
   "source": [
    "### Plots for Confusion Matrix\n",
    "\n",
    "Plot the confusion matrices for wine type. You can see that the model performs well for prediction of wine type from the confusion matrix and the loss metrics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "C3hvTYxIaf3n"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATgAAAEQCAYAAAAkgGgxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3debxVdb3/8debQZRJmUUEnNAuaqmZ2aCRZqJZWjdvOBSVXaufdq0sU+/95VD0s2s53MqyweJmhuSQpCYSZmqpqGgqk5AkIMgoKqgI53x+f6zvkS2evc9ecDb77MX7yWM99l7f/V3f9d3D+fAd1qCIwMysiDrVuwJmZrXiAGdmheUAZ2aF5QBnZoXlAGdmheUAZ2aF5QC3CUk7SPqDpBck/W4LyjlF0p3tWbd6kXSYpDk1KDf3Zy3pbkmfa++6bLKPT0u6r4bl/1HS2JL1b0taIek5ScMkrZHUuVb735Z0qXcFNpekk4GvAm8BXgIeA8ZFxJb+MD8ODAL6RcSGzS0kIn4D/GYL61JzkgIYERHzyuWJiHuBfWqw+4qftaQLgb0i4tQa7LtuIuKYlueShgJnA8MjYllK7lmXihVQQ7bgJH0VuAL4DtkfyDDgKuD4dih+OPDUlgS3IpFUy/8E/Vlnn8HKkuC22Wr8XTWmiGioBdgRWAOcWCFPN7IAuDgtVwDd0mujgEVk/2suA5YAn0mvXQS8BqxP+zgNuBC4tqTs3YAAuqT1TwNPk7Ui5wOnlKTfV7Ldu4GHgBfS47tLXrsb+Bbw11TOnUD/Mu+tpf7nlNT/BOBY4ClgFXB+Sf5DgPuB1SnvD4Ht0mv3pPeyNr3fT5SU/w3gOeDXLWlpmz3TPg5K67sAK4BRZer7L+n9rQZmAB8p91lvst3oTV7/ezWfFXAo8Le0v7+Xq1fKOxS4CVgOrAR+WOa7uxJYCLwIPAIctsnn+3B6bSlwWUrfHrg2lbs6feeDSt7D54APAK8Azek9/oo3/752BH6RvrtngW8DnUvq+Vfg8vSdfLvef58dbal7BXJXOPvhb2j5AZTJczHwADAQGJB+8N9Kr41K218MdCULDC8DfdLrF/LGgLbp+us/QKBH+mHvk14bDOxb8uO7Lz3vCzwPfDJtd1Ja75devxv4B7A3sENav6TMe2up/zdT/f89/YFeB/QC9gVeBfZI+d9O9kffJdV9FvDlkvKCrBu4afnfJfuPYgdKAlzK8++pnO7AZOB7ZeraFZgHnA9sBxxBFpT2ae2zbWX7N71e6bMChpAFlGPJeidHpfUBrZTdmSwAXp6+x+2B92763aX1U4F+6TM8myzwb59eux/4ZHreEzg0Pf888If0GXVO30PvkvfwuZLPu/Sz3Y03BrjfA1enOg4EpgGfL6nnBuBLqW471Pvvs6MtjdhF7QesiMrdmlOAiyNiWUQsJ2stfLLk9fXp9fURcTvZ/56bO8bUDOwnaYeIWBIRM1rJ8yFgbkT8OiI2RMRvgdnAh0vy/DIinoqIV4CJwAEV9rmebLxxPTAB6A9cGREvpf3PAN4KEBGPRMQDab//JPtjeV8V7+mCiFiX6vMGEfEzYC7wIFlQ/88y5RxK9kd/SUS8FhF3AbeSBfgtUe6zOhW4PSJuj4jmiJhC1ro6tpUyDiFrfX49ItZGxKtRZvw2Iq6NiJXpM/w+WeBv+b2sB/aS1D8i1kTEAyXp/cj+82hK38OLed6kpEHAMWT/Ia2NrBt7OTCmJNviiPhBqtubvqttXSMGuJVA/zbGG3YBnilZfyalvV7GJgHyZTZjYDci1pJ1674ALJF0m6S3VFGfljoNKVl/Lkd9VkZEU3re8qNeWvL6Ky3bS9pb0q1phu5FsnHL/hXKBlgeEa+2kednwH7ADyJiXZk8uwALI6K5JG3T9705yn1Ww4ETJa1uWYD3kgXhTQ0FnmnjP0oAJJ0taVaa7V1N1m1s+QxPI2tNzpb0kKTjUvqvyVq3EyQtlvTfkrrmfJ/DyVrBS0rez9VkLbkWC3OWuU1pxAB3P1kX7IQKeRaT/ThaDEtpm2MtWTejxc6lL0bE5Ig4iuyPaDbZH35b9Wmp07ObWac8fkxWrxER0Zusu6g2tql4iRlJPcnGNX8BXCipb5msi4Ghkkp/Z3ned95L3SwEfh0RO5UsPSLikjJ5h7U1MC/pMLLxyH8jG8bYiWwcVQARMTciTiILOt8FbpDUI/UOLoqIkWTjr8cBn9qM97OObIyx5f30joh9S/L4ckAVNFyAi4gXyMaffiTpBEndJXWVdIyk/07Zfgv8l6QBkvqn/Ndu5i4fAw5PxyftCJzX8oKkQZI+IqkH2Q9xDdDUShm3A3tLOllSF0mfAEaSdddqrRfZOOGa1Lr84iavLwX2yFnmlcAjEfE54DbgJ2XyPUj2H8Q56TsaRdYtn1DlfpYCu20SICu5FviwpKMldZa0vaRRknZtJe80soH7SyT1SHnf00q+XmTjXMuBLpK+CfRueVHSqZIGpFbq6pTcJOn9kvZPx7O9SNZlbe23UVZELCGbRPm+pN6SOknaU1JbQwyWNFyAA4iIy8iOgfsvsh/eQuBMsgFZyGaaHgYeB54Apqe0zdnXFOD6VNYjvDEodSIbdF5MNov1PuD/tFLGSrL/wc8m62KfAxwXESs2p045fQ04mWxw/2dk76XUhcD41AX6t7YKk3Q82UTPF1LSV4GDJJ2yad6IeA34CNk40gqyQ3k+FRGzq6x7y8G/KyVNbytzRCwkO1TofDb+Lr5OK7/z1MX/MLAXsIBs5vgTrRQ7Gfgj2Qz1M2S9h9Ju4WhghqQ1ZIF/TOre7wzcQBbcZgF/YfP+k/0U2QTNTLKJqRtovcttrVCEW7i1Imk02Y++M/DzMl0l60AkXUP2n9GyiNiv3vWxLdOQLbhGkLomPyJrvYwETpI0sr61sir8iqxVZgXgAFc7hwDzIuLp1FWbQPucaWE1FBH3kA03WAE4wNXOEN44VrOILT88wsxycICrndYOxfCAp9lW5ABXO4vIDiZtsSubfyyemW0GB7jaeQgYIWl3SduRnV4zqc51MtumOMDVSDoF6Eyy46hmARPLnKdqHYik35KdLbOPpEWSTqt3nWzz+Tg4Mysst+DMrLAc4MyssBzgzKywHODMrLAc4LYCSafXuw6Wj7+zYnCA2zr8x9J4/J0VgAOcmRVWhzoOrn/fzrHb0LyXre/4lq9sYkC/Yt6o/KnHu7edqQGtZx1d6VbvarS7V1nLa7GurUvWV3T0+3vEylXVXZz4kcfXTY6Iul1+qkPdKHa3oV2ZNnlo2xmtwzh6l0o3/7KO5sGYusVlrFzVxLTJw6rK23nw3LZucFRTHSrAmVnHF0AzzW3m6wgc4MwslyBYH7nun1M3nmQws9yaq/xXiaR9JD1Wsrwo6cuS+kqaImlueuxTss15kuZJmiPp6Lbq6QBnZrkEQVNUt1QsJ2JORBwQEQcAbye7iffNwLnA1IgYAUxN66R7mowB9iW7b8ZV6d4nZTnAmVluzURVSw5HAv+IiGfI7l0yPqWPZ+NN3o8HJkTEuoiYD8wju/dJWR6DM7NcAmhq/6vvjyG7YTvAoHTTayJiiaSBKX0I8EDJNm3e58QtODPLLUcLrr+kh0uWN50hkq54/RE23ui7nNz3OXELzsxyCWB99ScIrIiIg9vIcwwwPSKWpvWlkgan1ttgYFlKz32fE7fgzCyXIGiqcqnSSWzsnkJ275Kx6flY4JaS9DGSuknaHRgBTKtUsFtwZpZPQFM7DcFJ6g4cBXy+JPkSYGK6H8YC4ESAiJghaSIwE9gAnBFR+YA8BzgzyyU7k6Gdyop4Gei3SdpKslnV1vKPA8ZVW74DnJnlJJpaHe/veBzgzCyXbJLBAc7MCig7Ds4BzswKqtktODMrIrfgzKywAtHUIIfQOsCZWW7uoppZIQXitWiMe4w4wJlZLtmBvu6imllBeZLBzAopQjSFW3BmVlDNbsGZWRFlkwyNEToao5Zm1mF4ksHMCq3Jx8GZWRH5TAYzK7Rmz6KaWRFlJ9s7wJlZAQVivU/VMrMiisAH+ppZUalhDvRtjDBsZh1GkLXgqlnaImknSTdImi1plqR3SeoraYqkuemxT0n+8yTNkzRH0tFtle8AZ2a5NdGpqqUKVwJ3RMRbgLcBs4BzgakRMQKYmtaRNBIYA+wLjAauklRxMNABzsxyCURzVLdUIqk3cDjwC4CIeC0iVgPHA+NTtvHACen58cCEiFgXEfOBecAhlfbhMTgzyyW7bWC7hI49gOXALyW9DXgEOAsYFBFLACJiiaSBKf8Q4IGS7ReltLLcgjOznLIbP1ezAP0lPVyynF5SUBfgIODHEXEgsJbUHS274zeLSjV1C87McglyncmwIiIOLvPaImBRRDyY1m8gC3BLJQ1OrbfBwLKS/ENLtt8VWFxp527BmVluOVpwZUXEc8BCSfukpCOBmcAkYGxKGwvckp5PAsZI6iZpd2AEMK3SPtyCM7NcItSe56J+CfiNpO2Ap4HPkDW8Jko6DVgAnJjtN2ZImkgWBDcAZ0REU6XCHeDMLJdskqF9TtWKiMeA1rqwR5bJPw4YV235DnBmlpPvyWBmBZVNMjTGqVoOcGaWmy+XZGaF1HImQyNwgNtCc+a9xklfeO719aefWc9FX+/Hs89t4NY717LddmKP4V255oqB7LTjxoHZBYvWs9/7FnDB1/py9hf7tFa01cHZv/gi7/zQ21m97AVOf+vZ9a5Oh9UoN51pjFp2YPvstR3T/zSM6X8axkOTh9J9h06ccEwPPnB4dx6/exiP3TWMvffsyiU/eP4N2331ghWMPqJ7nWpt5dz5q7s5/5iqJ+m2SRGwvrlTVUu9uQXXjqbe+wp77taV4UOzpcU7D9qeG29d8/r67/+4hj2Gd6VH98Zo5m9Lnrh3FoOGD6h3NTq0rIta/+BVjcaoZYO4/paXGHNCzzel/3LCi4w+ogcAa19u5tIfPc83z+67tatn1m7a40yGraGmAU7S6HRhunmSKp1E2/Beey34w+S1fPzDbwxw37liFV06i1P+NUu/8NJVnHX6TvTs4f9brDG1HCaypZdL2hpq1kVNF6L7EXAU2UmyD0maFBEza7XPevrjXWs5cP9uDBqw8SMdP/FFbvvTWqZMHIKUfdnTpr/Kjbeu4dxvrWT1i8106gTbdxNnfHanelXdLKfG6aLWcgzuEGBeRDwNIGkC2QXrChngJvx+DWM+2uv19TvuWsulP3yeP9+0K927b/wx/OWWXV9/ftH3VtKzRycHN2s4vidDdiG6hSXrbV6crlG9/HIzf7rnZT52bI/X0/7jP1fw0trg6DHPctAHFvDFc5ZVKME6ivN/cxZX/m0cQ/fZhesW/ITRnz2i3lXqcLJZ1M5VLfVWyxZcVRenSxfAOx1g2JDGnNTt3r0Ty2fu8Ya0p+4f3uZ2F3ytX62qZJvpO6dcWe8qdHiNdKBvLVtwVV2cLiJ+GhEHR8TBA/rVP+KbWdua060D21rqrZZNpoeAEenCdM+S3Q3n5Bruz8y2Ap9sD0TEBklnApOBzsA1ETGjVvszs63Hs6hARNwO3F7LfZjZ1hUhNjjAmVlRbfNdVDMrJo/BmVmhOcCZWSH5ODgzK7T2Og5O0j8lPSHpMUkPp7S+kqZImpse+5TkPy9dvGOOpKPbKt8BzsxyiYANzZ2qWqr0/og4ICJabh94LjA1IkYAU9M6kkaSHU+7LzAauCpd1KMsBzgzy63Gl0s6Hhifno8HTihJnxAR6yJiPjCP7KIeZTnAmVkuLWNw7RTgArhT0iPpvHSAQRGxBCA9DkzpuS/g4UkGM8stqm+d9W8ZW0t+GhE/LVl/T0QsljQQmCJpdoWyqrqARykHODPLLceJ9CtKxtbeJCIWp8dlkm4m63IulTQ4IpZIGgy0XGusqgt4lHIX1cxyiWifMThJPST1ankOfBB4EpgEjE3ZxgK3pOeTgDGSuqWLeIwAplXah1twZpaTaGqfWwIOAm5Ol/PvAlwXEXdIegiYKOk0YAFwIkBEzJA0keyq4BuAMyKiqdIOHODMLLccY3AVyoingbe1kr4SOLLMNuOAqm9c6wBnZrn4XFQzK67IxuEagQOcmeXWES5HXg0HODPLJdpvkqHmHODMLDd3Uc2ssNpjFnVrcIAzs1wiHODMrMB8mIiZFZbH4MyskALR7FlUMyuqBmnAOcCZWU6eZDCzQmuQJpwDnJnl5hacmRVSAM3NDnBmVkQBuAVnZkXl4+DMrLgc4MysmORJBjMrMLfgzKyQAsKzqGZWXI0R4BrjjFkz61iiyqUKkjpLelTSrWm9r6Qpkuamxz4lec+TNE/SHElHt1W2A5yZ5deOAQ44C5hVsn4uMDUiRgBT0zqSRgJjgH2B0cBVkjpXKtgBzszyaTnQt5qlDZJ2BT4E/Lwk+XhgfHo+HjihJH1CRKyLiPnAPOCQSuU7wJlZbhHVLUB/SQ+XLKdvUtQVwDlAc0naoIhYku0nlgADU/oQYGFJvkUprSxPMphZftXPoq6IiINbe0HSccCyiHhE0qgqymptpxU7wm0GOEkCTgH2iIiLJQ0Ddo6IaVVUyMwKSO1zHNx7gI9IOhbYHugt6VpgqaTBEbFE0mBgWcq/CBhasv2uwOJKO6imi3oV8C7gpLT+EvCj6t+DmRVKtRMMbQTBiDgvInaNiN3IJg/uiohTgUnA2JRtLHBLej4JGCOpm6TdgRFAxYZWNV3Ud0bEQZIeTZV6XtJ2VWxnZoVU3QTCFrgEmCjpNGABcCJARMyQNBGYCWwAzoiIpkoFVRPg1qep2ACQNIA3Dgia2bamnU/Vioi7gbvT85XAkWXyjQPGVVtuNV3U/wFuBgZKGgfcB3yn2h2YWQE1V7nUWZstuIj4jaRHyCKqgBMiYlYbm5lZURXpgpdp1vRl4A+laRGxoJYVM7OOq51mUWuumjG428hitsimcncH5pCdLmFm26KiBLiI2L90XdJBwOdrViMzs3aS+0yGiJgu6R21qMxTj3fn6F0OqEXRViPLznx3vatgOWy4/oF2KacwXVRJXy1Z7QQcBCyvWY3MrGML8pyqVVfVtOB6lTzfQDYmd2NtqmNmDaEILbh0gG/PiPj6VqqPmTWAhu+iSuoSERvSpIKZ2UaNHuDITmI9CHhM0iTgd8Dalhcj4qYa183MOqoCBLgWfYGVwBFsPB4uAAc4s22QogBdVLJzT78KPMnGwNaiQd6emdVEAWZROwM92YyraJpZsRWhBbckIi7eajUxs8ZRgADXGG1QM9u6CjIG1+oF58zMGr4FFxGrtmZFzKxxqANczLIavi+qmRWW74tqZvk1ehfVzKxVDTTJ4C6qmeXXDvdFlbS9pGmS/i5phqSLUnpfSVMkzU2PfUq2OU/SPElzJB3dVjUd4Mwsv3YIcMA64IiIeBtwADBa0qHAucDUiBgBTE3rSBpJdoPofYHRwFXpikdlOcCZWS4im0WtZqkkMmvSate0BHA8MD6ljwdOSM+PByZExLqImA/MAw6ptA8HODPLJzaecN/WAvSX9HDJcnppUZI6S3oMWAZMiYgHgUERsQQgPQ5M2YcAC0s2X5TSyvIkg5nlV/0kw4qIOLhsMRFNwAGSdgJulrRfhbJynxfvFpyZ5dc+Y3Abi4tYDdxNNra2VNJggPS4LGVbBAwt2WxXYHGlch3gzCy3HF3U8mVIA1LLDUk7AB8AZgOTgLEp21jglvR8EjBGUjdJuwMjyC7MW5a7qGaWX/scBzcYGJ9mQjsBEyPiVkn3AxMlnQYsAE4EiIgZkiYCM8lugHVG6uKW5QBnZvlE+5yLGhGPAwe2kr6SMhf7iIhxwLhq9+EAZ2b5NciZDA5wZpZbo5yq5QBnZvk5wJlZIeU8BKSeHODMLBfhLqqZFZgDnJkVlwOcmRWWA5yZFVIDXdHXAc7M8nOAM7OiapTbBjrAmVlu7qKaWTH5QF8zKzQHODMrIp/JYGaFpubGiHAOcGaWj8fgzKzI3EU1s+JygDOzonILzsyKq0ECnO+Lamb5pLtqVbNUImmopD9LmiVphqSzUnpfSVMkzU2PfUq2OU/SPElzJB3dVlXdgquRAbv245zxZ9J3551obg5u/9mfuPl/bq93tQwYtFNPxn1yNP16dScCbvjbE1z3l0cBOOnwAxhz2Ntoag7umTGfKybdC8Bnj3oHHz10P5qbm/nujXfzt9nP1PMt1FU7Hge3ATg7IqZL6gU8ImkK8GlgakRcIulc4FzgG5JGAmOAfYFdgD9J2rvSvVEd4GqkaUMTV3/tf5n36Hx26Lk9Vz38XR6Z8jgLZi2qd9W2eU3NwfduvofZi5bRvVtXJnz9FB6Y8wz9enVn1P578vHvXsv6DU307bkDAHvs3JfRB+3Dx/7f/zKwdw+uPvNf+ci3fkVzNEg/rRba4b1HxBJgSXr+kqRZwBDgeGBUyjYeuBv4RkqfEBHrgPmS5gGHAPeX24e7qDWy6rnVzHt0PgCvrHmVBbOepf+QvnWulQGseHEtsxctA+Dldet5eukqBu7YkxPf+zaumfIQ6zdkDYJVa14BYNT+e3LH9Dms39DEs6teZOHy1ew3fOe61b8jUFS3AP0lPVyynN5qedJuZDeBfhAYlIJfSxAcmLINARaWbLYopZXlFtxWMGj4APY6cHdmPzi33lWxTezStzdvGTKAJ555jq8cfxgH7TmELx33btZtaOKy39/DjAVLGbRjTx7/55LXt1m6eg0Dd+pZx1rXWb4DfVdExMGVMkjqCdwIfDkiXpRUNmuZ2pRVsxacpGskLZP0ZK320Qi277E937zha/z4K7/k5ZdeqXd1rMQO23Xl+6cdx6U3/YW1r75Gl06d6N29G6deNoHLf38Pl37mQ1nGVv6sYlvuntI+kwwAkrqSBbffRMRNKXmppMHp9cHAspS+CBhasvmuwOJK5deyi/orYHQNy+/wOnfpzAU3nM1d193LfTdPq3d1rESXTp247LTjuP3h2Ux9fB4AS19Yw9S/Z8+fXLCU5gj69NyBpavXMKhPr9e3HbRTT5a/sLYu9e4o2mkWVcAvgFkRcVnJS5OAsen5WOCWkvQxkrpJ2h0YAVT8w6pZgIuIe4BVtSq/EZz98y+yYPaz3Hj5rfWuim3iwpOP4umlq/j1n6e/nvbnx//BIXtnDYThA3aia+fOPL/mFf7yxNOMPmgfunbpzJC+vRk2oA9PPvNcvapef0E2yVDNUtl7gE8CR0h6LC3HApcAR0maCxyV1omIGcBEYCZwB3BGpRlU6ABjcGnQ8XSA7ele59q0n33f8xaO+tT7ePrxZ/jJ9EsBuOY/r2PaHx+tc83swD124cOHjOSpZ5dz/TmnAPCDW//KzQ88ycUnf5Abz/0k65ua+L/XTgbgH8+t5M5Hn+Lm8z9FU1Mz3/ndXdv2DCrtc5hIRNxH6+NqAEeW2WYcMK7afaiWYwlpZuTWiNivmvy91TfeqVbfl3VQy858d72rYDnMvf4yXl66sOwofjV69hkaB7z/rKry/vXmrz/S1iRDLdW9BWdmjcUXvDSz4opomAte1vIwkd+SHWG8j6RFkk6r1b7MbCuLKpc6q1kLLiJOqlXZZlZf7qKaWTEF0CBdVAc4M8uvMeKbA5yZ5ecuqpkVVqPMojrAmVk+HWSGtBoOcGaWS3agb2NEOAc4M8uvikshdQQOcGaWm1twZlZMHoMzs+JqnHNRHeDMLD93Uc2skKK6+y10BA5wZpafW3BmVliNEd8c4MwsPzU3Rh/VAc7M8gka5kDfWt4X1cwKSASK6pY2y2rlBvGS+kqaImlueuxT8tp5kuZJmiPp6LbKd4Azs/za576o0PoN4s8FpkbECGBqWkfSSGAMsG/a5ipJnSsV7gBnZvm1U4Arc4P444Hx6fl44ISS9AkRsS4i5gPzgEMqle8AZ2b5tIzBVbNAf0kPlyynV7GHQRGxBCA9DkzpQ4CFJfkWpbSyPMlgZrnlmEVd0Y43fm7thtUVm4luwZlZTlV2Tzf/YOClkgYDpMdlKX0RMLQk367A4koFOcCZWT5BrQPcJGBsej4WuKUkfYykbpJ2B0YA0yoV5C6qmeXXTsfBpRvEjyIbq1sEXABcAkxMN4tfAJwIEBEzJE0EZgIbgDMioqlS+Q5wZpZbe13wssIN4o8sk38cMK7a8h3gzCw/n2xvZoUUAU2Nca6WA5yZ5ecWnJkVlgOcmRVSAL4ng5kVU0B4DM7MiijwJIOZFZjH4MyssBzgzKyYtug8063KAc7M8gnAN50xs8JyC87MismnaplZUQWEj4Mzs8LymQxmVlgegzOzQorwLKqZFZhbcGZWTEE0VbwVQofhAGdm+fhySWZWaD5MxMyKKIBwC87MCil8wUszK7BGmWRQdKDpXknLgWfqXY8a6A+sqHclLJeifmfDI2LAlhQg6Q6yz6caKyJi9Jbsb0t0qABXVJIejoiD610Pq56/s2LoVO8KmJnVigOcmRWWA9zW8dN6V8By83dWAA5wW0FE1PWPRVKTpMckPSnpd5K6b0FZv5L08fT855JGVsg7StK7N2Mf/5RU7SB2TdT7O7P24QC3bXglIg6IiP2A14AvlL4oqfPmFBoRn4uImRWyjAJyBziz9uIAt+25F9grta7+LOk64AlJnSVdKukhSY9L+jyAMj+UNFPSbcDAloIk3S3p4PR8tKTpkv4uaaqk3cgC6VdS6/EwSQMk3Zj28ZCk96Rt+0m6U9Kjkq4GtHU/EisqH+i7DZHUBTgGuCMlHQLsFxHzJZ0OvBAR75DUDfirpDuBA4F9gP2BQcBM4JpNyh0A/Aw4PJXVNyJWSfoJsCYivpfyXQdcHhH3SRoGTAb+BbgAuC8iLpb0IeD0mn4Qts1wgNs27CDpsfT8XuAXZF3HaRExP6V/EHhry/gasCMwAjgc+G1ENAGLJd3VSvmHAve0lBURq8rU4wPASOn1BlpvSb3SPj6Wtr1N0vOb+T7N3sABbtvwSkQcUJqQgsza0iTgSxExeZN8x5KdX12JqsgD2ZDIuyLilVbq4iPOrd15DM5aTAa+KKkrgKS9JfUA7gHGpDG6wcD7W9n2fuB9knZP2/ZN6S8BvUry3Qmc2bIiqSXo3txH+0sAAACGSURBVAOcktKOAfq027uybZoDnLX4Odn42nRJTwJXk7XwbwbmAk8APwb+sumGEbGcbNzsJkl/B65PL/0B+GjLJAPwH8DBaRJjJhtncy8CDpc0nayrvKBG79G2MT4X1cwKyy04MyssBzgzKywHODMrLAc4MyssBzgzKywHODMrLAc4Myus/w/PJxxPkUqY1wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_confusion_matrix(test_Y[1], np.round(type_pred), title='Wine Type', labels = [0, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "GW91ym8P2I5y"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAREAAAEWCAYAAABBkaM2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAdZ0lEQVR4nO3df5wcdZ3n8debJOiERYJkWMlgElQMt8hCQuSHnCwsq3ngKkbg4UXlXL1dsnHRVU9zyq6/2L073Udcd0X2iMFfoG68A0LOH0jUE111BcxPAkp2+U0m/AhgiMAoIfncH10Te3q6p6unu6q7qt/Px2Me0/Wtmu5Pipk336r61rcUEZiZTdYB3S7AzIrNIWJmbXGImFlbHCJm1haHiJm1xSFiZm1xiJhZWxwilpqkJ6u+9kkaqVp+S7frs+6QB5vZZEi6F/iziPhet2ux7nJPxNoi6TmSHpd0XFXb4UkvZVDSGZK2S/orSY9Kure615L8/Ccl3S/pYUkrJQ10519jk+EQsbZExG+ArwEXVDW/CfheROxMll8AzASGgD8BVkmal6z7O+ClwAnAS5JtPpJD6dYhDhHrhCuBN0sa/X36z8CXa7b5cET8JiJ+CHwLeKMkARcC742IxyPiV8D/BJbkVbi1b2q3C7Dii4ibJT0F/IGkB6n0KL5etckvI+KpquX7gFnAIDAd2FDJEwAETMm+ausUh4h1ypVUDmkeAq6JiF9XrTtU0kFVQTIbuA14FBgBjo2I4VyrtY7x4Yx1ypeBN1AJkqvqrL9E0oGSXgm8Frg6IvYBVwD/IOlwAElDkhblVbS1zyFiHRER24GNQAA/qln9EPBLYAfwVWBZRNyRrPsAcCdwk6TdwPeAeVhheJyIdYykLwA7IuJDVW1nAF+JiCO7VphlyudErCMkzQXOBeZ3txLLmw9nrG2S/pbKidIVEXFPt+uxfPlwxsza4p6ImbWlcOdEZs6cGXPnzu12GWYd9fhTzzC8a4SDnzOVOYcdxG/H3nXHhg0bHo2IwTTbFi5E5s6dy/r167tdhlnHrL7lfi5es5U3zxvk8gtO5LnTuj9gV9J9abfN7HBG0jxJm6u+dkt6T802Z0h6omob33hlfWU0QM7soQBpVWY9kYjYRuXOTCRNAYaB6+ps+qOIeG1WdZj1qjIECOR3YvUs4K6ISN1FMiuzsgQI5BciS4DVDdadKmmLpG9LOrbeBpKWSlovaf3OnTvrbWJWGGUKEMghRCQdCJwDXF1n9UZgTkQcD3wGWFvvPSJiVUQsjIiFg4OpThib9aSyBQjkc3XmbGBjRDxcuyIidle9vl7S/5I0MyIezaEu6zNrNw2zYt02duwaYdaMAZYvmsfi+UO5fX4ZAwTyCZE30eBQRtILgIcjIiSdRKVn9FgONZVet/9ges3aTcNcvGYrI3v2AjC8a4SL12wFyGW/lDVAIOPDGUnTgVcBa6ralklaliyeD9wmaQtwKbAkPA6/baN/MMO7Rgh++wezdlP/zvuzYt22/QEyamTPXlas25b5Z5c5QCDjnkhEPA0cVtO2sur1ZcBlWdbQjyb6g+nX3siOXSMttXdK2QMEfO9MKXXrD6aXzZpR/ykUjdo7oR8CBBwipdSNP5het3zRPAZq/ogHpk1h+aJsJlHrlwABh0gp5f0HUwSL5w/x8XOPY2jGAAKGZgzw8XOPy+Twrp8CBAp4A541t3j+EOvve5zVNz/A3gimSJx34lDfng8ZtXh+9vug3wIE3BMppbWbhrl2wzB7kwtdeyO4dsNwX1+dyUM/Bgg4REqpm5cz+1W/Bgg4RErJV2fy1c8BAg6RUvLVmfz0e4CAQ6SUfHUmHw6QCl+dKaHRKxC+dyY7DpDfcoiUVB6XM/uVA2QsH86YtcABMp5DxCwlB0h9DhGzFBwgjfmcSEl5UqLOcYBMzCFSQt2exatMHCDN+XCmhDzsvTMcIOm4J1JCww2Gtzdq7xetHOI5QNJziJTQFGn/Hby17f2qlUM8B0hrfDhTQvUCZKL2fpD2EM8B0jqHSAkNNbjRrlF7P0hzZ7MDZHIcIiV05jH1nxLYqL0fNLuz2QEyeQ6RErrxjvrPK27U3g8mClYHSHt8YrWEPCnReN+69cG67Ws2DvOVm+53gLShFCHi0ZljzZoxUPdybj9PSvTLp/fUbX/6mb0OkDZldjgjaZ6kzVVfuyW9p2YbSbpU0p2SbpW0oNXP8SMjx/OkRK1xgLQnsxCJiG0RcUJEnACcCDwNXFez2dnA0cnXUuDyVj/HozPHy/MZK0UxY2Ba3fZDnjvVAdKmvE6sngXcFRH31bS/HrgqKm4CZkg6opU39vG/pfGxc45l2gFjB9tNFVzy+pd1qaLyyCtElgCr67QPAQ9ULW9P2saQtFTSeknrd+4ce4XBkxKPt3bTMMuv2TLmEG/5NVv6+hBv8fwhFsw5dP/yAcCSk2f3de+sUzIPEUkHAucAV9dbXadt3LDKiFgVEQsjYuHg4NhLdT7+H++Sb9zOnr1jd+OevcEl37i9SxV13weuuZWb73l8//I+8AO9OiSPnsjZwMaIeLjOuu3AC6uWjwR2tPLmPv4fr9GViEbtZbf6lvv53+sfGNfe7+fOOiWPS7xvov6hDMDXgXdK+hpwMvBERNS/oD8BT0psjYwOJGuk3+9s7oRMeyKSpgOvAtZUtS2TtCxZvB64G7gTuAL4iyzr6ReNrkQ0ai+r6pGojX7R+/nO5k7JNEQi4umIOCwinqhqWxkRK5PXEREXRcSLI+K4iFifZT394mPnHDvuP+wBSXu/qB3Kvq/Bdv18Z3On+N6ZkpoyRRMul5nvhcmXQ6SEVqzbVvfqTD+cRHSA5M8hUkL9OgDPAdIdDpES6scBeA6Q7nGIlFC/DcBzgHSXQ6SEFs8f4rwTh/Zfvpwicd6J5RxL4wDpPodICa3dNMy1G4b3X77cG1HKId4OkN7gECmhfpgewQHSOxwiJVT2h1c5QHqLQ8QKxQHSezzHqhWGA6Q3FT5EWnk8Yr8o42M0HSC9q/CHM/1wErFVLxqc3lJ7r3OA9LbCh0i/DvGeyJ2PPNVSey9zgPS+wodIPw7xbqbRze1Fu+ndAVIMhQ+R5YvmMa3mNvdpU1TaId79wgFSHIUPEWD8/2KL9r9cG8MBUiyFD5EV67axZ1/N3Bn7+mPujDJygBRP4UPEJ1bLwwFSTIUPEZ9YLQcHSHEVPkTOPGawpXbrPQ6QYit8iNx4x86W2q23OECKr/Ah4nMixeUAKYfCh4jPiRSTA6Q8Ch8i/TafaBk4QMql8Hfxjt6p66kAisEBUj6ZhoikGcDngJdRGUf6XyLip1XrzwD+L3BP0rQmIv6m1c/xA72LwQFSTln3RD4N3BAR50s6EKh3L/qPIuK1GddhXeYAKa/MQkTS84DTgbcBRMQzwDNZfZ71LgdIuWXZE3kRsBP4oqTjgQ3AuyOidlKLUyVtAXYA74+I22vfSNJSYCnA7Nmzx33QW674KT+56/H9y6e9+Pl89cJTO/XvsDY4QMovy6szU4EFwOURMR94CvhgzTYbgTkRcTzwGWBtvTeKiFURsTAiFg4Ojh2JWhsgAD+563HecsVPse5ygPSHLENkO7A9Im5Olq+hEir7RcTuiHgyeX09ME3SzFY+pDZAmrVbPhwg/SOzEImIh4AHJI0O2DgL+Hn1NpJeIFVmD5Z0UlLPY1nVZPlwgPSXrK/OvAv4anJl5m7g7ZKWAUTESuB84B2SngVGgCURdaYpt8JwgPSfTEMkIjYDC2uaV1atvwy4LMsaLD8OkP5U+GHv1hscIP3LIWJtc4D0N4eItcUBYg4RmzQHiEHKEJH0YknPSV6fIekvk5vrrE85QGxU2p7ItcBeSS8BPg8cBfxzZlVZT3OAWLW0IbIvIp4F3gD8Y0S8Fzgiu7KsVzlArFbaENkj6U3AnwDfTNqmZVOS9SoHiNWTNkTeDpwK/I+IuEfSUcBXsisrPbXYbpPjALFGUo1YjYifA39ZtXwP8ImsimpFozHyHjvfOQ4Qm0jaqzOnSfqupH+TdLekeyTdnXVxaQw1mNW9Ubu1xgFizaS9d+bzwHupTCy0N7tyWjf3sAGG6zxjZu5hDpF2OUAsjbQh8kREfDvTSibpprt/2VK7peMAsbTShsiNklYAa4DfjDZGxMZMqmrB3gYzBzRqt+YcINaKtCFycvK9+rb+AP6ws+VYtzlArFVpr86cmXUh1n0OEJuMtFdnDpH0KUnrk6+/l3RI1sVZfhwgNllpB5t9AfgV8MbkazfwxayKsnw5QKwdac+JvDgizqtavkTS5iwKsnw5QKxdaXsiI5L+4+iCpNOoTKxsBeYAsU5I2xN5B3Blch5EwOMkj8e0YnKAWKekvTqzGTg+eb4uEbE706osUw4Q66QJQ0TSBRHxFUn/taYdgIj4VIa1WQYcINZpzXoiByXfD66zzkNCC8gBYp02YYhExGeTl9+LiJ9Ur0tOrlrBOECs09JenflMyrYxJM2QdI2kOyT9QtKpNesl6VJJd0q6VdKCRu9lneEAsU5rdk7kVOAVwGDNeZHnAWl+Ez8N3BAR5yfP451es/5s4Ojk62Tgcn57n45lwAFindbsnMiBwO8k21WfF9lN5WHcDSVXck4nuRQcEc8Az9Rs9nrgquQh3jclPZcjIuLB1P8CM+uqZudEfgj8UNKXIuK+Ft/7RcBO4IuSjqcyodG7I+Kpqm2GgAeqlrcnbWNCRNJSYCnA7NmzWyzDzLKU9pzI56ofViXpUEnrmvzMVGABcHlEzAeeAj5Ys029+ZTHXfWJiFURsTAiFg4ODqYs2czykDZEZkbErtGFiPglcHiTn9kObI+Im5Pla6iESu02L6xaPhLYkbImq2P1Lfd3uwTrM6kfXiVp/3GEpDk0GScSEQ8BD0ialzSdBfy8ZrOvA29NrtKcQmUaRp8PmaTRgWRmeUp778xfAz+W9MNk+XSScxRNvAv4anJl5m7g7ZKWAUTESuB64DXAncDTVJ5vY5NQPRL1xm07u12O9ZG0987ckIzhOIXKeYz3RsSjKX5uM2OnVARYWbU+gIvSl2v11A5lP+bDN3S7JOsjEx7OSDom+b4AmE3lfMUwMNsDw3qD74WxbmvWE3kfcCHw93XWeaLmLnOAWC9oNk7kwuS7J2ruMQ4Q6xXNhr2fO9H6iFjT2XIsDQeI9ZJmhzOvS74fTuUemu8ny2cCP6DyMCvLkQPEek2zw5m3A0j6JvB7o2M4JB0B/FP25Vk1B4j1orSDzebWDAJ7GHhpBvVYAw4Q61VpB5v9ILlXZjWVqzJLgBszq8rGcIBYL0s72Oydkt5AZaQqwKqIuC67smyUA8R6XdqeCMBG4FcR8T1J0yUdHBG/yqowc4BYMaR9Fu+FVO7CHZ1zdQhYm1VR5gCx4kh7YvUi4DQqM5oREf9O86kAbJIcIFYkaUPkN8n0hgBImoofGZEJB4gVTdoQ+aGkvwIGJL0KuBr4RnZl9ScHiBVR2hD5AJX5UrcCf05lHpAPZVVUP3KAZEv1JuKcoN3Sa3p1RtIBwK0R8TLgiuxL6j8OkOxFg4PvRu2WXtOeSETsA7ZUT49oneMAyceMgWkttVt6aceJHAHcLukWKrO2AxAR52RSVZ9wgOTnmWf3ttRu6aUNkUsyraIPOUDy9fSefS21W3rN5hN5LrAMeAmVk6qfj4hn8yiszBwgVibNzolcSWWi5a1Unptbb5pEa4EDpDsOnV7/3EejdkuvWYj8XkRcEBGfpfLs3VfmUFNpOUC656OvO5YpB4y9njvlAPHR1x3bpYrKo1mI7Bl94cOY9jhAuq/2lz3tICmbWLP9eLyk3cnXr4DfH30taXceBZaBA6T7Vqzbxp59YweF7NkXrFi3rUsVlUez6RH9294mB0hv2LFrpKV2Sy/THp2keyVtlbRZ0vo668+Q9ESyfrOkj2RZT94cIL1j1oyBltotvTwOC8+MiBMiovZxmqN+lKw/ISL+Jod6cuEA6S3LF81joOa/wcC0KSxfNK/BT1harcxsZik5QHrP4vlDQOXcyI5dI8yaMcDyRfP2t9vkZR0iAXxHUgCfjYhVdbY5VdIWKs/5fX9E3J5xTZlygPSuxfOHHBoZyDpETouIHZIOB74r6Y6I+Jeq9RuBORHxpKTXUJly8ejaN5G0FFgKMHt2794H6ACxfpTpOZGI2JF8fwS4DjipZv3uiHgyeX09ME3SzDrvsyoiFkbEwsHBwSxLnjQHSO9bu2mY0z7xfY764Lc47RPfZ+2m4W6XVAqZhYikgyQdPPoaeDVwW802L5Aq08JIOimp57GsasqKA6T3rd00zMVrtjK8a4QAhneNcPGarQ6SDsiyJ/K7wI+T8x23AN+KiBskLZO0LNnmfOC2ZJtLgSURxZomxgFSDCvWbWNkz9jb/kf27PVgsw7I7JxIRNwNHF+nfWXV68uAy7KqIWsOkOLwYLPs+PaBSXKAFIsHm2XHITIJDpDi8WCz7HiwWYscIMXkwWbZcYi0wAFSbB5slg0fzqTkADGrzyGSggPErDGHSBMOELOJOUQm4AAxa84h0oADxCwdh0gdDhCz9BwiNRwgZq1xiFRxgJi1ziGSKFOAHHRg/dobtZu1wyFCuQIEYOSZ+k+6b9Ru1o6+D5GyBQhAo+fcN2o3a0dfh0gZA8Qsb30bIg4Qs87oyxBxgJh1Tt+FiAPErLP6KkQcIGad1zch4gAxy0ZfhIgDxCw7pQ8RB4hZtkodIg4Qs+yVOkQcIGbZK3WI9GuAVJ5unL7drB2ZhoikeyVtlbRZ0vo66yXpUkl3SrpV0oJWP+PQ6dPqtk89QH0ZIACNnmZcrKccW1Hk8dyZMyPi0QbrzgaOTr5OBi5Pvqf26z3170ydegB9GSAAQzMGGK7zjNkhPzLSMtDtw5nXA1dFxU3ADElHtPIGI3vq35v662f793+7Zx4z2FK7WTuyDpEAviNpg6SlddYPAQ9ULW9P2saQtFTSeknrd+7cmVGp5XHjHfX3UaN2s3ZkHSKnRcQCKoctF0k6vWZ9vVN947oQEbEqIhZGxMLBwbH/N53eYLauRudK+sGOOocyE7WbtSPTEImIHcn3R4DrgJNqNtkOvLBq+UhgR9r3X33L/TxdZ7auAwQffd2xLddbFocM1A/QRu1m7cgsRCQdJOng0dfAq4Hbajb7OvDW5CrNKcATEfFgmvcfHUg2a8Zzx63bF7D+vsfb+wcUmC/xWp6y7In8LvBjSVuAW4BvRcQNkpZJWpZscz1wN3AncAXwF2neuHok6o5dv667zVdvur/tf0BR7Xp6T0vtZu3I7BJvRNwNHF+nfWXV6wAuauV9H3/qmTEjUY/58A31P7/FesvkkIFp7BoZHxg+nLEsdPsSb8uGd4307UjUtHw4Y3kqXIgc/JypDpAmfDhjeSpciMw57KAxATIwrf4/oVF7P5jVYGRqo3azdhTuL622S96oR9LPPZXli+YxUPPvH5g2heWL5nWpIiuzPO6dyZS77uMtnl8Z9Lti3TZ27Bph1owBli+at7/drJMKHyKzGtxs1u9d98XzhxwalovCHc7UctfdrLsK3xNx192suwofIuCuu1k3Ff5wxsy6yyFiZm1xiJhZW0pxTmTtpmGfWDXrksKHyNpNw1y8ZisjyYTNw7tGuHjNVgAHiVkOCn84s2Ldtv0BMmpkz15WrNvWpYrM+kvhQ6TeaNWJ2s2sswofIlMaTJLRqN3MOqvwIbK3wWPdGrWbWWcVPkQaPdXNT3szy0fhQ8Q34Jl1V+FDZPH8Ic47cWj/OZApEued6HtpzPJS+BBZu2mYazcM7z8HsjeCazcMs3bTcJcrM+sPhQ8RjxMx667Ch4jHiZh1V+FDxONEzLor8xCRNEXSJknfrLPuDElPSNqcfH2k1ff3OBGz7srjBrx3A78Antdg/Y8i4rWTffOhBhM1e5yIWT4y7YlIOhL4Y+BzWX2Gx4mYdVfWhzP/CPw3YN8E25wqaYukb0s6ttUPWDx/iI+fexxDMwYQlR7Ix889zuNEzHKS2eGMpNcCj0TEBklnNNhsIzAnIp6U9BpgLXB0nfdaCiwFmD179rg38UTNZt2TZU/kNOAcSfcCXwP+UNJXqjeIiN0R8WTy+npgmqSZtW8UEasiYmFELBwcHMywZDNrVWYhEhEXR8SRETEXWAJ8PyIuqN5G0gukyrVYSScl9TyWVU1m1nm5T48oaRlARKwEzgfeIelZYARYEuFrs2ZFoqL9zS5cuDDWr1/f7TLMSk3ShohYmGbbwo9YNbPuKlxPRNJO4L4Gq2cCj+ZYzkR6pRbXMV6v1NIrdcD4WuZERKqrGIULkYlIWp+2C5a1XqnFdYzXK7X0Sh3QXi0+nDGztjhEzKwtZQuRVd0uoEqv1OI6xuuVWnqlDmijllKdEzGz/JWtJ2JmOXOImFlbChkiku6VtDWZDW3c8FVVXCrpTkm3SlrQpTranrmthVpmSLpG0h2SfiHp1Jr1ee2TZnXksk8kzav6jM2Sdkt6T802me+TlHXktU/eK+l2SbdJWi3puTXrJ7c/IqJwX8C9wMwJ1r8G+DYg4BTg5i7VcQbwzZz2yZXAnyWvDwRmdGmfNKsjt31S9ZlTgIeoDKDKfZ+kqCPzfQIMAfcAA8ny/wHe1on9UcieSAqvB66KipuAGZKO6HZRWZH0POB04PMAEfFMROyq2SzzfZKyjm44C7grImpHOuf9e9KojrxMBQYkTQWmAztq1k9qfxQ1RAL4jqQNyYRFtYaAB6qWtydtedcBbc7cltKLgJ3AF1WZFPtzkg6q2SaPfZKmDshnn1RbAqyu057X70mzOiDjfRIRw8AngfuBB4EnIuI7NZtNan8UNUROi4gFwNnARZJOr1lf73kRWVzLblbH6MxtxwOfoTJzWxamAguAyyNiPvAU8MGabfLYJ2nqyGufACDpQOAc4Op6q+u0ZTLmoUkdme8TSYdS6WkcBcwCDpJ0Qe1mdX606f4oZIhExI7k+yPAdcBJNZtsB15YtXwk47tumdcRKWdu64DtwPaIuDlZvobKH3PtNlnvk6Z15LhPRp0NbIyIhxvUm/nvSbM6ctonfwTcExE7I2IPsAZ4Rc02k9ofhQsRSQdJOnj0NfBq4Laazb4OvDU523wKla7bg3nXoZxmbouIh4AHJI1OcX8W8POazTLfJ2nqyGufVHkTjQ8hMt8naerIaZ/cD5wiaXryWWdReZRLtcntjyzPCGd0lvlFwJbk63bgr5P2ZcCy5LWAfwLuArYCC7tUxzuTdVuAm4BXZLhfTgDWA7dS6Q4fmvc+SVlHnvtkOpU/xkOq2rqxT5rVkcs+AS4B7qDyP7svA8/pxP7wsHcza0vhDmfMrLc4RMysLQ4RM2uLQ8TM2uIQMbO2OET6jKTDqu4WfUjScNXygR14/49J+nhN2wmSasck1P7M+9v9bOuO3J+AZ90VEY9RGcuBpI8BT0bEJ0fXS5oaEc+28RGrqdwJenFV2xLgn9t4T+th7okYkr4k6VOSbgT+rrZnkMw/MTd5fYGkW5Key2clTal+r4jYBuySdHJV8xuBr0m6UNLPkhvNrpU0vU4tP5C0MHk9U5UHwiNpiqQVyc/fKunPk/YjJP1LUs9tkl7Z0Z1jTTlEbNRLgT+KiPc12kDSfwD+E5UbD08A9gJvqbPpaiq9D5Lh049FxL8DayLi5VG50ewXwJ+2UN+fUhmG/XLg5cCFko4C3gysS+o5HtjcwntaB/hwxkZdHRF7m2xzFnAi8LPkVo8B4JE6230N+FdJ72Ps7e8vk/TfgRnA7wDrWqjv1cDvSzo/WT4EOBr4GfAFSdOAtRHhEMmZQ8RGPVX1+lnG9lJHp9ETcGVEVJ/vGCciHkgOQ/4AOA8YnSLxS8DiiNgi6W1UZvSqVf3Z1dP3CXhXRIwLnmQKhj8GvixpRURcNVF91lk+nLF67iW5hV+VeTaPStr/H3C+pMOTdc+XNKfBe6wG/oHKTF7bk7aDgQeTXkO9w6DRzz4xeX1+Vfs64B3JzyLppcmd1HOARyLiCiozqmUyd6w15hCxeq4Fni9pM/AO4N8AIuLnwIeozOZ2K/BdoNH0eVcDx1I5tBn1YeDm5OfuaPBzn6QSFv9K5SHToz5HZVqBjZJuAz5LpSd9BrBZ0iYqvZ5Pt/Qvtbb5Ll4za4t7ImbWFoeImbXFIWJmbXGImFlbHCJm1haHiJm1xSFiZm35/3MArgau+f5kAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "scatter_plot = plot_diff(test_Y[0], quality_pred, title='Type')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "include_colab_link": true,
   "name": "exercise-answer.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}