t81_558_class_04_1_feature_encode.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# T81-558: Applications of Deep Neural Networks\n",
    "**Module 4: Training for Tabular Data**\n",
    "* Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), McKelvey School of Engineering, [Washington University in St. Louis](https://engineering.wustl.edu/Programs/Pages/default.aspx)\n",
    "* For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Module 4 Material\n",
    "\n",
    "* **Part 4.1: Encoding a Feature Vector for Keras Deep Learning** [[Video]](https://www.youtube.com/watch?v=Vxz-gfs9nMQ&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_04_1_feature_encode.ipynb)\n",
    "* Part 4.2: Keras Multiclass Classification for Deep Neural Networks with ROC and AUC [[Video]](https://www.youtube.com/watch?v=-f3bg9dLMks&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_04_2_multi_class.ipynb)\n",
    "* Part 4.3: Keras Regression for Deep Neural Networks with RMSE [[Video]](https://www.youtube.com/watch?v=wNhBUC6X5-E&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_04_3_regression.ipynb)\n",
    "* Part 4.4: Backpropagation, Nesterov Momentum, and ADAM Neural Network Training [[Video]](https://www.youtube.com/watch?v=VbDg8aBgpck&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_04_4_backprop.ipynb)\n",
    "* Part 4.5: Neural Network RMSE and Log Loss Error Calculation from Scratch [[Video]](https://www.youtube.com/watch?v=wmQX1t2PHJc&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_04_5_rmse_logloss.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 4.1: Encoding a Feature Vector for Keras Deep Learning\n",
    "\n",
    "Neural networks can accept many types of data.  We will begin with tabular data, where there are well defined rows and columns.  This is the sort of data you would typically see in Microsoft Excel.  An example of tabular data is shown below.\n",
    "\n",
    "Neural networks require numeric input.  This numeric form is called a feature vector.  Each row of training data typically becomes one vector.  The individual input neurons each receive one feature (or column) from this vector.  In this section, we will see how to encode the following tabular data into a feature vector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>job</th>\n",
       "      <th>area</th>\n",
       "      <th>income</th>\n",
       "      <th>aspect</th>\n",
       "      <th>subscriptions</th>\n",
       "      <th>dist_healthy</th>\n",
       "      <th>save_rate</th>\n",
       "      <th>dist_unhealthy</th>\n",
       "      <th>age</th>\n",
       "      <th>pop_dense</th>\n",
       "      <th>retail_dense</th>\n",
       "      <th>crime</th>\n",
       "      <th>product</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>vv</td>\n",
       "      <td>c</td>\n",
       "      <td>50876.0</td>\n",
       "      <td>13.100000</td>\n",
       "      <td>1</td>\n",
       "      <td>9.017895</td>\n",
       "      <td>35</td>\n",
       "      <td>11.738935</td>\n",
       "      <td>49</td>\n",
       "      <td>0.885827</td>\n",
       "      <td>0.492126</td>\n",
       "      <td>0.071100</td>\n",
       "      <td>b</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>kd</td>\n",
       "      <td>c</td>\n",
       "      <td>60369.0</td>\n",
       "      <td>18.625000</td>\n",
       "      <td>2</td>\n",
       "      <td>7.766643</td>\n",
       "      <td>59</td>\n",
       "      <td>6.805396</td>\n",
       "      <td>51</td>\n",
       "      <td>0.874016</td>\n",
       "      <td>0.342520</td>\n",
       "      <td>0.400809</td>\n",
       "      <td>c</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>pe</td>\n",
       "      <td>c</td>\n",
       "      <td>55126.0</td>\n",
       "      <td>34.766667</td>\n",
       "      <td>1</td>\n",
       "      <td>3.632069</td>\n",
       "      <td>6</td>\n",
       "      <td>13.671772</td>\n",
       "      <td>44</td>\n",
       "      <td>0.944882</td>\n",
       "      <td>0.724409</td>\n",
       "      <td>0.207723</td>\n",
       "      <td>b</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>11</td>\n",
       "      <td>c</td>\n",
       "      <td>51690.0</td>\n",
       "      <td>15.808333</td>\n",
       "      <td>1</td>\n",
       "      <td>5.372942</td>\n",
       "      <td>16</td>\n",
       "      <td>4.333286</td>\n",
       "      <td>50</td>\n",
       "      <td>0.889764</td>\n",
       "      <td>0.444882</td>\n",
       "      <td>0.361216</td>\n",
       "      <td>b</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>kl</td>\n",
       "      <td>d</td>\n",
       "      <td>28347.0</td>\n",
       "      <td>40.941667</td>\n",
       "      <td>3</td>\n",
       "      <td>3.822477</td>\n",
       "      <td>20</td>\n",
       "      <td>5.967121</td>\n",
       "      <td>38</td>\n",
       "      <td>0.744094</td>\n",
       "      <td>0.661417</td>\n",
       "      <td>0.068033</td>\n",
       "      <td>a</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id job area   income     aspect  subscriptions  dist_healthy  save_rate  \\\n",
       "0   1  vv    c  50876.0  13.100000              1      9.017895         35   \n",
       "1   2  kd    c  60369.0  18.625000              2      7.766643         59   \n",
       "2   3  pe    c  55126.0  34.766667              1      3.632069          6   \n",
       "3   4  11    c  51690.0  15.808333              1      5.372942         16   \n",
       "4   5  kl    d  28347.0  40.941667              3      3.822477         20   \n",
       "\n",
       "   dist_unhealthy  age  pop_dense  retail_dense     crime product  \n",
       "0       11.738935   49   0.885827      0.492126  0.071100       b  \n",
       "1        6.805396   51   0.874016      0.342520  0.400809       c  \n",
       "2       13.671772   44   0.944882      0.724409  0.207723       b  \n",
       "3        4.333286   50   0.889764      0.444882  0.361216       b  \n",
       "4        5.967121   38   0.744094      0.661417  0.068033       a  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv(\n",
    "    \"https://data.heatonresearch.com/data/t81-558/jh-simple-dataset.csv\",\n",
    "    na_values=['NA','?'])\n",
    "\n",
    "display(df[0:5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following observations can be made from the above data:\n",
    "* The target column is the column that you seek to predict.  There are several candidates here.  However, we will initially use product.  This field specifies what product someone bought.\n",
    "* There is an ID column.  This column should not be fed into the neural network as it contains no information useful for prediction.\n",
    "* Many of these fields are numeric and might not require any further processing.\n",
    "* The income column does have some missing values.\n",
    "* There are categorical values: job, area, and product.\n",
    "\n",
    "To begin with, we will convert the job code into dummy variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2000, 33)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>job_11</th>\n",
       "      <th>job_al</th>\n",
       "      <th>job_am</th>\n",
       "      <th>job_ax</th>\n",
       "      <th>job_bf</th>\n",
       "      <th>job_by</th>\n",
       "      <th>job_cv</th>\n",
       "      <th>job_de</th>\n",
       "      <th>job_dz</th>\n",
       "      <th>job_e2</th>\n",
       "      <th>...</th>\n",
       "      <th>job_pe</th>\n",
       "      <th>job_po</th>\n",
       "      <th>job_pq</th>\n",
       "      <th>job_pz</th>\n",
       "      <th>job_qp</th>\n",
       "      <th>job_qw</th>\n",
       "      <th>job_rn</th>\n",
       "      <th>job_sa</th>\n",
       "      <th>job_vv</th>\n",
       "      <th>job_zz</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10 rows × 33 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   job_11  job_al  job_am  job_ax  job_bf  job_by  job_cv  job_de  job_dz  \\\n",
       "0       0       0       0       0       0       0       0       0       0   \n",
       "1       0       0       0       0       0       0       0       0       0   \n",
       "2       0       0       0       0       0       0       0       0       0   \n",
       "3       1       0       0       0       0       0       0       0       0   \n",
       "4       0       0       0       0       0       0       0       0       0   \n",
       "5       0       0       0       0       0       0       0       0       0   \n",
       "6       0       0       0       0       0       0       0       0       0   \n",
       "7       0       0       0       0       0       0       0       0       0   \n",
       "8       0       1       0       0       0       0       0       0       0   \n",
       "9       0       0       0       0       0       0       0       0       0   \n",
       "\n",
       "   job_e2  ...  job_pe  job_po  job_pq  job_pz  job_qp  job_qw  job_rn  \\\n",
       "0       0  ...       0       0       0       0       0       0       0   \n",
       "1       0  ...       0       0       0       0       0       0       0   \n",
       "2       0  ...       1       0       0       0       0       0       0   \n",
       "3       0  ...       0       0       0       0       0       0       0   \n",
       "4       0  ...       0       0       0       0       0       0       0   \n",
       "5       1  ...       0       0       0       0       0       0       0   \n",
       "6       0  ...       0       0       0       0       0       0       0   \n",
       "7       0  ...       0       0       0       0       0       0       0   \n",
       "8       0  ...       0       0       0       0       0       0       0   \n",
       "9       0  ...       1       0       0       0       0       0       0   \n",
       "\n",
       "   job_sa  job_vv  job_zz  \n",
       "0       0       1       0  \n",
       "1       0       0       0  \n",
       "2       0       0       0  \n",
       "3       0       0       0  \n",
       "4       0       0       0  \n",
       "5       0       0       0  \n",
       "6       0       0       0  \n",
       "7       0       0       0  \n",
       "8       0       0       0  \n",
       "9       0       0       0  \n",
       "\n",
       "[10 rows x 33 columns]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "dummies = pd.get_dummies(df['job'],prefix=\"job\")\n",
    "print(dummies.shape)\n",
    "display(dummies[0:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because there are 33 different job codes, there are 33 dummy variables.  We also specified a prefix, because the job codes (such as \"ax\") are not that meaningful by themselves.  Something such as \"job_ax\" also tells us the origin of this field.\n",
    "\n",
    "Next, we must merge these dummies back into the main data frame.  We also drop the original \"job\" field, as it is now represented by the dummies. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>area</th>\n",
       "      <th>income</th>\n",
       "      <th>aspect</th>\n",
       "      <th>subscriptions</th>\n",
       "      <th>dist_healthy</th>\n",
       "      <th>save_rate</th>\n",
       "      <th>dist_unhealthy</th>\n",
       "      <th>age</th>\n",
       "      <th>pop_dense</th>\n",
       "      <th>...</th>\n",
       "      <th>job_pe</th>\n",
       "      <th>job_po</th>\n",
       "      <th>job_pq</th>\n",
       "      <th>job_pz</th>\n",
       "      <th>job_qp</th>\n",
       "      <th>job_qw</th>\n",
       "      <th>job_rn</th>\n",
       "      <th>job_sa</th>\n",
       "      <th>job_vv</th>\n",
       "      <th>job_zz</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>c</td>\n",
       "      <td>50876.0</td>\n",
       "      <td>13.100000</td>\n",
       "      <td>1</td>\n",
       "      <td>9.017895</td>\n",
       "      <td>35</td>\n",
       "      <td>11.738935</td>\n",
       "      <td>49</td>\n",
       "      <td>0.885827</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>c</td>\n",
       "      <td>60369.0</td>\n",
       "      <td>18.625000</td>\n",
       "      <td>2</td>\n",
       "      <td>7.766643</td>\n",
       "      <td>59</td>\n",
       "      <td>6.805396</td>\n",
       "      <td>51</td>\n",
       "      <td>0.874016</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>c</td>\n",
       "      <td>55126.0</td>\n",
       "      <td>34.766667</td>\n",
       "      <td>1</td>\n",
       "      <td>3.632069</td>\n",
       "      <td>6</td>\n",
       "      <td>13.671772</td>\n",
       "      <td>44</td>\n",
       "      <td>0.944882</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>c</td>\n",
       "      <td>51690.0</td>\n",
       "      <td>15.808333</td>\n",
       "      <td>1</td>\n",
       "      <td>5.372942</td>\n",
       "      <td>16</td>\n",
       "      <td>4.333286</td>\n",
       "      <td>50</td>\n",
       "      <td>0.889764</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>d</td>\n",
       "      <td>28347.0</td>\n",
       "      <td>40.941667</td>\n",
       "      <td>3</td>\n",
       "      <td>3.822477</td>\n",
       "      <td>20</td>\n",
       "      <td>5.967121</td>\n",
       "      <td>38</td>\n",
       "      <td>0.744094</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>c</td>\n",
       "      <td>70854.0</td>\n",
       "      <td>40.400000</td>\n",
       "      <td>1</td>\n",
       "      <td>14.893343</td>\n",
       "      <td>87</td>\n",
       "      <td>20.340593</td>\n",
       "      <td>43</td>\n",
       "      <td>0.866142</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>d</td>\n",
       "      <td>38726.0</td>\n",
       "      <td>30.975000</td>\n",
       "      <td>3</td>\n",
       "      <td>3.822477</td>\n",
       "      <td>33</td>\n",
       "      <td>9.480399</td>\n",
       "      <td>39</td>\n",
       "      <td>0.976378</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>a</td>\n",
       "      <td>55162.0</td>\n",
       "      <td>26.966667</td>\n",
       "      <td>2</td>\n",
       "      <td>4.312097</td>\n",
       "      <td>17</td>\n",
       "      <td>29.219896</td>\n",
       "      <td>44</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>c</td>\n",
       "      <td>67311.0</td>\n",
       "      <td>32.383333</td>\n",
       "      <td>0</td>\n",
       "      <td>25.093772</td>\n",
       "      <td>169</td>\n",
       "      <td>10.927357</td>\n",
       "      <td>45</td>\n",
       "      <td>0.952756</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>a</td>\n",
       "      <td>63344.0</td>\n",
       "      <td>38.233333</td>\n",
       "      <td>1</td>\n",
       "      <td>2.816034</td>\n",
       "      <td>3</td>\n",
       "      <td>21.915695</td>\n",
       "      <td>42</td>\n",
       "      <td>0.897638</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10 rows × 46 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   id area   income     aspect  subscriptions  dist_healthy  save_rate  \\\n",
       "0   1    c  50876.0  13.100000              1      9.017895         35   \n",
       "1   2    c  60369.0  18.625000              2      7.766643         59   \n",
       "2   3    c  55126.0  34.766667              1      3.632069          6   \n",
       "3   4    c  51690.0  15.808333              1      5.372942         16   \n",
       "4   5    d  28347.0  40.941667              3      3.822477         20   \n",
       "5   6    c  70854.0  40.400000              1     14.893343         87   \n",
       "6   7    d  38726.0  30.975000              3      3.822477         33   \n",
       "7   8    a  55162.0  26.966667              2      4.312097         17   \n",
       "8   9    c  67311.0  32.383333              0     25.093772        169   \n",
       "9  10    a  63344.0  38.233333              1      2.816034          3   \n",
       "\n",
       "   dist_unhealthy  age  pop_dense  ...  job_pe  job_po job_pq  job_pz  job_qp  \\\n",
       "0       11.738935   49   0.885827  ...       0       0      0       0       0   \n",
       "1        6.805396   51   0.874016  ...       0       0      0       0       0   \n",
       "2       13.671772   44   0.944882  ...       1       0      0       0       0   \n",
       "3        4.333286   50   0.889764  ...       0       0      0       0       0   \n",
       "4        5.967121   38   0.744094  ...       0       0      0       0       0   \n",
       "5       20.340593   43   0.866142  ...       0       0      0       0       0   \n",
       "6        9.480399   39   0.976378  ...       0       0      0       0       0   \n",
       "7       29.219896   44   1.000000  ...       0       0      0       0       0   \n",
       "8       10.927357   45   0.952756  ...       0       0      0       0       0   \n",
       "9       21.915695   42   0.897638  ...       1       0      0       0       0   \n",
       "\n",
       "   job_qw  job_rn  job_sa  job_vv  job_zz  \n",
       "0       0       0       0       1       0  \n",
       "1       0       0       0       0       0  \n",
       "2       0       0       0       0       0  \n",
       "3       0       0       0       0       0  \n",
       "4       0       0       0       0       0  \n",
       "5       0       0       0       0       0  \n",
       "6       0       0       0       0       0  \n",
       "7       0       0       0       0       0  \n",
       "8       0       0       0       0       0  \n",
       "9       0       0       0       0       0  \n",
       "\n",
       "[10 rows x 46 columns]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df = pd.concat([df,dummies],axis=1)\n",
    "df.drop('job', axis=1, inplace=True)\n",
    "display(df[0:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also introduce dummy variables for the area column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>income</th>\n",
       "      <th>aspect</th>\n",
       "      <th>subscriptions</th>\n",
       "      <th>dist_healthy</th>\n",
       "      <th>save_rate</th>\n",
       "      <th>dist_unhealthy</th>\n",
       "      <th>age</th>\n",
       "      <th>pop_dense</th>\n",
       "      <th>retail_dense</th>\n",
       "      <th>...</th>\n",
       "      <th>job_qp</th>\n",
       "      <th>job_qw</th>\n",
       "      <th>job_rn</th>\n",
       "      <th>job_sa</th>\n",
       "      <th>job_vv</th>\n",
       "      <th>job_zz</th>\n",
       "      <th>area_a</th>\n",
       "      <th>area_b</th>\n",
       "      <th>area_c</th>\n",
       "      <th>area_d</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>50876.0</td>\n",
       "      <td>13.100000</td>\n",
       "      <td>1</td>\n",
       "      <td>9.017895</td>\n",
       "      <td>35</td>\n",
       "      <td>11.738935</td>\n",
       "      <td>49</td>\n",
       "      <td>0.885827</td>\n",
       "      <td>0.492126</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>60369.0</td>\n",
       "      <td>18.625000</td>\n",
       "      <td>2</td>\n",
       "      <td>7.766643</td>\n",
       "      <td>59</td>\n",
       "      <td>6.805396</td>\n",
       "      <td>51</td>\n",
       "      <td>0.874016</td>\n",
       "      <td>0.342520</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>55126.0</td>\n",
       "      <td>34.766667</td>\n",
       "      <td>1</td>\n",
       "      <td>3.632069</td>\n",
       "      <td>6</td>\n",
       "      <td>13.671772</td>\n",
       "      <td>44</td>\n",
       "      <td>0.944882</td>\n",
       "      <td>0.724409</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>51690.0</td>\n",
       "      <td>15.808333</td>\n",
       "      <td>1</td>\n",
       "      <td>5.372942</td>\n",
       "      <td>16</td>\n",
       "      <td>4.333286</td>\n",
       "      <td>50</td>\n",
       "      <td>0.889764</td>\n",
       "      <td>0.444882</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>28347.0</td>\n",
       "      <td>40.941667</td>\n",
       "      <td>3</td>\n",
       "      <td>3.822477</td>\n",
       "      <td>20</td>\n",
       "      <td>5.967121</td>\n",
       "      <td>38</td>\n",
       "      <td>0.744094</td>\n",
       "      <td>0.661417</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>70854.0</td>\n",
       "      <td>40.400000</td>\n",
       "      <td>1</td>\n",
       "      <td>14.893343</td>\n",
       "      <td>87</td>\n",
       "      <td>20.340593</td>\n",
       "      <td>43</td>\n",
       "      <td>0.866142</td>\n",
       "      <td>0.673228</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>38726.0</td>\n",
       "      <td>30.975000</td>\n",
       "      <td>3</td>\n",
       "      <td>3.822477</td>\n",
       "      <td>33</td>\n",
       "      <td>9.480399</td>\n",
       "      <td>39</td>\n",
       "      <td>0.976378</td>\n",
       "      <td>0.874016</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>55162.0</td>\n",
       "      <td>26.966667</td>\n",
       "      <td>2</td>\n",
       "      <td>4.312097</td>\n",
       "      <td>17</td>\n",
       "      <td>29.219896</td>\n",
       "      <td>44</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.724409</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>67311.0</td>\n",
       "      <td>32.383333</td>\n",
       "      <td>0</td>\n",
       "      <td>25.093772</td>\n",
       "      <td>169</td>\n",
       "      <td>10.927357</td>\n",
       "      <td>45</td>\n",
       "      <td>0.952756</td>\n",
       "      <td>0.681102</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>63344.0</td>\n",
       "      <td>38.233333</td>\n",
       "      <td>1</td>\n",
       "      <td>2.816034</td>\n",
       "      <td>3</td>\n",
       "      <td>21.915695</td>\n",
       "      <td>42</td>\n",
       "      <td>0.897638</td>\n",
       "      <td>0.724409</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10 rows × 49 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   id   income     aspect  subscriptions  dist_healthy  save_rate  \\\n",
       "0   1  50876.0  13.100000              1      9.017895         35   \n",
       "1   2  60369.0  18.625000              2      7.766643         59   \n",
       "2   3  55126.0  34.766667              1      3.632069          6   \n",
       "3   4  51690.0  15.808333              1      5.372942         16   \n",
       "4   5  28347.0  40.941667              3      3.822477         20   \n",
       "5   6  70854.0  40.400000              1     14.893343         87   \n",
       "6   7  38726.0  30.975000              3      3.822477         33   \n",
       "7   8  55162.0  26.966667              2      4.312097         17   \n",
       "8   9  67311.0  32.383333              0     25.093772        169   \n",
       "9  10  63344.0  38.233333              1      2.816034          3   \n",
       "\n",
       "   dist_unhealthy  age  pop_dense  retail_dense  ...  job_qp job_qw  job_rn  \\\n",
       "0       11.738935   49   0.885827      0.492126  ...       0      0       0   \n",
       "1        6.805396   51   0.874016      0.342520  ...       0      0       0   \n",
       "2       13.671772   44   0.944882      0.724409  ...       0      0       0   \n",
       "3        4.333286   50   0.889764      0.444882  ...       0      0       0   \n",
       "4        5.967121   38   0.744094      0.661417  ...       0      0       0   \n",
       "5       20.340593   43   0.866142      0.673228  ...       0      0       0   \n",
       "6        9.480399   39   0.976378      0.874016  ...       0      0       0   \n",
       "7       29.219896   44   1.000000      0.724409  ...       0      0       0   \n",
       "8       10.927357   45   0.952756      0.681102  ...       0      0       0   \n",
       "9       21.915695   42   0.897638      0.724409  ...       0      0       0   \n",
       "\n",
       "   job_sa  job_vv  job_zz  area_a  area_b  area_c  area_d  \n",
       "0       0       1       0       0       0       1       0  \n",
       "1       0       0       0       0       0       1       0  \n",
       "2       0       0       0       0       0       1       0  \n",
       "3       0       0       0       0       0       1       0  \n",
       "4       0       0       0       0       0       0       1  \n",
       "5       0       0       0       0       0       1       0  \n",
       "6       0       0       0       0       0       0       1  \n",
       "7       0       0       0       1       0       0       0  \n",
       "8       0       0       0       0       0       1       0  \n",
       "9       0       0       0       1       0       0       0  \n",
       "\n",
       "[10 rows x 49 columns]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df = pd.concat([df,pd.get_dummies(df['area'],prefix=\"area\")],axis=1)\n",
    "df.drop('area', axis=1, inplace=True)\n",
    "display(df[0:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The last remaining transformation is to fill in missing income values. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "med = df['income'].median()\n",
    "df['income'] = df['income'].fillna(med)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are more advanced ways of filling in missing values, but they require more analysis.  The idea would be to see if another field might give a hint as to what the income were.  For example, it might be beneficial to calculate a median income for each of the areas or job categories.  This is something to keep in mind for the class Kaggle competition.\n",
    "\n",
    "At this point, the Pandas dataframe is ready to be converted to Numpy for neural network training. We need to know a list of the columns that will make up *x* (the predictors or inputs) and *y* (the target). \n",
    "\n",
    "The complete list of columns is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['id', 'income', 'aspect', 'subscriptions', 'dist_healthy', 'save_rate', 'dist_unhealthy', 'age', 'pop_dense', 'retail_dense', 'crime', 'product', 'job_11', 'job_al', 'job_am', 'job_ax', 'job_bf', 'job_by', 'job_cv', 'job_de', 'job_dz', 'job_e2', 'job_f8', 'job_gj', 'job_gv', 'job_kd', 'job_ke', 'job_kl', 'job_kp', 'job_ks', 'job_kw', 'job_mm', 'job_nb', 'job_nn', 'job_ob', 'job_pe', 'job_po', 'job_pq', 'job_pz', 'job_qp', 'job_qw', 'job_rn', 'job_sa', 'job_vv', 'job_zz', 'area_a', 'area_b', 'area_c', 'area_d']\n"
     ]
    }
   ],
   "source": [
    "print(list(df.columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This includes both the target and predictors.  We need a list with the target removed.  We also remove **id** because it is not useful for prediction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['income', 'aspect', 'subscriptions', 'dist_healthy', 'save_rate', 'dist_unhealthy', 'age', 'pop_dense', 'retail_dense', 'crime', 'job_11', 'job_al', 'job_am', 'job_ax', 'job_bf', 'job_by', 'job_cv', 'job_de', 'job_dz', 'job_e2', 'job_f8', 'job_gj', 'job_gv', 'job_kd', 'job_ke', 'job_kl', 'job_kp', 'job_ks', 'job_kw', 'job_mm', 'job_nb', 'job_nn', 'job_ob', 'job_pe', 'job_po', 'job_pq', 'job_pz', 'job_qp', 'job_qw', 'job_rn', 'job_sa', 'job_vv', 'job_zz', 'area_a', 'area_b', 'area_c', 'area_d']\n"
     ]
    }
   ],
   "source": [
    "x_columns = df.columns.drop('product').drop('id')\n",
    "print(list(x_columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generate X and Y for a Classification Neural Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now generate *x* and *y*.  Note, this is how we generate y for a classification problem.  Regression would not use dummies and would simply encode the numeric value of the target."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Convert to numpy - Classification\n",
    "x_columns = df.columns.drop('product').drop('id')\n",
    "x = df[x_columns].values\n",
    "dummies = pd.get_dummies(df['product']) # Classification\n",
    "products = dummies.columns\n",
    "y = dummies.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can display the *x* and *y* matrices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[5.08760000e+04 1.31000000e+01 1.00000000e+00 ... 0.00000000e+00\n",
      "  1.00000000e+00 0.00000000e+00]\n",
      " [6.03690000e+04 1.86250000e+01 2.00000000e+00 ... 0.00000000e+00\n",
      "  1.00000000e+00 0.00000000e+00]\n",
      " [5.51260000e+04 3.47666667e+01 1.00000000e+00 ... 0.00000000e+00\n",
      "  1.00000000e+00 0.00000000e+00]\n",
      " ...\n",
      " [2.85950000e+04 3.94250000e+01 3.00000000e+00 ... 0.00000000e+00\n",
      "  0.00000000e+00 1.00000000e+00]\n",
      " [6.79490000e+04 5.73333333e+00 0.00000000e+00 ... 0.00000000e+00\n",
      "  1.00000000e+00 0.00000000e+00]\n",
      " [6.14670000e+04 1.68916667e+01 0.00000000e+00 ... 0.00000000e+00\n",
      "  1.00000000e+00 0.00000000e+00]]\n",
      "[[0 1 0 ... 0 0 0]\n",
      " [0 0 1 ... 0 0 0]\n",
      " [0 1 0 ... 0 0 0]\n",
      " ...\n",
      " [0 0 0 ... 0 1 0]\n",
      " [0 0 1 ... 0 0 0]\n",
      " [0 0 1 ... 0 0 0]]\n"
     ]
    }
   ],
   "source": [
    "print(x)\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The x and y values are now ready for a neural network.  Make sure that you construct the neural network for a classification problem.  Specifically,\n",
    "\n",
    "* Classification neural networks have an output neuron count equal to the number of classes.\n",
    "* Classification neural networks should use **categorical_crossentropy** and a **softmax** activation function on the output layer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generate X and Y for a Regression Neural Network\n",
    "\n",
    "For a regression neural network, the *x* values are generated the same.  However, *y* does not use dummies.  Make sure to replace **income** with your actual target."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = df['income'].values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Module 4 Assignment\n",
    "\n",
    "You can find the first assignment here: [assignment 4](https://github.com/jeffheaton/t81_558_deep_learning/blob/master/assignments/assignment_yourname_class1.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3.6 (tensorflow)",
   "language": "python",
   "name": "rga"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}