final_model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "alone-doubt",
   "metadata": {},
   "source": [
    "# Final Model Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sustainable-meter",
   "metadata": {},
   "source": [
    "<img src='mi_diagrama.png'>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "academic-march",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/jose/Desktop/Udacity_Project/tools.py:16: FutureWarning: Passing a negative integer is deprecated in version 1.0 and will not be supported in future version. Instead, use None to not limit the column width.\n",
      "  pd.set_option('display.max_colwidth', -1)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import math\n",
    "from tools import *\n",
    "\n",
    "# tfidf:\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "# models:\n",
    "from sklearn.linear_model import Ridge\n",
    "from sklearn.svm import SVR\n",
    "from sklearn.neural_network import MLPRegressor\n",
    "# metrics:\n",
    "from sklearn.metrics import mean_squared_error, r2_score, mean_absolute_error\n",
    "# pipeline:\n",
    "from sklearn.pipeline import Pipeline\n",
    "# for pre-processing:\n",
    "from nltk.stem import WordNetLemmatizer, PorterStemmer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "collect-charles",
   "metadata": {},
   "outputs": [],
   "source": [
    "def add_creative_features(dataframe):\n",
    "    \"\"\"\n",
    "    This function applies the tools.py functions I wrote into every excerpt.\n",
    "    \"\"\"\n",
    "    dataframe['num_punct_marks'] = dataframe['excerpt'].apply(num_punct_marks)\n",
    "    dataframe['num_uniq_words'] = dataframe['excerpt'].apply(num_unique_words)\n",
    "    dataframe['avg_word_len'] = dataframe['excerpt'].apply(avg_word_len)\n",
    "    dataframe['rarity'] = dataframe['excerpt'].apply(rarity)\n",
    "\n",
    "\n",
    "def add_clasic_test(dataframe):\n",
    "    \"\"\"\n",
    "    This function applies all the publicly available readability test into every excerpt.\n",
    "    \"\"\"\n",
    "    clasical_complexity_tests = {'fre_test': textstat.flesch_reading_ease,\n",
    "                                 'fkg_test': textstat.flesch_kincaid_grade,\n",
    "                                 'gf_test': textstat.gunning_fog,\n",
    "                                 'si_test': textstat.smog_index,\n",
    "                                 'dcrs_test': textstat.dale_chall_readability_score}\n",
    "\n",
    "    # Creating text complexity feature for every test:\n",
    "    for test in clasical_complexity_tests.keys():\n",
    "        test_func = clasical_complexity_tests[test]\n",
    "\n",
    "        dataframe[test] = dataframe['excerpt'].apply(lambda value: test_func(value))\n",
    "\n",
    "        \n",
    "def clean_stem_and_lemmatize(dataframe):\n",
    "    \"\"\"lemmatize and stem excerpts to improve the TF-IDF process\"\"\"\n",
    "    texts = dataframe['excerpt'].values\n",
    "    \n",
    "    cleaned_texts = []\n",
    "    for text in texts:\n",
    "        text = text.lower()\n",
    "        \n",
    "        # bye bye punctuation marks\n",
    "        text = text.replace('?', '').replace('.', '').replace(',', '')\n",
    "        text = text.replace(':', '').replace(';', '').replace('!', '')\n",
    "        text = text.replace('(', '').replace(')', '')\n",
    "\n",
    "        # streaming:\n",
    "        ps = PorterStemmer()\n",
    "        stems = [ps.stem(word) for word in text.split()]\n",
    "        text = ' '.join(stems)\n",
    "\n",
    "        # lemmatizing:\n",
    "        wnl = WordNetLemmatizer()\n",
    "        lemma = [wnl.lemmatize(word) for word in text.split()]\n",
    "        text = ' '.join(lemma)\n",
    "        \n",
    "        cleaned_texts.append(text)\n",
    "\n",
    "    dataframe['excerpt'] = cleaned_texts"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "seasonal-magic",
   "metadata": {},
   "source": [
    "## Final Model\n",
    "\n",
    "* After a lot of iteration and testing, this is the best model I have found so far.\n",
    "* The main improvement here is adding the TF-IDF (Term frequency - Inverse Frequency) pre-procesing to the mix.\n",
    "* Also I tweak the hyperparameters of best models to find the best combination.\n",
    "\n",
    "#### I'm going to import the train.csv again to mantain order:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "secondary-wiring",
   "metadata": {},
   "outputs": [],
   "source": [
    "train = pd.read_csv('train.csv', usecols=['id', 'excerpt', 'target'])\n",
    "validation = pd.read_csv('validation.csv', usecols=['id', 'excerpt', 'target'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bottom-cooper",
   "metadata": {},
   "source": [
    "#### To ilustrate the data-cleaning process here you have a before and after of the excerptwhen you apply the streaming and lemmatization:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "parallel-luxembourg",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"An earthquake (also known as a quake, tremor or temblor) is the perceptible shaking of the surface of the Earth, resulting from the sudden release of energy in the Earth's crust that creates seismic waves. Earthquakes can be violent enough to toss people around and destroy whole cities. The seismicity or seismic activity of an area refers to the frequency, type and size of earthquakes experienced over a period of time.\\nEarthquakes are measured using observations from seismometers. The moment magnitude is the most common scale on which earthquakes larger than approximately 5 are reported for the entire globe. The more numerous earthquakes smaller than magnitude 5 reported by national seismological observatories are measured mostly on the local magnitude scale, also referred to as the Richter magnitude scale. These two scales are numerically similar over their range of validity. Magnitude 3 or lower earthquakes are mostly imperceptible or weak and magnitude 7 and over potentially cause serious damage over larger areas, depending on their depth.\""
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# before:\n",
    "train['excerpt'].iloc[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "appointed-vintage",
   "metadata": {},
   "outputs": [],
   "source": [
    "add_creative_features(train)\n",
    "add_clasic_test(train)\n",
    "clean_stem_and_lemmatize(train)\n",
    "\n",
    "add_creative_features(validation)\n",
    "add_clasic_test(validation)\n",
    "clean_stem_and_lemmatize(validation)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "cognitive-satisfaction",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"an earthquak also known a a quak tremor or temblor is the percept shake of the surfac of the earth result from the sudden releas of energi in the earth' crust that creat seismic wave earthquak can be violent enough to toss peopl around and destroy whole citi the seismic or seismic activ of an area refer to the frequenc type and size of earthquak experienc over a period of time earthquak are measur use observ from seismomet the moment magnitud is the most common scale on which earthquak larger than approxim 5 are report for the entir globe the more numer earthquak smaller than magnitud 5 report by nation seismolog observatori are measur mostli on the local magnitud scale also refer to a the richter magnitud scale these two scale are numer similar over their rang of valid magnitud 3 or lower earthquak are mostli impercept or weak and magnitud 7 and over potenti caus seriou damag over larger area depend on their depth\""
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# after:\n",
    "train['excerpt'].iloc[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "finished-donna",
   "metadata": {},
   "source": [
    "##### As you can see, the text have been stemmed and lemmatized properly!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "expressed-minutes",
   "metadata": {},
   "source": [
    "# Background: TF-IDF "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "surprising-discipline",
   "metadata": {},
   "source": [
    "TD-IDF: aka \"Term Frequency — Inverse Document Frequency” is a numerical statistic that is intended to reflect how important a word is to a document in a collection or corpus. It is often used as a weighting factor in searches of information retrieval, text mining, and user modeling."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "increased-classification",
   "metadata": {},
   "source": [
    "So in this case we will use the whole training exceprts as the corpus, an the TFIDF pre processing will help us determine which words are giving more information about the complexity of the text."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "prescribed-notification",
   "metadata": {},
   "source": [
    "<img src='https://miro.medium.com/max/1200/1*qQgnyPLDIkUmeZKN2_ZWbQ.png'>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "seven-vaccine",
   "metadata": {},
   "source": [
    "The tf–idf value increases proportionally to the number of times a word appears in the document and is offset by the number of documents in the corpus that contain the word, which helps to adjust for the fact that some words appear more frequently in general."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "detected-royalty",
   "metadata": {},
   "source": [
    "# Background: TF-IDF in sklearn"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "respected-lender",
   "metadata": {},
   "source": [
    "Sklearn helps to transform the raw texts in the corpus to a normalized tf-idf representation. This TF-IDF representation is a matrix that contains the weights each word in the corpus have."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "figured-auction",
   "metadata": {},
   "source": [
    "<img src='https://content.codecademy.com/programs/chatbots/tfidf/tfidf-transformer.gif'>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "liable-porter",
   "metadata": {},
   "source": [
    "You may ask, why is this useful to detect the complexity? easy, it helps reduce the ammount of words that doesn't contribute to understand how complex a text is, then we can focus more on the ocurrence of the 'hard' words more."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "referenced-local",
   "metadata": {},
   "source": [
    "Bellow you will see how this sparce matrix that represents the text, gives a nice numeric input to the ridge regression model, which will come to a nice solution to the complexity determination. Notice that the way I used the TfidfVectorizer was inside an sklearn Pipeline to ease the .fit() process of the regression model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "lasting-local",
   "metadata": {},
   "source": [
    "### So let's find the best model using the TF-IDF matrix as input:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "brave-cooler",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MLPRegressor vs target... R2: 0.517\n",
      "MLPRegressor  vs target... MAE: 0.569\n",
      "MLPRegressor  vs target... RMSE: 0.707 \n",
      "\n",
      "SVR vs target... R2: 0.518\n",
      "SVR  vs target... MAE: 0.581\n",
      "SVR  vs target... RMSE: 0.707 \n",
      "\n",
      "Ridge vs target... R2: 0.551\n",
      "Ridge  vs target... MAE: 0.566\n",
      "Ridge  vs target... RMSE: 0.682 \n",
      "\n"
     ]
    }
   ],
   "source": [
    "X_train = train['excerpt']\n",
    "y_train = train['target']\n",
    "\n",
    "X_val = validation['excerpt']\n",
    "y_val = validation['target']\n",
    "\n",
    "models = {'MLPRegressor': MLPRegressor(),\n",
    "          'SVR': SVR(kernel='rbf'),\n",
    "          'Ridge': Ridge()}\n",
    "\n",
    "for model, regr in models.items():\n",
    "    pipeline = Pipeline([('tfidf', TfidfVectorizer()), ('rgr', regr)])\n",
    "    pipeline.fit(X_train, y_train)\n",
    "    \n",
    "    # validation predictions\n",
    "    preds = pipeline.predict(X_val)\n",
    "    \n",
    "    # metrics\n",
    "    r2 = round(pipeline.score(X_val, y_val), 3)\n",
    "    mae = round(mean_absolute_error(y_val, preds), 3)\n",
    "    rmse = round(math.sqrt(mean_squared_error(y_val, preds)), 3)\n",
    "    \n",
    "    # Printing results\n",
    "    print(f'{model} vs target... R2: {r2}')\n",
    "    print(f'{model}  vs target... MAE: {mae}')\n",
    "    print(f'{model}  vs target... RMSE: {rmse}', '\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "matched-blanket",
   "metadata": {},
   "source": [
    "#### So, the Ridge regression yielded the best results! Let's use those predictions + the custom features to improve the final model a little"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bronze-notification",
   "metadata": {},
   "source": [
    "### Training new model that uses the Ridge predictions + the custom features:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "institutional-elite",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MLPRegressor vs target... R2: 0.542\n",
      "MLPRegressor  vs target... MAE: 0.567\n",
      "MLPRegressor  vs target... RMSE: 0.689 \n",
      "\n",
      "Ridge_a2 vs target... R2: 0.525\n",
      "Ridge_a2  vs target... MAE: 0.582\n",
      "Ridge_a2  vs target... RMSE: 0.701 \n",
      "\n",
      "SVR_C5 vs target... R2: 0.571\n",
      "SVR_C5  vs target... MAE: 0.552\n",
      "SVR_C5  vs target... RMSE: 0.666 \n",
      "\n"
     ]
    }
   ],
   "source": [
    "train['ridge_preds'] = pipeline.predict(X_train)\n",
    "validation['ridge_preds'] = pipeline.predict(X_val)\n",
    "\n",
    "variables = ['rarity', 'avg_word_len', 'fre_test', 'dcrs_test', 'ridge_preds']\n",
    "\n",
    "# Let's iterate over every candidate model, train it and compare results\n",
    "\n",
    "models = {'MLPRegressor': MLPRegressor(max_iter=1000, learning_rate='adaptive', early_stopping=True),\n",
    "          'Ridge_a2': Ridge(alpha=2),\n",
    "          'SVR_C5': SVR(kernel='rbf', C=5),\n",
    "         }\n",
    "\n",
    "for model, regressor in models.items():\n",
    "    # training:\n",
    "    X_train = train[variables].values\n",
    "    \n",
    "    # fitting model\n",
    "    regressor.fit(X_train, y_train)\n",
    "    \n",
    "    # checking the model results in the validation set\n",
    "    X_val = validation[variables].values\n",
    "    X_val_pred = regressor.predict(X_val)\n",
    "    \n",
    "    # metrics\n",
    "    r2 = round(regressor.score(X_val, y_val), 3)\n",
    "    mae = round(mean_absolute_error(y_val, X_val_pred), 3)\n",
    "    rmse = round(math.sqrt(mean_squared_error(y_val, X_val_pred)), 3)\n",
    "    \n",
    "    # Printing results\n",
    "    print(f'{model} vs target... R2: {r2}')\n",
    "    print(f'{model}  vs target... MAE: {mae}')\n",
    "    print(f'{model}  vs target... RMSE: {rmse}', '\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "about-trinity",
   "metadata": {},
   "source": [
    "#### As you can see, the SVR have an RMSE of 0.666 the best result so far! A little better than using the Ridge alone.\n",
    "### So this is our final model! let's see how it performs on the test set"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "banned-aruba",
   "metadata": {},
   "source": [
    "## Comparing results with test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "existing-collaboration",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(141, 3)\n"
     ]
    }
   ],
   "source": [
    "test = pd.read_csv('test.csv', usecols=['id', 'excerpt', 'target'])\n",
    "print(test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "grand-detail",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's apply same pre-process to the test data set.\n",
    "add_creative_features(test)\n",
    "add_clasic_test(test)\n",
    "clean_stem_and_lemmatize(test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "regulation-norman",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(steps=[('tfidf', TfidfVectorizer()), ('rgr', Ridge())])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Cheking pipeline\n",
    "pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "fuzzy-massage",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Aplaying the TF-IDF + Ridge Regression pipeline to have the first model:\n",
    "X_test = test['excerpt']\n",
    "y_test = test['target']\n",
    "pipeline_preds = pipeline.predict(X_test)\n",
    "\n",
    "# saving results\n",
    "test['ridge_preds'] = pipeline_preds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "ordered-commitment",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVR_C5 vs target... R2: 0.478639\n",
      "SVR_C5  vs target... MAE: 0.613966\n",
      "SVR_C5  vs target... RMSE: 0.774197 \n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Metrics with just the first model:\n",
    "r2 = round(pipeline.score(X_test, y_test), 6)\n",
    "mae = round(mean_absolute_error(y_test, pipeline_preds), 6)\n",
    "rmse = round(math.sqrt(mean_squared_error(y_test, pipeline_preds)), 6)\n",
    "\n",
    "# Printing results\n",
    "print(f'{model} vs target... R2: {r2}')\n",
    "print(f'{model}  vs target... MAE: {mae}')\n",
    "print(f'{model}  vs target... RMSE: {rmse}', '\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "recreational-california",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVR(C=5)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# we can check the regressor variable is still the trained svr model:\n",
    "regressor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "environmental-shelter",
   "metadata": {},
   "outputs": [],
   "source": [
    "# aplying SVR model with extra features ('rarity', 'avg_word_len', 'fre_test', 'dcrs_test')\n",
    "variables = ['rarity', 'avg_word_len', 'fre_test', 'dcrs_test', 'ridge_preds']\n",
    "\n",
    "# preparing input:\n",
    "X_test_final_features = test[variables].values\n",
    "# aplying model:\n",
    "svr_preds = regressor.predict(X_test_final_features)\n",
    "\n",
    "# saving results:\n",
    "test['final_pred'] = svr_preds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "introductory-preference",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ridge_preds</th>\n",
       "      <th>final_pred</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.112013</td>\n",
       "      <td>0.215251</td>\n",
       "      <td>0.610319</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.955616</td>\n",
       "      <td>-0.909717</td>\n",
       "      <td>-1.926422</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.070250</td>\n",
       "      <td>0.000402</td>\n",
       "      <td>-0.013471</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.234369</td>\n",
       "      <td>-0.148283</td>\n",
       "      <td>0.009684</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-0.370253</td>\n",
       "      <td>-0.304843</td>\n",
       "      <td>-0.684945</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   ridge_preds  final_pred    target\n",
       "0  0.112013     0.215251    0.610319\n",
       "1 -0.955616    -0.909717   -1.926422\n",
       "2 -0.070250     0.000402   -0.013471\n",
       "3 -0.234369    -0.148283    0.009684\n",
       "4 -0.370253    -0.304843   -0.684945"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test[['ridge_preds', 'final_pred', 'target']].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "bottom-dancing",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVR_C5 vs target... R2: 0.478906\n",
      "SVR_C5  vs target... MAE: 0.615625\n",
      "SVR_C5  vs target... RMSE: 0.773999 \n",
      "\n"
     ]
    }
   ],
   "source": [
    "# metrics\n",
    "r2 = round(regressor.score(X_test_final_features, y_test), 6)\n",
    "mae = round(mean_absolute_error(y_test, svr_preds), 6)\n",
    "rmse = round(math.sqrt(mean_squared_error(y_test, svr_preds)), 6)\n",
    "\n",
    "# Printing results\n",
    "print(f'{model} vs target... R2: {r2}')\n",
    "print(f'{model}  vs target... MAE: {mae}')\n",
    "print(f'{model}  vs target... RMSE: {rmse}', '\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fewer-activation",
   "metadata": {},
   "source": [
    "### So this are the metrics of my final model against the Test data set!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hearing-degree",
   "metadata": {},
   "source": [
    "# Comparison with the Benchmark:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "amazing-signature",
   "metadata": {},
   "source": [
    "#### Remember that in the exploratory analysis notebook, the benchmark scored as follows:\n",
    "* test set R2: 0.188189\n",
    "* test set mae: 0.767187\n",
    "* test set rmse: 0.966072"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "certain-dispute",
   "metadata": {},
   "source": [
    "#### So this final model improve in every simple metric! The RMSE is 24% better.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "renewable-insertion",
   "metadata": {},
   "source": [
    "## Comparison with the Best Kaggle submition so far:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "controversial-cannon",
   "metadata": {},
   "source": [
    "<img src='kaggle.png'>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "micro-waste",
   "metadata": {},
   "source": [
    "The best performer got an RMSE of 0.440, well above my result but obviosly he is spending more time on the project that I can possible can. \n",
    "\n",
    "Bear in mind:\n",
    "* I trained my data on just a portion of the whole training set.\n",
    "* The test set that kaggle uses is private and different than the one I used, so is not fair to compare RMSE unless we use the same test set.\n",
    "* The person on the top of this list may be overfitting the testing set (he/she have tried a total of 106 times against the same test set)\n",
    "* I used the test set just once to avoid overfiting it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "spread-package",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}