Lab 2 - End-to-end project in ML.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# An end-to-end project in Machine Learning\n",
    "\n",
    "## Using machine learning to predict bike rentals\n",
    "\n",
    "### 3r October 2020 by Mauricio A Álvarez\n",
    "\n",
    "Our dataset comes from a popular machine learning repository that hosts open source datasets for educational and research purposes, the [UCI Machine Learning Repository](https://archive.ics.uci.edu/ml/index.php). We are going to follow several of the steps in the ML project checklist and use several utilities and models in [scikit-learn](https://scikit-learn.org/stable/) for predicting bike rentals. The description of the dataset can be found [here](https://archive.ics.uci.edu/ml/datasets/Seoul+Bike+Sharing+Demand#).\n",
    "\n",
    "### Get the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import urllib.request\n",
    "urllib.request.urlretrieve('https://archive.ics.uci.edu/ml/machine-learning-databases/00560/SeoulBikeData.csv', './SeoulBikeData.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd \n",
    "bike_sharing_data = pd.read_csv('SeoulBikeData.csv', encoding= 'unicode_escape')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can get a description of the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bike_sharing_data.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see some of the rows in the dataset "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bike_sharing_data.sample(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The target variable $y$ corresponds to the Rented Bike Count variable of the second column. The following columns correspond to the variables in the feature vector $\\mathbf{x}$, *e.g.*, *Hour* is $x_1$ up until *Functioning Day* which is $x_D$. The original dataset also has a date column that we are not going to use in this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bike_sharing_data = bike_sharing_data.drop('Date', axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We follow some of the steps in the ML checklist we used in the lecture, including data exploration, data preprocessing, and fine-tuning the ML model. It is important to remember that the testing data that we use for assessing the generalisation performance has to be set aside once we get the data. Also, any data preprocessing that you do has to be done only on the training data and several quantities need to be saved for the test stage. Separating the dataset into training and test before any preprocessing has happened, help us to recreate the real world scenario where we will deploy our system and for which the data will come without any preprocessing.\n",
    "\n",
    "We use scikit-learn to separate the data into training and test sets. Let us first look at how many instances we have in the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bike_sharing_data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Several algorithms that we will use assume the inputs to be type 'float' instead of 'int', so we transform those variables in the dataset from int64 to float64."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for col in ['Rented Bike Count', 'Hour', 'Humidity(%)', 'Visibility (10m)']:\n",
    "    bike_sharing_data[col] = bike_sharing_data[col].astype('float64')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bike_sharing_data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset has a few thousand observations. We will use 85% of the data for training and 15% for testing. The `train_test_split` function in scikit-learn allows to easily get these partitions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "bs_train_set, bs_test_set = train_test_split(bike_sharing_data, test_size=0.15, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The train and test sets are chosen randomly from all the available data. By specifying a value for `random_state`, we are making sure that every time we run this instruction, the train and test set will have the exact same instances. `random_state` \"controls the shuffling applied to the data before applying the split\". \n",
    "\n",
    "### Explore the data\n",
    "\n",
    "There are different tools we can use to explore the dataset. \n",
    "\n",
    "#### Histograms\n",
    "\n",
    "Let us first look at histograms for each of the continuous attributes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline \n",
    "import matplotlib.pyplot as plt\n",
    "bs_train_set.hist(bins=50, figsize=(20,15))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some observations from the histograms are:\n",
    "\n",
    "1. The values for the variables Rainfall, Snowfall, Solar Radition and Visibility are concentrated at one of the ends of the plots. This is an indication that several instances might contain outliers. One can consider removing these outliers from the data or binning the data into a few discrete values. \n",
    "\n",
    "2. Both the Rented Bike Count and the Wind Speed are [skewed to the right](https://en.wikipedia.org/wiki/Skewness), this is, the mean of the distribution is to the right of the median. Some ML algorithms find it harder to detect patterns for this type of distribution. One might consider transforming these features using $\\log(x)$ or $\\sqrt{x}$ so that they look more like a bell-shaped distribution. \n",
    "\n",
    "#### Question 1\n",
    "\n",
    "1. Compute the mean and the median for the variables Rented Bike Count and Wind Speed and verify that the mean is to the right of the median.\n",
    "\n",
    "2. How would the histograms for Rented Bike Count and the Wind Speed look like if we transform the values using $\\sqrt{x}$? \n",
    "\n",
    "3. Would it be possible to use $\\log{x}$ instead of $\\sqrt{x}$? If not, what would you do to the variable to be able to use it?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Provide your answer here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Scatter plots\n",
    "\n",
    "The Scatter plot is a tool we can use to explore dependencies between the different variables. It contains plots of each variable against each other in the dataset. If there are many variables in the feature vector, including all scatter plots might not be convenient to visualise. Let us look at the scatter plot for the target variable and four of the attributes. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "from pandas.plotting import scatter_matrix\n",
    "attributes = ['Rented Bike Count', 'Hour', 'Temperature(°C)', 'Humidity(%)', 'Wind speed (m/s)']\n",
    "figscat = scatter_matrix(bs_train_set[attributes], figsize=(20, 15))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The variables Hour and Temperature seem well correlatd with Rented Bike Count. The relationship between Humidity and Wind Speed with Rented Bike Count looks less clear though.\n",
    "\n",
    "### Correlation coefficients\n",
    "\n",
    "Additionally, we can study the correlation coefficient between the numerical attributes and the Rented Bike Count."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "corr_matrix = bs_train_set.corr()\n",
    "corr_matrix['Rented Bike Count'].sort_values(ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we suspected by having looked at the scatter plots, Temperature and Hour are strongly correlated with the target value. \n",
    "\n",
    "#### Question 2\n",
    "\n",
    "What would be the correlation coefficients if the variables Rented Bike Count and Wind Speed are transformed using $\\sqrt{x}$? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Provide your answer here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prepare the data\n",
    "\n",
    "We will now prepare the data so that it is suitable for the machine learning models. We consider the following processes for the dataset in this notebook: using one-hot-encoding for the categorical attributes and feature scaling for the numerical attributes. scikit-learn provides utilities for these tasks:\n",
    "\n",
    "1. [OneHotEncoder()](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html?highlight=onehotencoder#sklearn.preprocessing.OneHotEncoder) allows to transform a categorical variable to a one-hot encoding representation. \n",
    "\n",
    "2. [StandardScaler()](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html?highlight=standardscaler#sklearn.preprocessing.StandardScaler) performs feature scaling by standardisation.\n",
    "\n",
    "`OneHotEncoder()` and `StandardScaler()` are part of the scikit-learn [preprocessing module](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.preprocessing). \n",
    "\n",
    "#### Question 3\n",
    "\n",
    "Explore the scikit-learn [preprocessing module](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.preprocessing). List and explain two of the utilities availaible that you believe are useful for data analysis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Provide your answer here*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`OneHotEncoder()` and `StandardScaler()` are examples of [data transformations](https://scikit-learn.org/stable/data_transforms.html). In scikit-learn these are referred to as *transformers* and they map the data from one format to another. In a programming context, transformers are classes. They come with the following methods:\n",
    "\n",
    "- `fit` that is used to learn the  transformation from data. \n",
    "- `transform` that is used to transform the data once the transformer has been fitted.   \n",
    "- `fit_transform` that applies first `fit` and then `transform` to the data.\n",
    "\n",
    "Typically, we use either a `fit` or `fit_transform` for the training data and `transform` to the validation or test data.\n",
    "\n",
    "Since the one-hot-encoding and standardisation transformations will be applied to different columns in the dataset, it is convenient to have a function that applies them both. It is particulaly useful since we need to apply those transformations to the train, validation and test set. We could code such function from scratch, but here, we make use of the [ColumnTransformer()](https://scikit-learn.org/stable/modules/generated/sklearn.compose.ColumnTransformer.html?highlight=columntransformer#sklearn.compose.ColumnTransformer), an estimator available in scikit-learn that allows to group different transformations into a single method. `ColumnTransformer` is an example of an *estimator* in scikit-learn. An estimator is an object that provides predictions for new data. Besides the `fit`, `transform` and `fir_transform` methods, they most of the times provide the additional `predict` method for making predictions on the test data. \n",
    "\n",
    "#### Question 4\n",
    "\n",
    "A [pipeline](https://scikit-learn.org/stable/modules/compose.html#pipeline) is a very convenient estimator in scikit-learn. Explain what is a pipeline and describe in which situations it is useful."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Provide your answer here*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us go back to the dataset and apply the transformations we mentioned before. We first define lists with the names of the attributes, a list for the categorical attributes and a list for the numerical attributes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "attributes_cat = ['Seasons', 'Holiday', 'Functioning Day']\n",
    "attributes_num = ['Hour', 'Temperature(°C)', 'Humidity(%)', 'Wind speed (m/s)', 'Visibility (10m)', \\\n",
    "                  'Dew point temperature(°C)', 'Solar Radiation (MJ/m2)', 'Rainfall(mm)', 'Snowfall (cm)']   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now import `OneHotEncoder`, `StandardScaler` and `ColumnTransformer` and create the actual transformer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import OneHotEncoder\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.compose import ColumnTransformer\n",
    "\n",
    "full_transform = ColumnTransformer([\n",
    "    (\"num\", StandardScaler(), attributes_num),\n",
    "    (\"cat\", OneHotEncoder(), attributes_cat),\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before applying the full transformation, we separate the target feature from the attributes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bs_train_set_attributes = bs_train_set.drop('Rented Bike Count', axis=1)\n",
    "bs_train_set_labels = bs_train_set['Rented Bike Count']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now apply the fit and apply the full transformation to the training data using `fit_transform`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bs_train_set_attributes_prepared = full_transform.fit_transform(bs_train_set_attributes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Short-list models and fine-tune them\n",
    "\n",
    "Up until this point, we have managed to prepare the data so that it can be used for fitting a predictive model. Scikit-learn includes routines for [several different predictive models for regression and for classification](https://scikit-learn.org/stable/supervised_learning.html#supervised-learning). The application of these methods follows a similar template to the point that applying a method intead of other is just a matter of changing the name of the method.\n",
    "\n",
    "In this notebook, we will focus on Linear Regression but in a larger ML project, you will be encouraged to try different predictive models, e.g. from those available in scikit-learn, and short-list two to three that look promising.  \n",
    "\n",
    "We import the [LinearRegression()](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html?highlight=linearregression#sklearn.linear_model.LinearRegression) method and fit it to the training data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "lin_reg = LinearRegression()\n",
    "lin_reg.fit(bs_train_set_attributes_prepared, bs_train_set_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And that's it! We have fit the ML model. What's next? Well, by now, one may feel tempted to apply the model to the test data to see how it performs. However, one should only do this when being absolutely sure that this is the best performing model on a *validation set*. \n",
    "\n",
    "We have not used a validation set up until this point because we have not needed to compare between two alternative models. To see how to fine-tune the model, *let us use a validation set to decide whether including the features Rainfall and Snowfall has any benefits*\n",
    "\n",
    "#### Fine-tuning the model\n",
    "\n",
    "We take the original train set and split it again into a train set and a validation set. Due to the size of the dataset, we use a simple way for validating the model known as *holdout validation*, for which we hold out a single set of data for validation purposes. When the dataset is smaller, you can perform k-fold cross-validation or if the data is really small, leave-one-out cross validation. Both are implemented in scikit-learn ([k-fold cross validation](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.KFold.html#sklearn.model_selection.KFold) and [leave-one-out cross validation](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.LeaveOneOut.html?highlight=leave%20one%20out#sklearn.model_selection.LeaveOneOut).) \n",
    "\n",
    "From the original train set, we use 85% for the train set and 15% for the validation set. We could have split the dataset from the beginning into a train set, a validation set and a test set using 70%, 15% and 15% of the available data, respectively. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bs_train2_set, bs_val_set = train_test_split(bs_train_set, test_size=0.15, random_state=42)\n",
    "bs_train2_set_attributes = bs_train2_set.drop('Rented Bike Count', axis=1)\n",
    "bs_train2_set_labels = bs_train2_set['Rented Bike Count']\n",
    "bs_val_set_attributes = bs_val_set.drop('Rented Bike Count', axis=1)\n",
    "bs_val_set_labels = bs_val_set['Rented Bike Count']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will be comparing between two transformations, the one we already described with `full_transform` and one that looks similar except from not including Rainfall and Snowfall."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "attributes_num_partial = ['Hour', 'Temperature(°C)', 'Humidity(%)', 'Wind speed (m/s)', 'Visibility (10m)', \\\n",
    "                  'Dew point temperature(°C)', 'Solar Radiation (MJ/m2)']   \n",
    "partial_transform = ColumnTransformer([\n",
    "    (\"num\", StandardScaler(), attributes_num_partial),\n",
    "    (\"cat\", OneHotEncoder(), attributes_cat),\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now use this new transformation to fit_transform the new train set "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bs_train2_set_no_RS_attributes = partial_transform.fit_transform(bs_train2_set_attributes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now train the linear regression model that only uses the partial transformed attributes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lin_reg_mod = LinearRegression()\n",
    "lin_reg_mod.fit(bs_train2_set_no_RS_attributes, bs_train2_set_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us now assess the performance of this model over the validation data. We first need to prepare the validation input data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bs_val_set_no_RS_attributes = partial_transform.transform(bs_val_set_attributes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now compute the predictions made by the linear model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bs_val_set_predictions_mod = lin_reg_mod.predict(bs_val_set_no_RS_attributes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now compute the RMSE obtained with this predictive model. We can use the [scikit-learn routine for computing the mean squared error](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.mean_squared_error.html#sklearn.metrics.mean_squared_error) and then compute the square root."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import mean_squared_error\n",
    "import numpy as np\n",
    "error_mod = np.sqrt(mean_squared_error(bs_val_set_labels, bs_val_set_predictions_mod))\n",
    "error_mod"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us now look into using all the numerical attributes. The train set has changed, so we need to fit_transform a new full transformer. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bs_train2_set_all_attributes = full_transform.fit_transform(bs_train2_set_attributes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We creat the new linear regression model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lin_reg = LinearRegression()\n",
    "lin_reg.fit(bs_train2_set_all_attributes, bs_train2_set_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Transform the validation data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bs_val_set_all_attributes = full_transform.transform(bs_val_set_attributes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We finally perform the prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bs_val_set_predictions = lin_reg.predict(bs_val_set_all_attributes)\n",
    "error = np.sqrt(mean_squared_error(bs_val_set_labels, bs_val_set_predictions))\n",
    "error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We conclude from this that the variables Rainfall and Snowfall actually help to slightly improve the predictions.\n",
    "\n",
    "### Question 5\n",
    "\n",
    "Perhaps other transformations to the dataset can help to improve the predictions. Try the following transformations and see whether the RMSE over the validation set reduces even more: \n",
    "\n",
    "1. Before standardising the feature Wind speed, first transform it using $\\sqrt{x}$. \n",
    "2. Transform the Rainfall and the Snowfall to discrete features using the scikit-learn utility [KBinsDiscretizer](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.KBinsDiscretizer.html#sklearn.preprocessing.KBinsDiscretizer) with $K=5$.\n",
    "3. Instead of doing standardisation over the other numerical features, use normalisation.\n",
    "4. Keep the one-hot-encoding for the categorical features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Provide your answer bere"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Between chosing to include Rainfall and Snowfall or not, the stage of validation tells us we should include them. If this was the only hyperparameter to choose from, we would be done and we could proceed to compute the generalisation error on the test set. Since we are not considering more fine-tuning at the moment, let us compute the RMSE over the test set. We have already prepared the whole training data (what we called train2+val) before using the full transform, we called it `bs_train_set_attributes_prepared`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lin_reg = LinearRegression()\n",
    "lin_reg.fit(bs_train_set_attributes_prepared, bs_train_set_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us transform the test data so that we can apply the fitted model correctly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bs_test_set_attributes = bs_test_set.drop('Rented Bike Count', axis=1)\n",
    "bs_test_set_labels = bs_test_set['Rented Bike Count']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now transform the attributes in the test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bs_test_set_attributes_prepared = full_transform.transform(bs_test_set_attributes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We perform the prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bs_test_set_predictions = lin_reg.predict(bs_test_set_attributes_prepared)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And compute the RMSE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "error_test = np.sqrt(mean_squared_error(bs_test_set_labels, bs_test_set_predictions))\n",
    "error_test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The performance in the test set is slightly worse when compared to the performance in the validation set. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}