05. Logistic Regression - Python.ipynb

{
  "cells": [
    {
      "metadata": {
        "collapsed": true
      },
      "cell_type": "markdown",
      "source": "Exercise 5 - Logistic Regression\n=====\n\nLogistic regression predicts binary (yes/no) events. For example, we may want to predict if someone will arrive at work on time, or if a person shopping will buy a product. \n\nThis exercise will demonstrate simple logistic regression: predicting an outcome from only one feature.\n\nStep 1\n-----\n\nWe want to place a bet on the outcome of the next football (soccer) match. It is the final of a competition, so there will not be a draw. We have historical data about our favourite team playing in matches such as this. Complete the exercise below to preview our data.\n\n### In the cell below replace:\n#### 1. `<addFilePath>` with `'Data/football data.txt' ` (including the quotation marks)\n#### 2. `<printDataHere>` with `print(dataset.head())`\n\n#### and then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# This part sets up the graphing configuration\nimport warnings\nwarnings.filterwarnings(\"ignore\")\nimport matplotlib.pyplot as graph\n%matplotlib inline\ngraph.rcParams['figure.figsize'] = (15,5)\ngraph.rcParams[\"font.family\"] = 'DejaVu Sans'\ngraph.rcParams[\"font.size\"] = '12'\ngraph.rcParams['image.cmap'] = 'rainbow'\nimport pandas as pd\n\n\n###\n# REPLACE <addFilePath> BELOW WITH 'Data/football data.txt' (INCLUDING THE QUOTES) TO LOAD THE DATA FROM THAT FILE\n###\ndataset = pd.read_csv(<addFilePath>, index_col = False, sep = '\\t', header = 0)\n###\n\n###\n# REPLACE <printDataHere> BELOW WITH print(dataset.head()) TO PREVIEW OUR DATASET\n###\n<printDataHere>\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "This data shows the average goals per match of our team for that season in the left column. In the right column it lists a 1 if our team won the competition or a 0 if they did not.\n\nStep 2\n----\n\nLet's graph the data so we have a better idea of what's going on here. Complete the exercise below to make an x-y scatter plot.\n\n### In the cell below replace:\n#### 1. `<addWonCompetition>` with `'won_competition'`\n#### 2. `<addAverageGoals>` with `'average_goals_per_match'`\n#### then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE <addWonCompetition> BELOW WITH 'won_competition' (INCLUDING THE QUOTES)\n###\ntrain_Y = dataset[<addWonCompetition>]\n###\n\n###\n# REPLACE <addAverageGoals> BELOW WITH 'average_goals_per_match' (INCLUDING THE QUOTES)\n###\ntrain_X = dataset[<addAverageGoals>]\n###\n\n# The 'won_competition' will be displayed on the vertical axis (y axis)\n# The 'average_goals_per_match' will be displayed on the horizontal axis (x axis)\n\ngraph.scatter(train_X, train_Y, c = train_Y, marker = 'D')\n\ngraph.yticks([0, 1], ['No', 'Yes'])\ngraph.ylabel(\"Competition Win\")\ngraph.ylim([-0.5, 1.5])\ngraph.xlabel(\"Average number of goals scored per match\")\n\ngraph.show()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "We can see from this graph that generally, when our team has a good score average, they tend to win the competition.\n\nStep 3\n----\n\nHow can we predict whether the team will win this season? Let's apply AI to this problem, by making a logisitic regression model using this data and then graph it. This will tell us whether we will likely win this season.\n\n#### Below replace `<buildLinearRegression>` with `linear_model.LogisticRegression()` and then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "import numpy as np\nfrom sklearn import linear_model\n\n# Here we build a logistic regression model\n\n###\n# REPLACE <buildLinearRegression> BELOW WITH linear_model.LogisticRegression() TO BUILD A LOGISTIC REGRESSION MODEL\n###\nclf = <buildLinearRegression>\n###\n\n# This step fits (calculates) the model\n# We are using our feature (x - number of goals scored) and our outcome/label (y - won/lost)\nclf.fit(train_X[:, np.newaxis], train_Y)\n\n# This works out the loss\ndef sigmoid(train_X):\n    return 1 / (1 + np.exp(-train_X))\nX_test = np.linspace(0, 3, 300)\nloss = sigmoid(X_test * clf.coef_ + clf.intercept_).ravel()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Alright, that's the model done. Now __run the code__ below to graph it."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# This makes the graph\n# The data points\ngraph.scatter(train_X, train_Y, c = train_Y, marker = 'D')\n# The curve\ngraph.plot(X_test, loss, color = 'gold', linewidth = 3)\n# Define the y-axis\ngraph.yticks([0, 1], ['No = 0.0', 'Yes = 1.0'])\ngraph.ylabel(\"Competition Win Likelihood\")\ngraph.xlabel(\"Average number of goals per match\")\ngraph.show()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "We now have a line fit to our data. This yellow line is our logistic regression model.\n\nStep 4\n------\n\nWe can read the model above like so:\n* Take the average number of goals per match for the current year. Let's say it is 2.5.\n* Find 2.5 on the x-axis. \n* What value (on the y axis) does the line have at x=2.5?\n* If this value is above 0.5, then the model thinks our team will win this year. If it is less than 0.5, it thinks our team will lose.\n\nBecause this line is just a mathematical function (equation) we don't have to do this visually.\n\nIn the exercise below, __choose the number of goals you want to evaluate__.\n\nThe code will calculate the probability that our team will win with your chosen number of goals in the match.\n\n### In the cell below replace:\n#### 1. `<numberOfGoals>` with the number of goals in a year (any number from 0 to 3)\n#### 2. `<replaceWithP>` with `p`\n#### then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE <numberOfGoals> BELOW WITH THE NUMBER OF GOALS IN A MATCH THIS YEAR. USE ANY NUMBER FROM 0 TO 3\n###\np = <numberOfGoals>\n###\n\n# Next we're going to use our model again - clf is the name of our model.\n# We'll use a method to predict the probability of a positive result\n# Use the variable p which we just made in this method.\n\n###\n# REPLACE <replaceWithP> BELOW WITH p TO PREDICT USING THIS VALUE\n###\nprobOfWinning = clf.predict_proba([[ <replaceWithP> ]])[0][1]\n###\n\n# This prints out the result\nprint(\"Probability of winning this year\")\nprint(str(probOfWinning * 100) + \"%\")\n\n# This plots the result\ngraph.scatter(train_X, train_Y, c = train_Y, marker = 'D')\ngraph.yticks([0, probOfWinning, 1], ['No = 0.0', round(probOfWinning,3), 'Yes = 1.0'])\ngraph.plot(X_test, loss, color = 'gold', linewidth = 3)\n\ngraph.plot(p, probOfWinning, 'ko') # result point\ngraph.plot(np.linspace(0, p, 2), np.full([2],probOfWinning), dashes = [6, 3], color = 'black') # dashed lines (to y-axis)\ngraph.plot(np.full([2],p), np.linspace(0, probOfWinning, 2), dashes = [6, 3], color = 'black') # dashed lines (to x-axis)\n\ngraph.ylabel(\"Competition Win Likelihood\")\ngraph.xlabel(\"Average number of goals per match\")\ngraph.show()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Conclusion\n-----\n\nWell done! We have calculated the likelihood that our team will win this year's competition.\n\nYou can go back to the course now and click __'Next Step'__ "
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Optional: Step 5\n-----\n\nOf course, these predictions are only one model.\n\nLet's return to what we did in step 3, but we'll replace `linear_model.LogisticRegression()` with `linear_model.LogisticRegression(C=200)`. This will tell the model to make a steeper decision boundary. Then repeat Step 4 with this boundary. Did your results change?\n\nThere are methods we can use to choose sensible parameters for many models. This is currently outside the scope of this course, but it is important to remember that a model is only as good as the data we give it, the parameters we choose, and the assumptions we make.\n\n#### Follow the instructions in the cell below to replace `<numberOfGoals>` and `<buildLinearRegression>` and __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Let's do that again.\n# We will repeat what we did in step 3, but change the decision boundary.\n\nimport numpy as np\nfrom sklearn import linear_model\n\n###\n# REPLACE THE <numberOfGoals> WITH THE NUMBER OF GOALS YOU WANT TO EVALUATE\n###\np = <numberOfGoals>\n###\n\n# Here we build the new logistic regression model.\n# The C=200 is where we change the decision boundary.\n###\n# REPLACE <buildLinearRegression> BELOW WITH linear_model.LogisticRegression(C=200) TO BUILD A LOGISTIC REGRESSION MODEL\n###\nclf = <buildLinearRegression>\n###\n\n# This step fits (calculates) the model\n# We are using our feature (x - number of goals scored) and our outcome/label (y - won/lost)\nclf.fit(train_X[:, np.newaxis], train_Y)\n\n# This works out the loss\ndef sigmoid(train_X):\n    return 1 / (1 + np.exp(-train_X))\nX_test = np.linspace(0, 3, 300)\nloss = sigmoid(X_test * clf.coef_ + clf.intercept_).ravel()\n\n# This makes the prediction for your chosen number of goals.\nprobOfWinning = clf.predict_proba([[p]])[0][1]\n\n# This prints out the result.\nprint(\"Probability of winning this year\")\nprint(str(probOfWinning * 100) + \"%\")\n\n# This plots the result.\ngraph.scatter(train_X, train_Y, c = train_Y, marker = 'D')\ngraph.yticks([0, probOfWinning, 1], ['No = 0.0', round(probOfWinning,3), 'Yes = 1.0'])\ngraph.plot(X_test, loss, color = 'gold', linewidth = 3)\n\ngraph.plot(p, probOfWinning, 'ko') # result point\ngraph.plot(np.linspace(0, p, 2), np.full([2],probOfWinning), dashes = [6, 3], color = 'black') # dashed lines (to y-axis)\ngraph.plot(np.full([2],p), np.linspace(0, probOfWinning, 2), dashes = [6, 3], color = 'black') # dashed lines (to x-axis)\n\ngraph.ylabel(\"Competition Win Likelihood\")\ngraph.xlabel(\"Average number of goals per match\")\ngraph.show()",
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "kernelspec": {
      "name": "python36",
      "display_name": "Python 3.6",
      "language": "python"
    },
    "language_info": {
      "mimetype": "text/x-python",
      "nbconvert_exporter": "python",
      "name": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.6",
      "file_extension": ".py",
      "codemirror_mode": {
        "version": 3,
        "name": "ipython"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}