FakeNewsPrediction/fake_news_prediction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <center>Fake News Prediction </center>\n",
    "\n",
    "In this project, we are going to make a machine learning system that predicts whether news are authentic or fake. This is a binary classification problem statement\n",
    "\n",
    "About the Dataset:\n",
    "\n",
    "id: unique id for a news article\n",
    "\n",
    "title: the title of a news article\n",
    "\n",
    "author: author of the news article\n",
    "\n",
    "text: the text of the article; could be incomplete\n",
    "\n",
    "label: a label that marks whether the news article is real or fake:\n",
    "\n",
    "1: Fake news\n",
    "\n",
    "0: Real/Authentic News\n",
    "\n",
    "\n",
    "\n",
    "#### Import the required packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package stopwords to\n",
      "[nltk_data]     C:\\Users\\onale\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Package stopwords is already up-to-date!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import re #regular expression for searching words in a text or paragraph\n",
    "from nltk.corpus import stopwords #words that do not add much value to a paragraph or text\n",
    "from nltk.stem.porter import PorterStemmer #used to stem our words i.e. used to give us a root word for a particular word\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer #used to convert the text into feature vectors\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import accuracy_score\n",
    "import nltk\n",
    "nltk.download('stopwords')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stopwords\n",
    "Stop words are the words in a stop list which are filtered out before or after processing of natural language data because they are insignificant. They do not add any value to our dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['i', 'me', 'my', 'myself', 'we', 'our', 'ours', 'ourselves', 'you', \"you're\", \"you've\", \"you'll\", \"you'd\", 'your', 'yours', 'yourself', 'yourselves', 'he', 'him', 'his', 'himself', 'she', \"she's\", 'her', 'hers', 'herself', 'it', \"it's\", 'its', 'itself', 'they', 'them', 'their', 'theirs', 'themselves', 'what', 'which', 'who', 'whom', 'this', 'that', \"that'll\", 'these', 'those', 'am', 'is', 'are', 'was', 'were', 'be', 'been', 'being', 'have', 'has', 'had', 'having', 'do', 'does', 'did', 'doing', 'a', 'an', 'the', 'and', 'but', 'if', 'or', 'because', 'as', 'until', 'while', 'of', 'at', 'by', 'for', 'with', 'about', 'against', 'between', 'into', 'through', 'during', 'before', 'after', 'above', 'below', 'to', 'from', 'up', 'down', 'in', 'out', 'on', 'off', 'over', 'under', 'again', 'further', 'then', 'once', 'here', 'there', 'when', 'where', 'why', 'how', 'all', 'any', 'both', 'each', 'few', 'more', 'most', 'other', 'some', 'such', 'no', 'nor', 'not', 'only', 'own', 'same', 'so', 'than', 'too', 'very', 's', 't', 'can', 'will', 'just', 'don', \"don't\", 'should', \"should've\", 'now', 'd', 'll', 'm', 'o', 're', 've', 'y', 'ain', 'aren', \"aren't\", 'couldn', \"couldn't\", 'didn', \"didn't\", 'doesn', \"doesn't\", 'hadn', \"hadn't\", 'hasn', \"hasn't\", 'haven', \"haven't\", 'isn', \"isn't\", 'ma', 'mightn', \"mightn't\", 'mustn', \"mustn't\", 'needn', \"needn't\", 'shan', \"shan't\", 'shouldn', \"shouldn't\", 'wasn', \"wasn't\", 'weren', \"weren't\", 'won', \"won't\", 'wouldn', \"wouldn't\"]\n"
     ]
    }
   ],
   "source": [
    "# print stopwords in English\n",
    "print(stopwords.words('english'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Pre-processing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load the dataset to a pandas DataFrame\n",
    "news_dataset = pd.read_csv('datasets/train.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(20800, 5)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "news_dataset.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset has 20800 rows and 5 columns/features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>author</th>\n",
       "      <th>text</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>House Dem Aide: We Didn’t Even See Comey’s Let...</td>\n",
       "      <td>Darrell Lucus</td>\n",
       "      <td>House Dem Aide: We Didn’t Even See Comey’s Let...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>FLYNN: Hillary Clinton, Big Woman on Campus - ...</td>\n",
       "      <td>Daniel J. Flynn</td>\n",
       "      <td>Ever get the feeling your life circles the rou...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>Why the Truth Might Get You Fired</td>\n",
       "      <td>Consortiumnews.com</td>\n",
       "      <td>Why the Truth Might Get You Fired October 29, ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>15 Civilians Killed In Single US Airstrike Hav...</td>\n",
       "      <td>Jessica Purkiss</td>\n",
       "      <td>Videos 15 Civilians Killed In Single US Airstr...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>Iranian woman jailed for fictional unpublished...</td>\n",
       "      <td>Howard Portnoy</td>\n",
       "      <td>Print \\nAn Iranian woman has been sentenced to...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id                                              title              author  \\\n",
       "0   0  House Dem Aide: We Didn’t Even See Comey’s Let...       Darrell Lucus   \n",
       "1   1  FLYNN: Hillary Clinton, Big Woman on Campus - ...     Daniel J. Flynn   \n",
       "2   2                  Why the Truth Might Get You Fired  Consortiumnews.com   \n",
       "3   3  15 Civilians Killed In Single US Airstrike Hav...     Jessica Purkiss   \n",
       "4   4  Iranian woman jailed for fictional unpublished...      Howard Portnoy   \n",
       "\n",
       "                                                text  label  \n",
       "0  House Dem Aide: We Didn’t Even See Comey’s Let...      1  \n",
       "1  Ever get the feeling your life circles the rou...      0  \n",
       "2  Why the Truth Might Get You Fired October 29, ...      1  \n",
       "3  Videos 15 Civilians Killed In Single US Airstr...      1  \n",
       "4  Print \\nAn Iranian woman has been sentenced to...      1  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# print the first 5 rows of the dataframe\n",
    "news_dataset.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "id           0\n",
       "title      558\n",
       "author    1957\n",
       "text        39\n",
       "label        0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# count the number of missing values in the dataset\n",
    "news_dataset.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the above, it is evident that the tittle feature has 558 missing values, author has 1957 and text has 39 missing values. The id and label columns have no missing values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# replace the null values with empty string\n",
    "news_dataset = news_dataset.fillna('')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because we have a lot of data for this project, we can afford to lose some of it. That is why we replaced missing values with null strings. For cases where the dataset is small we have to conduct different data imputation manoeuvres to handle the missing values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# merge the author name and news title features or columns into a new column\n",
    "news_dataset['content'] = news_dataset['author']+' '+news_dataset['title']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "we combined the author and text features as they are the ones we are going to use for making predictions. The text field has paragraphs which are too large and therefore will take a lot of computational power to preprocess it but it can still be used as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0        Darrell Lucus House Dem Aide: We Didn’t Even S...\n",
      "1        Daniel J. Flynn FLYNN: Hillary Clinton, Big Wo...\n",
      "2        Consortiumnews.com Why the Truth Might Get You...\n",
      "3        Jessica Purkiss 15 Civilians Killed In Single ...\n",
      "4        Howard Portnoy Iranian woman jailed for fictio...\n",
      "                               ...                        \n",
      "20795    Jerome Hudson Rapper T.I.: Trump a ’Poster Chi...\n",
      "20796    Benjamin Hoffman N.F.L. Playoffs: Schedule, Ma...\n",
      "20797    Michael J. de la Merced and Rachel Abrams Macy...\n",
      "20798    Alex Ansary NATO, Russia To Hold Parallel Exer...\n",
      "20799              David Swanson What Keeps the F-35 Alive\n",
      "Name: content, Length: 20800, dtype: object\n"
     ]
    }
   ],
   "source": [
    "print(news_dataset['content'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# separate the data & label\n",
    "X = news_dataset.drop(columns='label', axis=1)\n",
    "Y = news_dataset['label']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "          id                                              title  \\\n",
      "0          0  House Dem Aide: We Didn’t Even See Comey’s Let...   \n",
      "1          1  FLYNN: Hillary Clinton, Big Woman on Campus - ...   \n",
      "2          2                  Why the Truth Might Get You Fired   \n",
      "3          3  15 Civilians Killed In Single US Airstrike Hav...   \n",
      "4          4  Iranian woman jailed for fictional unpublished...   \n",
      "...      ...                                                ...   \n",
      "20795  20795  Rapper T.I.: Trump a ’Poster Child For White S...   \n",
      "20796  20796  N.F.L. Playoffs: Schedule, Matchups and Odds -...   \n",
      "20797  20797  Macy’s Is Said to Receive Takeover Approach by...   \n",
      "20798  20798  NATO, Russia To Hold Parallel Exercises In Bal...   \n",
      "20799  20799                          What Keeps the F-35 Alive   \n",
      "\n",
      "                                          author  \\\n",
      "0                                  Darrell Lucus   \n",
      "1                                Daniel J. Flynn   \n",
      "2                             Consortiumnews.com   \n",
      "3                                Jessica Purkiss   \n",
      "4                                 Howard Portnoy   \n",
      "...                                          ...   \n",
      "20795                              Jerome Hudson   \n",
      "20796                           Benjamin Hoffman   \n",
      "20797  Michael J. de la Merced and Rachel Abrams   \n",
      "20798                                Alex Ansary   \n",
      "20799                              David Swanson   \n",
      "\n",
      "                                                    text  \\\n",
      "0      House Dem Aide: We Didn’t Even See Comey’s Let...   \n",
      "1      Ever get the feeling your life circles the rou...   \n",
      "2      Why the Truth Might Get You Fired October 29, ...   \n",
      "3      Videos 15 Civilians Killed In Single US Airstr...   \n",
      "4      Print \\nAn Iranian woman has been sentenced to...   \n",
      "...                                                  ...   \n",
      "20795  Rapper T. I. unloaded on black celebrities who...   \n",
      "20796  When the Green Bay Packers lost to the Washing...   \n",
      "20797  The Macy’s of today grew from the union of sev...   \n",
      "20798  NATO, Russia To Hold Parallel Exercises In Bal...   \n",
      "20799    David Swanson is an author, activist, journa...   \n",
      "\n",
      "                                                 content  \n",
      "0      Darrell Lucus House Dem Aide: We Didn’t Even S...  \n",
      "1      Daniel J. Flynn FLYNN: Hillary Clinton, Big Wo...  \n",
      "2      Consortiumnews.com Why the Truth Might Get You...  \n",
      "3      Jessica Purkiss 15 Civilians Killed In Single ...  \n",
      "4      Howard Portnoy Iranian woman jailed for fictio...  \n",
      "...                                                  ...  \n",
      "20795  Jerome Hudson Rapper T.I.: Trump a ’Poster Chi...  \n",
      "20796  Benjamin Hoffman N.F.L. Playoffs: Schedule, Ma...  \n",
      "20797  Michael J. de la Merced and Rachel Abrams Macy...  \n",
      "20798  Alex Ansary NATO, Russia To Hold Parallel Exer...  \n",
      "20799            David Swanson What Keeps the F-35 Alive  \n",
      "\n",
      "[20800 rows x 5 columns]\n",
      "0        1\n",
      "1        0\n",
      "2        1\n",
      "3        1\n",
      "4        1\n",
      "        ..\n",
      "20795    0\n",
      "20796    0\n",
      "20797    0\n",
      "20798    1\n",
      "20799    1\n",
      "Name: label, Length: 20800, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "print(X)\n",
    "print(Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stemming:\n",
    "\n",
    "Stemming is the process of reducing a word to its Root word\n",
    "\n",
    "example: actor, actress, acting --> act"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "port_stem = PorterStemmer()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "#create a custom function called stemming\n",
    "def stemming(content):\n",
    "    stemmed_content = re.sub('[^a-zA-Z]',' ' ,content) #This replaces all characters other alphabets with empty white space. Basically,extract text which only has alphabets both small and capital leaving out numbers, punctuation and so forth. It just get words\n",
    "    stemmed_content = stemmed_content.lower() #converts the text to lower case, uppercase letters can have a significant impact in the model\n",
    "    stemmed_content = stemmed_content.split() #convert and split the content to a list\n",
    "    stemmed_content = [port_stem.stem(word) for word in stemmed_content if not word in stopwords.words('english')] #first part, we take each word and apply stemming to it removing/skipping the stopwords\n",
    "    stemmed_content = ' '.join(stemmed_content) #join all the words\n",
    "    return stemmed_content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "#use the stemming function on the content column\n",
    "news_dataset['content'] = news_dataset['content'].apply(stemming)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0        darrel lucu hous dem aid even see comey letter...\n",
      "1        daniel j flynn flynn hillari clinton big woman...\n",
      "2                   consortiumnew com truth might get fire\n",
      "3        jessica purkiss civilian kill singl us airstri...\n",
      "4        howard portnoy iranian woman jail fiction unpu...\n",
      "                               ...                        \n",
      "20795    jerom hudson rapper trump poster child white s...\n",
      "20796    benjamin hoffman n f l playoff schedul matchup...\n",
      "20797    michael j de la merc rachel abram maci said re...\n",
      "20798    alex ansari nato russia hold parallel exercis ...\n",
      "20799                            david swanson keep f aliv\n",
      "Name: content, Length: 20800, dtype: object\n"
     ]
    }
   ],
   "source": [
    "print(news_dataset['content'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "#separate the data and label\n",
    "X = news_dataset['content'].values\n",
    "Y = news_dataset['label'].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['darrel lucu hous dem aid even see comey letter jason chaffetz tweet'\n",
      " 'daniel j flynn flynn hillari clinton big woman campu breitbart'\n",
      " 'consortiumnew com truth might get fire' ...\n",
      " 'michael j de la merc rachel abram maci said receiv takeov approach hudson bay new york time'\n",
      " 'alex ansari nato russia hold parallel exercis balkan'\n",
      " 'david swanson keep f aliv']\n"
     ]
    }
   ],
   "source": [
    "print(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 0 1 ... 0 1 1]\n"
     ]
    }
   ],
   "source": [
    "print(Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(20800,)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TfidfVectorizer()"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert the textual data to numerical data\n",
    "vectorizer = TfidfVectorizer() \n",
    "vectorizer.fit(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Term frequency (tf) counts the number of times a particular word is repeating in a document/text/paragraph. This repeatition tells the model that it is a very important word. It will then assign a numerical value to that word.\n",
    "\n",
    "Inverse Document Frequency(IDF) finds values which are repeating so many times but detects if they are significant and if they are not, it will reduce its importance value.\n",
    "\n",
    "Then together, tfidf creates feature vectors. We only do this to X, which is texttual data. Y is already in numerical form."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = vectorizer.transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  (0, 15686)\t0.28485063562728646\n",
      "  (0, 13473)\t0.2565896679337957\n",
      "  (0, 8909)\t0.3635963806326075\n",
      "  (0, 8630)\t0.29212514087043684\n",
      "  (0, 7692)\t0.24785219520671603\n",
      "  (0, 7005)\t0.21874169089359144\n",
      "  (0, 4973)\t0.233316966909351\n",
      "  (0, 3792)\t0.2705332480845492\n",
      "  (0, 3600)\t0.3598939188262559\n",
      "  (0, 2959)\t0.2468450128533713\n",
      "  (0, 2483)\t0.3676519686797209\n",
      "  (0, 267)\t0.27010124977708766\n",
      "  (1, 16799)\t0.30071745655510157\n",
      "  (1, 6816)\t0.1904660198296849\n",
      "  (1, 5503)\t0.7143299355715573\n",
      "  (1, 3568)\t0.26373768806048464\n",
      "  (1, 2813)\t0.19094574062359204\n",
      "  (1, 2223)\t0.3827320386859759\n",
      "  (1, 1894)\t0.15521974226349364\n",
      "  (1, 1497)\t0.2939891562094648\n",
      "  (2, 15611)\t0.41544962664721613\n",
      "  (2, 9620)\t0.49351492943649944\n",
      "  (2, 5968)\t0.3474613386728292\n",
      "  (2, 5389)\t0.3866530551182615\n",
      "  (2, 3103)\t0.46097489583229645\n",
      "  :\t:\n",
      "  (20797, 13122)\t0.2482526352197606\n",
      "  (20797, 12344)\t0.27263457663336677\n",
      "  (20797, 12138)\t0.24778257724396507\n",
      "  (20797, 10306)\t0.08038079000566466\n",
      "  (20797, 9588)\t0.174553480255222\n",
      "  (20797, 9518)\t0.2954204003420313\n",
      "  (20797, 8988)\t0.36160868928090795\n",
      "  (20797, 8364)\t0.22322585870464118\n",
      "  (20797, 7042)\t0.21799048897828688\n",
      "  (20797, 3643)\t0.21155500613623743\n",
      "  (20797, 1287)\t0.33538056804139865\n",
      "  (20797, 699)\t0.30685846079762347\n",
      "  (20797, 43)\t0.29710241860700626\n",
      "  (20798, 13046)\t0.22363267488270608\n",
      "  (20798, 11052)\t0.4460515589182236\n",
      "  (20798, 10177)\t0.3192496370187028\n",
      "  (20798, 6889)\t0.32496285694299426\n",
      "  (20798, 5032)\t0.4083701450239529\n",
      "  (20798, 1125)\t0.4460515589182236\n",
      "  (20798, 588)\t0.3112141524638974\n",
      "  (20798, 350)\t0.28446937819072576\n",
      "  (20799, 14852)\t0.5677577267055112\n",
      "  (20799, 8036)\t0.45983893273780013\n",
      "  (20799, 3623)\t0.37927626273066584\n",
      "  (20799, 377)\t0.5677577267055112\n"
     ]
    }
   ],
   "source": [
    "print(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Split the dataset to training & test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.2, stratify=Y, random_state=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train the Model: Logistic Regression\n",
    "\n",
    "For binary  classification problem statements, logistic regresion performs really well. That is why we used it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = LogisticRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression()"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(X_train, Y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# accuracy score on the training data\n",
    "X_train_prediction = model.predict(X_train)\n",
    "training_data_accuracy = accuracy_score(X_train_prediction, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy score of the training data :  0.9865985576923076\n"
     ]
    }
   ],
   "source": [
    "print('Accuracy score of the training data : ', training_data_accuracy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# accuracy score on the test data\n",
    "X_test_prediction = model.predict(X_test)\n",
    "test_data_accuracy = accuracy_score(X_test_prediction, Y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy score of the test data :  0.9790865384615385\n"
     ]
    }
   ],
   "source": [
    "print('Accuracy score of the test data : ', test_data_accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model has low bias and low variance evidenced by its performance as it is really good. In other words, it is a perfectly balanced fit or generalized model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Making a Predictive System"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0]\n",
      "The news is Real\n"
     ]
    }
   ],
   "source": [
    "X_new = X_test[3]\n",
    "\n",
    "prediction = model.predict(X_new)\n",
    "print(prediction)\n",
    "\n",
    "if (prediction[0]==0):\n",
    "  print('The news is Real')\n",
    "else:\n",
    "  print('The news is Fake')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n"
     ]
    }
   ],
   "source": [
    "print(Y_test[3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}