NLP/Testing sentence segmentation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "toc": true
   },
   "source": [
    "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
    "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#Segment-reviews-in-order-to-extract-&quot;opinion-units&quot;\" data-toc-modified-id=\"Segment-reviews-in-order-to-extract-&quot;opinion-units&quot;-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span>Segment reviews in order to extract \"opinion units\"</a></span><ul class=\"toc-item\"><li><span><a href=\"#Load-reviews\" data-toc-modified-id=\"Load-reviews-1.1\"><span class=\"toc-item-num\">1.1&nbsp;&nbsp;</span>Load reviews</a></span></li><li><span><a href=\"#Extract-opinion-unit\" data-toc-modified-id=\"Extract-opinion-unit-1.2\"><span class=\"toc-item-num\">1.2&nbsp;&nbsp;</span>Extract opinion unit</a></span><ul class=\"toc-item\"><li><span><a href=\"#Adding-rules-for-sentence-segmentation\" data-toc-modified-id=\"Adding-rules-for-sentence-segmentation-1.2.1\"><span class=\"toc-item-num\">1.2.1&nbsp;&nbsp;</span>Adding rules for sentence segmentation</a></span></li><li><span><a href=\"#&quot;so&quot;\" data-toc-modified-id=\"&quot;so&quot;-1.2.2\"><span class=\"toc-item-num\">1.2.2&nbsp;&nbsp;</span>\"so\"</a></span></li></ul></li><li><span><a href=\"#&quot;Except&quot;-(few-cases)\" data-toc-modified-id=\"&quot;Except&quot;-(few-cases)-1.3\"><span class=\"toc-item-num\">1.3&nbsp;&nbsp;</span>\"Except\" (few cases)</a></span></li><li><span><a href=\"#Handling-line-breaks-in-case-of-empty-lines\" data-toc-modified-id=\"Handling-line-breaks-in-case-of-empty-lines-1.4\"><span class=\"toc-item-num\">1.4&nbsp;&nbsp;</span>Handling line breaks in case of empty lines</a></span></li></ul></li></ul></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Segment reviews in order to extract \"opinion units\"\n",
    "When users write reviews, they may provide their opinion on different aspects of the app (cf. __[this website](https://monkeylearn.com/blog/aspect-based-sentiment-analysis/)__). They might like one feature from the app, and whish another one was better. Also, in order to assess what topic are of highest interest, we'd like to segment app reviews in ordre to extract \"opinion units\" from users.\n",
    "\n",
    "After reading some app reviews, a first way of extracting opinion units would be to:\n",
    "* Use the default sentence segmentation from spaCy \n",
    "* Add a rule to the pipeline before the parser, to segment when \"but\" or \"althoug\" are used.\n",
    "* Change the rule to not segment "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import spacy\n",
    "nlp = spacy.load('en_core_web_sm')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load reviews"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "path = os.getcwd()\n",
    "filename = 'app_reviews_airvisual-air-quality-forecast_1048912974_by_lang_us.csv'\n",
    "subfolder = '/../data/1_preprocessed_data/'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(path+subfolder+filename, sep = ';')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>review_id</th>\n",
       "      <th>rating</th>\n",
       "      <th>title</th>\n",
       "      <th>review_date</th>\n",
       "      <th>user_name</th>\n",
       "      <th>review</th>\n",
       "      <th>response_id</th>\n",
       "      <th>dev_response</th>\n",
       "      <th>response_date</th>\n",
       "      <th>lang</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3095</td>\n",
       "      <td>6121840341</td>\n",
       "      <td>5</td>\n",
       "      <td>Happy to finally see when and why I can’t brea...</td>\n",
       "      <td>2020-06-25T23:18:57Z</td>\n",
       "      <td>Abbsteroni</td>\n",
       "      <td>Having allergies is annoying but I’m glad to s...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>en</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>982</td>\n",
       "      <td>6114444527</td>\n",
       "      <td>5</td>\n",
       "      <td>Super</td>\n",
       "      <td>2020-06-24T02:12:59Z</td>\n",
       "      <td>WillJosue</td>\n",
       "      <td>Easy to keep track on specific Local areas</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>en</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1965</td>\n",
       "      <td>6114325210</td>\n",
       "      <td>5</td>\n",
       "      <td>Great App</td>\n",
       "      <td>2020-06-24T01:31:38Z</td>\n",
       "      <td>Nejinater</td>\n",
       "      <td>Full of good information!</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>en</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>797</td>\n",
       "      <td>6111838742</td>\n",
       "      <td>5</td>\n",
       "      <td>Great app for filtering the air</td>\n",
       "      <td>2020-06-23T11:01:04Z</td>\n",
       "      <td>Jamieissad</td>\n",
       "      <td>Tells you everything you need to know about th...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>en</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>962</td>\n",
       "      <td>6104666348</td>\n",
       "      <td>5</td>\n",
       "      <td>I look everyday</td>\n",
       "      <td>2020-06-21T14:29:25Z</td>\n",
       "      <td>TorchPitchfork</td>\n",
       "      <td>This is part of my daily planning. I love the ...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>en</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0   review_id  rating  \\\n",
       "0        3095  6121840341       5   \n",
       "1         982  6114444527       5   \n",
       "2        1965  6114325210       5   \n",
       "3         797  6111838742       5   \n",
       "4         962  6104666348       5   \n",
       "\n",
       "                                               title           review_date  \\\n",
       "0  Happy to finally see when and why I can’t brea...  2020-06-25T23:18:57Z   \n",
       "1                                              Super  2020-06-24T02:12:59Z   \n",
       "2                                          Great App  2020-06-24T01:31:38Z   \n",
       "3                    Great app for filtering the air  2020-06-23T11:01:04Z   \n",
       "4                                    I look everyday  2020-06-21T14:29:25Z   \n",
       "\n",
       "        user_name                                             review  \\\n",
       "0      Abbsteroni  Having allergies is annoying but I’m glad to s...   \n",
       "1       WillJosue         Easy to keep track on specific Local areas   \n",
       "2       Nejinater                          Full of good information!   \n",
       "3      Jamieissad  Tells you everything you need to know about th...   \n",
       "4  TorchPitchfork  This is part of my daily planning. I love the ...   \n",
       "\n",
       "   response_id dev_response response_date lang  \n",
       "0          NaN          NaN           NaN   en  \n",
       "1          NaN          NaN           NaN   en  \n",
       "2          NaN          NaN           NaN   en  \n",
       "3          NaN          NaN           NaN   en  \n",
       "4          NaN          NaN           NaN   en  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extract opinion unit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Adding rules for sentence segmentation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "doc = nlp(u'Having allergies is annoying but I’m glad to see this and know when to take extra precautions.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Having allergies is annoying but I’m glad to see this and know when to take extra precautions.\n"
     ]
    }
   ],
   "source": [
    "for sent in doc.sents:\n",
    "    print(sent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['tagger', 'set_custom_boundaries', 'parser', 'ner']"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# add a new rule to the pipeline\n",
    "def set_custom_boundaries(doc):\n",
    "    for token in doc[:-1]:\n",
    "        if token.text == 'but' or token.text == 'although':\n",
    "            doc[token.i].is_sent_start = True\n",
    "    return doc\n",
    "            \n",
    "nlp.add_pipe(set_custom_boundaries, before = 'parser')\n",
    "\n",
    "nlp.pipe_names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Having allergies is annoying\n",
      "but I’m glad to see this and know when to take extra precautions.\n"
     ]
    }
   ],
   "source": [
    "doc2 = nlp(u'Having allergies is annoying but I’m glad to see this and know when to take extra precautions.')\n",
    "for sent in doc2.sents:\n",
    "    print(sent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "What a great app , beautiful design, useful handling , perfect information 👍 5 stars and high recommended.\n",
      "What I miss ?\n",
      "Only 1 thing , a complication for the Siri watchface , it would be so cool when I see t by rough the day some cards with the air data on my Siri watchface 👍 any chance for this feature?\n",
      "Keep working on this app ist a 5 Star app 👍\n"
     ]
    }
   ],
   "source": [
    "doc3 = nlp(u'What a great app , beautiful design, useful handling , perfect information 👍 5 stars and high recommended. What I miss ? Only 1 thing , a complication for the Siri watchface , it would be so cool when I see t by rough the day some cards with the air data on my Siri watchface 👍 any chance for this feature? Keep working on this app ist a 5 Star app 👍')\n",
    "for sent in doc3.sents:\n",
    "    print(sent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Great app, use it a lot - often multiple times a day to check air quality conditions.\n",
      "Really like the predictions provided for the next few days, in order to assist in planning outdoor activities.\n",
      "5 stars!\n"
     ]
    }
   ],
   "source": [
    "doc4 = nlp(u'Great app, use it a lot - often multiple times a day to check air quality conditions. Really like the predictions provided for the next few days, in order to assist in planning outdoor activities. 5 stars!')\n",
    "for sent in doc4.sents:\n",
    "    print(sent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I live in Seoul and I used this app all the time.\n",
      "However, I restarted my phone and this app will no longer download :(.\n",
      "It has to be because I’m in Seoul?  \n",
      "Every other app has redownloaded.\n"
     ]
    }
   ],
   "source": [
    "doc5 = nlp(u'I live in Seoul and I used this app all the time. However, I restarted my phone and this app will no longer download :(. It has to be because I’m in Seoul?  Every other app has redownloaded.')\n",
    "for sent in doc5.sents:\n",
    "    print(sent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Can track air quality of multiple cities easy and they get their information from multiple sources.\n",
      "Also shows helpful tips on what to do when air qualify is bad - like closings windows and when it’s best not to exercise outside.\n"
     ]
    }
   ],
   "source": [
    "doc6 = nlp(u'Can track air quality of multiple cities easy and they get their information from multiple sources. Also shows helpful tips on what to do when air qualify is bad - like closings windows and when it’s best not to exercise outside.')\n",
    "for sent in doc6.sents:\n",
    "    print(sent)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The segmentation seems to provide good results. Let's explore it further with other reviews. In the preprocessing of reviews, conversion of emojis and emoticons should also be performed, as they contain insights into the sentiments of the users."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### \"so\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Great app.  \n",
      "The predictive forecast is great,\n",
      "although not completely accurate.\n",
      "It seems as weather changes ability to forecast air quality\n",
      "does as well\n",
      "so you have to take that with a grain of salt\n"
     ]
    }
   ],
   "source": [
    "# It seems that there's a segmentation with \"so\" followed by subject + verb as well.\n",
    "s = \"Great app.  The predictive forecast is great, although not completely accurate. It seems as weather changes ability to forecast air quality does as well so you have to take that with a grain of salt\"\n",
    "#s = \"Great app.  The predictive forecast is great, although not completely accurate. It seems as weather changes ability to forecast air quality does as well you have to take that with a grain of salt\"\n",
    "\n",
    "doc7 = nlp(s)\n",
    "for sent in doc7.sents:\n",
    "    print(sent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "love using this to check the air quality in my area and whenever I travel.\n",
      "i like that it has the widget option\n",
      "so i can see it in my today view as well on iphone\n"
     ]
    }
   ],
   "source": [
    "s = \"love using this to check the air quality in my area and whenever I travel. i like that it has the widget option so i can see it in my today view as well on iphone\"\n",
    "\n",
    "doc8 = nlp(s)\n",
    "for sent in doc8.sents:\n",
    "    print(sent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "My cousin usually has acute pneumonia so this app has been very useful.\n",
      "We are able get access to the air quality wherever we go, and avoid unnecessary contact with fine dust.\n"
     ]
    }
   ],
   "source": [
    "s = \"My cousin usually has acute pneumonia so this app has been very useful. We are able get access to the air quality wherever we go, and avoid unnecessary contact with fine dust.\"\n",
    "\n",
    "doc9 = nlp(s)\n",
    "for sent in doc9.sents:\n",
    "    print(sent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Never followed this before\n",
      "but now I’m understanding what’s effecting my at times debilitating congestion is not allergies.\n",
      "Really accurate for my neighborhood so far (brooklyn, New York).\n",
      "You can even have the air quality number stay on top of the app the same way notifications usually appear.\n",
      "But seems like you have to open the app for it to be updated/refresh to the latest air quality number.\n",
      "I guess that’s the only thing that’s not the best so far.\n"
     ]
    }
   ],
   "source": [
    "s = \"Never followed this before but now I’m understanding what’s effecting my at times debilitating congestion is not allergies. Really accurate for my neighborhood so far (brooklyn, New York). You can even have the air quality number stay on top of the app the same way notifications usually appear. But seems like you have to open the app for it to be updated/refresh to the latest air quality number. I guess that’s the only thing that’s not the best so far.\"\n",
    "\n",
    "doc10 = nlp(s)\n",
    "for sent in doc10.sents:\n",
    "    print(sent)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\"so\" sometimes leads to sentence segmentation, while in the coupe of reviews taken as examples, there should not be a segmentation. We could remove the rule linked to \"so\". "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['tagger', 'set_custom_boundaries', 'parser', 'ner']"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# reset to the original\n",
    "nlp = spacy.load('en_core_web_sm')\n",
    "\n",
    "# add new rules to the pipeline\n",
    "def set_custom_boundaries(doc):\n",
    "    for token in doc[:-1]:\n",
    "        if token.text == 'but' or token.text == 'although':\n",
    "            doc[token.i].is_sent_start = True\n",
    "        elif token.text == 'so':\n",
    "            doc[token.i].is_sent_start = False\n",
    "    return doc\n",
    "            \n",
    "nlp.add_pipe(set_custom_boundaries, before = 'parser')\n",
    "\n",
    "nlp.pipe_names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "love using this to check the air quality in my area and whenever I travel.\n",
      "i like that it has the widget option so i can see it in my today view as well on iphone\n"
     ]
    }
   ],
   "source": [
    "s = \"love using this to check the air quality in my area and whenever I travel. i like that it has the widget option so i can see it in my today view as well on iphone\"\n",
    "\n",
    "doc8 = nlp(s)\n",
    "for sent in doc8.sents:\n",
    "    print(sent)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## \"Except\" (few cases)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Working and living in South Korea is a joy EXCEPT for the air quality.\n",
      "\n",
      " \n",
      "While I can survive with irritated eyes, my wife is most sensitive to the air quality which is sadly lacking due to Yellow Dust from the Gobi Desert as well as industrial pollution making its way from China.\n",
      "\n",
      " \n",
      "We depend on the app daily to determine the extent of our outdoor exposure as well as turning on our indoor air filters.\n"
     ]
    }
   ],
   "source": [
    "s = \"Working and living in South Korea is a joy EXCEPT for the air quality.\\n\\n While I can survive with irritated eyes, my wife is most sensitive to the air quality which is sadly lacking due to Yellow Dust from the Gobi Desert as well as industrial pollution making its way from China.\\n\\n We depend on the app daily to determine the extent of our outdoor exposure as well as turning on our indoor air filters.\"\n",
    "\n",
    "doc11 = nlp(s)\n",
    "for sent in doc11.sents:\n",
    "    print(sent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This app is very good, only except that it does not cover enough cities, e.g. Gainesville FL.\n",
      "Hope your later version can cover more cities.\n"
     ]
    }
   ],
   "source": [
    "s = u\"This app is very good, only except that it does not cover enough cities, e.g. Gainesville FL. Hope your later version can cover more cities.\"\n",
    "\n",
    "doc12 = nlp(s)\n",
    "for sent in doc12.sents:\n",
    "    print(sent)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A rule might be added to segment with \"except\" if \"except\" is followed by a structure such as \"subject\" + \"verb\"? (But since, there are only a few cases, might do it manually)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Handling line breaks in case of empty lines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Working and living in South Korea is a joy EXCEPT for the air quality.\n",
      "\n",
      " \n",
      "While I can survive with irritated eyes, my wife is most sensitive to the air quality which is sadly lacking due to Yellow Dust from the Gobi Desert as well as industrial pollution making its way from China.\n",
      "\n",
      " \n",
      "We depend on the app daily to determine the extent of our outdoor exposure as well as turning on our indoor air filters.\n"
     ]
    }
   ],
   "source": [
    "s = u\"Working and living in South Korea is a joy EXCEPT for the air quality.\\n\\n While I can survive with irritated eyes, my wife is most sensitive to the air quality which is sadly lacking due to Yellow Dust from the Gobi Desert as well as industrial pollution making its way from China.\\n\\n We depend on the app daily to determine the extent of our outdoor exposure as well as turning on our indoor air filters.\"\n",
    "\n",
    "doc13 = nlp(s)\n",
    "for sent in doc13.sents:\n",
    "    print(sent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Discovered this app a couple of months ago and find it very useful in evaluating air quality, ventilating my house, and considering how to improve air filtration in my house.\n",
      "With continuing fires and air pollution in N California, I have developed bronchial irritation and increased congestion (which closely tracks air quality conditions reported by air visual).\n",
      "I attribute the good info to lots of reporting stations in my area.\n",
      "\n",
      "Of particular interest is knowing when to ventilate my house (during periods of good air reported by air visual).\n",
      "Also, the app is helping me evaluate the performance of my whole house air filter system; my house filter helps\n",
      "but need improvement.\n",
      "\n",
      "As a result of the app's functionality, I'm considering buying a larger version of IQAir's home air purifier as an adjunct to my whole house filtering.\n",
      "\n",
      "I give a 4 star rating because there is always room for improvement.\n"
     ]
    }
   ],
   "source": [
    "doc14 = nlp(df.iloc[2147]['review'])\n",
    "for sent in doc14.sents:\n",
    "    print(sent)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the opinion unit extraction will be carried out, we might test for and get rid of empty sentences."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": true,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}