Movies_review_sentiment_analysis_using_nlpia.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Movies review sentiment analysis using nlpia.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyMInHW9C89foJkISe+jh4Kg",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/bray2020/Data-Science-Portfolio-NLP-Works/blob/main/Movies_review_sentiment_analysis_using_nlpia.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HxuwKGnGtR-e"
      },
      "source": [
        "# Movie reviews sentiment analysis: Based on sentiment/polarity score from nlpia package"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "R_pkKi01-hDu"
      },
      "source": [
        "#!pip install nlpia"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ujytLn45shh6",
        "outputId": "4594f676-91c1-4c18-ad6c-bc372a6b4154"
      },
      "source": [
        "import nlpia\n",
        "from nlpia.data.loaders import get_data\n",
        "movies = get_data('hutto_movies')"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/pugnlp/constants.py:136: FutureWarning: The pandas.datetime class is deprecated and will be removed from pandas in a future version. Import from datetime module instead.\n",
            "  [datetime.datetime, pd.datetime, pd.Timestamp])\n",
            "/usr/local/lib/python3.7/dist-packages/pugnlp/constants.py:158: FutureWarning: The pandas.datetime class is deprecated and will be removed from pandas in a future version. Import from datetime module instead.\n",
            "  MIN_TIMESTAMP = pd.Timestamp(pd.datetime(1677, 9, 22, 0, 12, 44), tz='utc')\n",
            "/usr/local/lib/python3.7/dist-packages/pugnlp/tutil.py:100: FutureWarning: The pandas.np module is deprecated and will be removed from pandas in a future version. Import numpy directly instead\n",
            "  np = pd.np\n",
            "/usr/local/lib/python3.7/dist-packages/pugnlp/util.py:80: FutureWarning: The pandas.np module is deprecated and will be removed from pandas in a future version. Import numpy directly instead\n",
            "  np = pd.np\n",
            "/usr/local/lib/python3.7/dist-packages/nlpia/futil.py:30: FutureWarning: The pandas.np module is deprecated and will be removed from pandas in a future version. Import numpy directly instead\n",
            "  np = pd.np\n",
            "/usr/local/lib/python3.7/dist-packages/nlpia/loaders.py:78: FutureWarning: The pandas.np module is deprecated and will be removed from pandas in a future version. Import numpy directly instead\n",
            "  np = pd.np\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 225
        },
        "id": "tRp4f96ts7Zz",
        "outputId": "2678e5f0-b49a-4a90-a99e-94b38ad9624a"
      },
      "source": [
        "movies[:5]"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sentiment</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>id</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2.266667</td>\n",
              "      <td>The Rock is destined to be the 21st Century's ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3.533333</td>\n",
              "      <td>The gorgeously elaborate continuation of ''The...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>-0.600000</td>\n",
              "      <td>Effective but too tepid biopic</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1.466667</td>\n",
              "      <td>If you sometimes like to go to the movies to h...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>1.733333</td>\n",
              "      <td>Emerges as something rare, an issue movie that...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "    sentiment                                               text\n",
              "id                                                              \n",
              "1    2.266667  The Rock is destined to be the 21st Century's ...\n",
              "2    3.533333  The gorgeously elaborate continuation of ''The...\n",
              "3   -0.600000                     Effective but too tepid biopic\n",
              "4    1.466667  If you sometimes like to go to the movies to h...\n",
              "5    1.733333  Emerges as something rare, an issue movie that..."
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lVk6nSF1u4Kn",
        "outputId": "f0b26ee0-4e2b-4198-9aaa-020af70ecff2"
      },
      "source": [
        "movies.shape"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(10605, 2)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_eit_zUPuSX3"
      },
      "source": [
        "**Applying Tokenization using keras package:**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "p7FuXsQ8s-6E"
      },
      "source": [
        "from keras.preprocessing.text import text_to_word_sequence"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 225
        },
        "id": "f4dSisPpuRQn",
        "outputId": "e7c18ff1-f42d-4bea-c75f-a6b46ef32bc5"
      },
      "source": [
        "#Applying word level tokenization and stroing into a column named, 'word_tokens'\n",
        "word_tokens = []\n",
        "for i in movies['text']:\n",
        "  word_tokens.append(text_to_word_sequence(i))\n",
        "movies['word_tokens'] = word_tokens\n",
        "movies.head()"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sentiment</th>\n",
              "      <th>text</th>\n",
              "      <th>word_tokens</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>id</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2.266667</td>\n",
              "      <td>The Rock is destined to be the 21st Century's ...</td>\n",
              "      <td>[the, rock, is, destined, to, be, the, 21st, c...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3.533333</td>\n",
              "      <td>The gorgeously elaborate continuation of ''The...</td>\n",
              "      <td>[the, gorgeously, elaborate, continuation, of,...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>-0.600000</td>\n",
              "      <td>Effective but too tepid biopic</td>\n",
              "      <td>[effective, but, too, tepid, biopic]</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1.466667</td>\n",
              "      <td>If you sometimes like to go to the movies to h...</td>\n",
              "      <td>[if, you, sometimes, like, to, go, to, the, mo...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>1.733333</td>\n",
              "      <td>Emerges as something rare, an issue movie that...</td>\n",
              "      <td>[emerges, as, something, rare, an, issue, movi...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "    sentiment  ...                                        word_tokens\n",
              "id             ...                                                   \n",
              "1    2.266667  ...  [the, rock, is, destined, to, be, the, 21st, c...\n",
              "2    3.533333  ...  [the, gorgeously, elaborate, continuation, of,...\n",
              "3   -0.600000  ...               [effective, but, too, tepid, biopic]\n",
              "4    1.466667  ...  [if, you, sometimes, like, to, go, to, the, mo...\n",
              "5    1.733333  ...  [emerges, as, something, rare, an, issue, movi...\n",
              "\n",
              "[5 rows x 3 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FuIRxR5dujMX"
      },
      "source": [
        "**Stop Words Removal using nltk and sklearn both:**\n",
        "- Remove the stop words from each record of movies['word_tokens']\n",
        "- First get the stop words from both the libraries - nltk and sklearn\n",
        "- Then remove the stop words according to this combined stop words list"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PJtxQspgug-3",
        "outputId": "629f3073-3328-48af-967c-0a30a4baf74f"
      },
      "source": [
        "#Stop words combining step:\n",
        "\n",
        "#get all the stop words from nltk library:\n",
        "import nltk\n",
        "nltk.download('stopwords')\n",
        "stop = nltk.corpus.stopwords.words('english')\n",
        "\n",
        "#get all the stop words from sklearn library:\n",
        "from sklearn.feature_extraction.text import ENGLISH_STOP_WORDS as sklearn_stop\n",
        "\n",
        "#now combine the stop words\n",
        "combined_stop = list(set(stop).union(sklearn_stop))\n",
        "len(combined_stop)"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Package stopwords is already up-to-date!\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "378"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "t4qZyLPAv3OA",
        "outputId": "d59b6ed9-ef2d-4b27-a59f-a472cd0d6853"
      },
      "source": [
        "#Stop words removal step: After removing stop wird storing the data into a list named, 'clean data'\n",
        "clean_data = []\n",
        "\n",
        "for sen in movies['word_tokens']:\n",
        "  for words in sen:\n",
        "      if words in combined_stop:\n",
        "        sen.remove(words)\n",
        "  clean_data.append(sen)\n",
        "\n",
        "#print only the 1st record after removing the stop words\n",
        "clean_data[0]    "
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['rock',\n",
              " 'destined',\n",
              " 'be',\n",
              " '21st',\n",
              " \"century's\",\n",
              " 'new',\n",
              " \"''conan''\",\n",
              " 'that',\n",
              " \"he's\",\n",
              " 'going',\n",
              " 'make',\n",
              " 'splash',\n",
              " 'greater',\n",
              " 'arnold',\n",
              " 'schwarzenegger',\n",
              " 'jean',\n",
              " 'claud',\n",
              " 'van',\n",
              " 'damme',\n",
              " 'steven',\n",
              " 'segal']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L-sqtz-z3DZh"
      },
      "source": [
        "**Based on the cleaned data performed Setiment analysis using vaderSentiment package:**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sMHndId9w4yo"
      },
      "source": [
        "#!pip install vaderSentiment"
      ],
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 965
        },
        "id": "Xyu4md8I7EpQ",
        "outputId": "ca746e87-5a13-4176-b963-7c34c58845aa"
      },
      "source": [
        "#Creting the column named, 'vader_sentiment' for each review \n",
        "score = []\n",
        "from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer\n",
        "sa = SentimentIntensityAnalyzer()\n",
        "for doc in clean_data:        \n",
        "  for review in doc:\n",
        "    scores = sa.polarity_scores(review)\n",
        "  score.append(scores)\n",
        "movies['vader_sentiment'] = score\n",
        "\n",
        "\n",
        "#Storing only the compound score to a column named, 'compound_scores'\n",
        "compound_scores = []\n",
        "\n",
        "for i in movies.vader_sentiment:\n",
        "  for key,value in i.items():\n",
        "    if key=='compound':\n",
        "      compound_scores.append(value)\n",
        "movies['compound_scores'] = compound_scores\n",
        "\n",
        "\n",
        "#printing the firts 30 reviews with their vader sentiment score details\n",
        "movies[:30]"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sentiment</th>\n",
              "      <th>text</th>\n",
              "      <th>word_tokens</th>\n",
              "      <th>vader_sentiment</th>\n",
              "      <th>compound_scores</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>id</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2.266667</td>\n",
              "      <td>The Rock is destined to be the 21st Century's ...</td>\n",
              "      <td>[rock, destined, be, 21st, century's, new, ''c...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3.533333</td>\n",
              "      <td>The gorgeously elaborate continuation of ''The...</td>\n",
              "      <td>[gorgeously, elaborate, continuation, ''the, l...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>-0.600000</td>\n",
              "      <td>Effective but too tepid biopic</td>\n",
              "      <td>[effective, too, tepid, biopic]</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1.466667</td>\n",
              "      <td>If you sometimes like to go to the movies to h...</td>\n",
              "      <td>[you, like, go, the, movies, have, fun, wasabi...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>1.733333</td>\n",
              "      <td>Emerges as something rare, an issue movie that...</td>\n",
              "      <td>[emerges, something, rare, issue, movie, that'...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 0.0, 'pos': 1.0, 'compound...</td>\n",
              "      <td>0.3612</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>2.533333</td>\n",
              "      <td>The film provides some great insight into the ...</td>\n",
              "      <td>[film, provides, great, insight, neurotic, min...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>2.466667</td>\n",
              "      <td>Offers that rare combination of entertainment ...</td>\n",
              "      <td>[offers, rare, combination, entertainment, edu...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>1.266667</td>\n",
              "      <td>Perhaps no picture ever made has more literall...</td>\n",
              "      <td>[no, picture, made, more, literally, showed, t...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>1.933333</td>\n",
              "      <td>Steers turns in a snappy screenplay that curls...</td>\n",
              "      <td>[steers, turns, a, snappy, screenplay, curls, ...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>1.733333</td>\n",
              "      <td>Take Care of My Cat offers a refreshingly diff...</td>\n",
              "      <td>[care, my, cat, offers, refreshingly, differen...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>2.066667</td>\n",
              "      <td>This is a film well worth seeing, talking and ...</td>\n",
              "      <td>[is, film, worth, seeing, talking, singing, he...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>1.666667</td>\n",
              "      <td>What really surprises about Wisegirls is its l...</td>\n",
              "      <td>[really, surprises, wisegirls, its, low, key, ...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 0.0, 'pos': 1.0, 'compound...</td>\n",
              "      <td>0.4215</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>3.133333</td>\n",
              "      <td>(Wendigo is) why we go to the cinema: to be fe...</td>\n",
              "      <td>[wendigo, why, go, cinema, be, fed, eye, the, ...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>3.666667</td>\n",
              "      <td>One of the greatest family oriented, fantasy a...</td>\n",
              "      <td>[of, greatest, family, oriented, fantasy, adve...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>1.333333</td>\n",
              "      <td>Ultimately, it ponders the reasons we need sto...</td>\n",
              "      <td>[ultimately, ponders, reasons, need, stories, ...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>2.200000</td>\n",
              "      <td>An utterly compelling 'who wrote it' in which ...</td>\n",
              "      <td>[utterly, compelling, 'who, wrote, it', which,...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>Illuminating if overly talky documentary.</td>\n",
              "      <td>[illuminating, overly, talky, documentary]</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>3.400000</td>\n",
              "      <td>A masterpiece four years in the making.</td>\n",
              "      <td>[masterpiece, years, the, making]</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>2.200000</td>\n",
              "      <td>The movie's ripe, enrapturing beauty will temp...</td>\n",
              "      <td>[movie's, ripe, enrapturing, beauty, tempt, wi...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>20</th>\n",
              "      <td>2.533333</td>\n",
              "      <td>Offers a breath of the fresh air of true sophi...</td>\n",
              "      <td>[offers, breath, the, fresh, air, true, sophis...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>21</th>\n",
              "      <td>2.800000</td>\n",
              "      <td>A thoughtful, provocative, insistently humaniz...</td>\n",
              "      <td>[thoughtful, provocative, insistently, humaniz...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>22</th>\n",
              "      <td>2.466667</td>\n",
              "      <td>With a cast that includes some of the top acto...</td>\n",
              "      <td>[a, cast, includes, of, top, actors, working, ...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>23</th>\n",
              "      <td>2.000000</td>\n",
              "      <td>A disturbing and frighteningly evocative assem...</td>\n",
              "      <td>[disturbing, frighteningly, evocative, assembl...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24</th>\n",
              "      <td>1.133333</td>\n",
              "      <td>Not for everyone, but for those with whom it w...</td>\n",
              "      <td>[but, for, those, whom, will, connect, a, nice...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25</th>\n",
              "      <td>1.200000</td>\n",
              "      <td>Scores a few points for doing what it does wit...</td>\n",
              "      <td>[scores, few, points, doing, it, with, dedicat...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>26</th>\n",
              "      <td>1.266667</td>\n",
              "      <td>Occasionally melodramatic, it's also extremely...</td>\n",
              "      <td>[occasionally, melodramatic, also, extremely, ...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 0.0, 'pos': 1.0, 'compound...</td>\n",
              "      <td>0.4767</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27</th>\n",
              "      <td>1.533333</td>\n",
              "      <td>An idealistic love story that brings out the l...</td>\n",
              "      <td>[idealistic, love, story, brings, the, latent,...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28</th>\n",
              "      <td>-0.333333</td>\n",
              "      <td>At about 95 minutes, Treasure Planet maintains...</td>\n",
              "      <td>[about, 95, minutes, treasure, planet, maintai...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>29</th>\n",
              "      <td>0.866667</td>\n",
              "      <td>It helps that Lil Bow Wow ... tones down his p...</td>\n",
              "      <td>[helps, lil, bow, wow, tones, his, pint, sized...</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>30</th>\n",
              "      <td>2.200000</td>\n",
              "      <td>Guaranteed to move anyone who ever shook, ratt...</td>\n",
              "      <td>[guaranteed, move, who, shook, rattled, rolled]</td>\n",
              "      <td>{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...</td>\n",
              "      <td>0.0000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "    sentiment  ... compound_scores\n",
              "id             ...                \n",
              "1    2.266667  ...          0.0000\n",
              "2    3.533333  ...          0.0000\n",
              "3   -0.600000  ...          0.0000\n",
              "4    1.466667  ...          0.0000\n",
              "5    1.733333  ...          0.3612\n",
              "6    2.533333  ...          0.0000\n",
              "7    2.466667  ...          0.0000\n",
              "8    1.266667  ...          0.0000\n",
              "9    1.933333  ...          0.0000\n",
              "10   1.733333  ...          0.0000\n",
              "11   2.066667  ...          0.0000\n",
              "12   1.666667  ...          0.4215\n",
              "13   3.133333  ...          0.0000\n",
              "14   3.666667  ...          0.0000\n",
              "15   1.333333  ...          0.0000\n",
              "16   2.200000  ...          0.0000\n",
              "17   1.000000  ...          0.0000\n",
              "18   3.400000  ...          0.0000\n",
              "19   2.200000  ...          0.0000\n",
              "20   2.533333  ...          0.0000\n",
              "21   2.800000  ...          0.0000\n",
              "22   2.466667  ...          0.0000\n",
              "23   2.000000  ...          0.0000\n",
              "24   1.133333  ...          0.0000\n",
              "25   1.200000  ...          0.0000\n",
              "26   1.266667  ...          0.4767\n",
              "27   1.533333  ...          0.0000\n",
              "28  -0.333333  ...          0.0000\n",
              "29   0.866667  ...          0.0000\n",
              "30   2.200000  ...          0.0000\n",
              "\n",
              "[30 rows x 5 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wESc-cz19_tc"
      },
      "source": [
        "**Observation:**\n",
        "- Out of the above displayed only 50 scores, we can see that 3 reviews are positive (considering the compound factor). \n",
        "- The positive review indeces are as 5, 12 and 26. \n",
        "- The rest reviews have turend out to be neutral as per this vader setiment analysis scoring technique."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MC35iTlr7b-x",
        "outputId": "0f829669-d886-4459-9206-dfd99c1f8cb8"
      },
      "source": [
        "#checking the positive review data:\n",
        "print(movies.iloc[4]['word_tokens'])\n",
        "print(movies.iloc[11]['word_tokens'])\n",
        "print(movies.iloc[25]['word_tokens'])"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['emerges', 'something', 'rare', 'issue', 'movie', \"that's\", 'honest', 'keenly', 'observed', 'it', 'feel', 'like']\n",
            "['really', 'surprises', 'wisegirls', 'its', 'low', 'key', 'quality', 'genuine', 'tenderness']\n",
            "['occasionally', 'melodramatic', 'also', 'extremely', 'effective']\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}