Skip to content

A few minor things I found while running the code #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

A few minor things I found while running the code #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 10 additions & 6 deletions nlp/VADERdemo.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Reproduces and extends the demo code packaged with VaderSentmiment
from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
from nltk import tokenize
import json
import random

# Spins up your sentiment analyzer
Expand All @@ -22,7 +21,7 @@ def exercise1a():
print("\t")

def exercise1b():
# - These are some positive examples
# - These are some negative examples
conceptList = ["riot", "fire", "fight", "blood", "mob", "war", "police", "tear gas"]
conceptSentiments = 0.0
for concept in conceptList:
Expand Down Expand Up @@ -143,7 +142,9 @@ def exercise2c():
def exercise3a():
# - VADER works best when analysis is done at the sentence level
# - (but it can work on single words or entire novels).
paragraph = "It was one of the worst movies I've seen, despite good reviews. Unbelievably bad acting!! Poor direction. VERY poor production. The movie was bad. Very bad movie. VERY BAD movie!"
paragraph = "It was one of the worst movies I've seen, despite good reviews. \
Unbelievably bad acting!! Poor direction. VERY poor production. \
The movie was bad. Very bad movie. VERY BAD movie!"
# -- For example, given the above paragraph text from a hypothetical movie review
# -- You could use NLTK to break the paragraph into sentence
# -- tokens for VADER, then average the results for the paragraph
Expand All @@ -161,7 +162,9 @@ def exercise3a():
def exercise3b():
# You may have noticed that the aggregation in VADER isn't order-aware.
# Do you think that's a problem?
paragraph = "It was one of the worst movies I've seen, despite good reviews. Unbelievably bad acting!! Poor direction. VERY poor production. The movie was bad. Very bad movie. VERY BAD movie!"
paragraph = "It was one of the worst movies I've seen, despite good reviews. \
Unbelievably bad acting!! Poor direction. VERY poor production. \
The movie was bad. Very bad movie. VERY BAD movie!"
sentence_list = tokenize.sent_tokenize(paragraph)
for iteration in range(10):
paragraphSentiments = 0.0
Expand All @@ -183,10 +186,11 @@ def scoreSequence(valences):

def exercise3c():
# Consider Implementing an aggregation technique which is sentence order specific
paragraph = "It was one of the worst movies I've seen, despite good reviews. Unbelievably bad acting!! Poor direction. VERY poor production. The movie was bad. Very bad movie. VERY BAD movie!"
paragraph = "It was one of the worst movies I've seen, despite good reviews. \
Unbelievably bad acting!! Poor direction. VERY poor production. \
The movie was bad. Very bad movie. VERY BAD movie!"
sentence_list = tokenize.sent_tokenize(paragraph)
for iteration in range(10):
paragraphSentiments = 0.0
random.shuffle(sentence_list)
scores = map(analyzer.polarity_scores, sentence_list)
vs = scoreSequence(scores)
Expand Down
3 changes: 1 addition & 2 deletions nlp/imdbSentiment.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,6 @@ def computeAccuracy(resultsDict):
# Using VADER sentiment analysis and simple sentiment scores
def bareVADER():
testPaths = getInstanceList(testDataDir)
trainPaths = getInstanceList(trainDataDir)
predictions = analyzeInstances(testPaths)
accuracy = computeAccuracy(predictions)
print(accuracy)
Expand Down Expand Up @@ -212,7 +211,7 @@ def sklearnUsingVADER():

if __name__ == "__main__":
# TODO: Split up into multiple assignment branches
# 1. Just the vader stuf
# 1. Just the vader stuff
# 2. The Vader Stuff filled in and the ML branch
bareVADER()
sklearnUsingVADER()