Sentiment analysis refers to the process of machines trying to do something humans find difficult much of the time – interpreting the emotion or feeling behind a body of text. In conversation, most people are very good at gauging sentiment, especially given the availability of cues such as facial expressions and vocal intonation. However, even the most emotionally intelligent person can have difficulties understanding the sentiments behind words when they are transmitted via a chat box on a computer. So how is it possible to train a machine to perform sentiment analysis accurately and reliably?
Sentiment analysis is a Natural Language Processing technique that is generally done through something called deep learning. During the process of deep learning, engineers “teach” machines to recognize emotion in text by pairing examples of natural language with labels such as “negative” or “positive”. Internet forums and social media are popular sources of teaching material. Applications of this technology range, and include customer service, educational, and investing. For example, imagine your chatbot can infer your customer is frustrated and escalate their ticket, or you can make predictions about a stock price based on the way it’s being discussed in online forums.
The accuracy of sentiment analysis is generally measured based on two criteria, polarity and degree. Polarity looks at whether a piece of text is positive or negative. Degree looks at how positive or negative it is. This measure is less clear, as even among people there can be disagreement over the degree of a piece of text. Ratings websites (such as those used to assign ratings to movies) provide one way to measure accuracy. Not only do users write reviews, but they also assign numerical or star scores to accompany them. This gives researchers the opportunity to pair a piece of text with a positive or negative rating.