Detecting Anger in Text with Deep Learning

Thomas Jefferson once said “When angry count to ten before you speak. If very angry, count to one hundred.” Unfortunately, most of us did not always apply this piece of advice when we got angry especially when it comes to messaging someone. When we got angry, we quickly sent a message that we we later regretted and had to deal with the consequences. This is especially true in the workplace as well where we can potentially burn bridges with our coworkers, get in trouble with management, and even have that message be escalated to human resources.

I wanted to create a preventative measure which would prevent these events from happening. For my final project at Metis my goal was to use topic modeling to find out what makes people angry, build a model that predicts anger in text, and then create an app that will detect the probability of how angry the text seems to be.

Before jumping into my model I wanted to see what what was making people angry. So I got 3100 posts Reddit from their Angry and Rage subreddit to look into this. Using TF-IDF, we are able to give a score of the term based on the relevancy. I then use Non Negative Matrix factorization (NMF) so that I can find the topics. The 7 topics that I got were work, race, relationship, family, pets, entertainment, and “life”. As you could see, a lot of these topics have to do with dealing with people.

For my model I wanted to create a binary classifier that would predict if the text was angry or not. I had 3 labeled datasets. The Emotion Stimulus and ISEAR were labeled sentences. The WASSA-2017 was tweets labeled. I got more data by getting tweets with the #angry from Twitters API. I also got angry and not angry posts from Reddits API as well. To build my model I went for a sequential deep learning model. I had a layer of word embedding, which I used transfer learning with pre-trained word embeddings from GloVe. I then used a Bidirectional LSTM.

The metric that I used was the F1 score. I used the F1 score to take into consideration the class imbalance of my model. The F1 score was a 65.47%.

As you could see with the confusion matrix on the left, it was able to correctly label 620 angry texts and 2487 not angry texts. However, it did have 357 false positives and 297 false negatives.

For my final goal I wanted to create a Streamlit app where people could put in text and get a probability score from 0 to 1. I wanted to test it out myself and put in some angry statements.

I also wanted to test it out with some passive aggressive statements.

As you could see, my model did not do so well when it came to detecting passive aggressive statements. Overall, my model did do well in detecting the more “aggressive” statements. For future work I would like to tune my model better for detecting passive aggressive statements, especially in the workplace.

Related posts:

Elon Musk has built up several billion-dollar companies from the ground up. While most people associate him with Tesla and SpaceX, he also founded X.com, which would eventually become PayPal. Some…

lies tuned to impress accelerating so fast that we don’t even know they’re lies until they spin the globe seven times hitting us in the back of the head standing inside a tattoo shop we never…

A few days ago a friend, who runs a business, told me how he is about to take his first vacation in five years. I told him that it’s nice for him and how stupid it is he postponed it for more than…