You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Sentiment analysis, while a powerful method to extract insights from data, is far from perfect or straightforward. After all, it seeks to interpret natural language, which is constantly evolving. Speaking of evolution, did you know that the [Cambridge dictionary added 6,000 words](https://www.npr.org/2025/08/19/nx-s1-5506163/cambridge-dictionary-adds-more-than-6-000-words-including-skibidi-and-delulu) only this year, including "broligarchy" and "delulu", many of which are widely used by Gen Alpha? This constant expansion of language highlights just how dynamic the texts we analyze can be.
7
+
8
+
Language is also inherently rich, ambiguous, and culturally nuanced. Lexicon-based approaches, for instance, rely on predefined word lists and often struggle to capture subtleties in human expression.
9
+
10
+
In practice, sentiment analysis encounters issues like *code-switching*, where people mix languages in a single post, or compound sentences with mixed sentiments, such as “The movie had great acting, but the ending was lame,” which are difficult to score accurately. *Context dependence* further complicates interpretation: words can flip polarity depending on the domain, like “cheap,” which is positive when describing flights or monetary advantage in general, but negative when describing fabric or referring to quality.
11
+
12
+
Temporal dynamics also play a role, as *slang and cultural references* evolve rapidly, e.g., “bad” meaning “good” in some communities. Ambiguity adds another layer of difficulty: polysemous words like “sick” can mean either “ill” or “awesome”.
13
+
14
+
Another complication is to deal with *sarcasm* and irony which can completely invert the intended sentiment as in: “Oh great, another awesome Monday morning traffic jam!".
15
+
16
+
*Implicit sentiment* may be present even when emotional words are absent, as in “The waiter ignored us for 30 minutes before taking our order.” These factors collectively make sentiment analysis a useful but inherently imperfect tool for understanding human language and emotion.
17
+
18
+
However, it is important to emphasize that as described before, we have only explored sentiment analysis through a lexicon-based approach, and that, as illustrated in Figure ? below, there are other methods, including machine learning, deep learning and their combination (hybrid), that can be employed to extract emotions from text, including user generated content, all with their own limitations and challenges.
19
+
20
+
{fig-align="center" width="500"}
21
+
22
+
For example, Amazon relies on deep learning algorithms to determine the sentiment of customer reviews by identifying positive, negative, or neutral tones in the text. The models are trained on a vast dataset of Amazon’s product descriptions and reviews and are regularly updated with new information. This robust approach enables Amazon to efficiently analyze and interpret customer feedback on a large scale.
23
+
24
+
While there are more advanced approaches to sentiment analysis, including AI-assisted methods, these are discussion topics for future workshops!
Mao, Y., Liu, Q., & Zhang, Y. (2024). Sentiment analysis methods, applications, and challenges: A systematic literature review. *Journal of King Saud University - Computer and Information Sciences, 36*(4), 102048. <https://doi.org/10.1016/j.jksuci.2024.102048>
Copy file name to clipboardExpand all lines: chapters/3.SentimentAnalysis/emotion.qmd
+168-3Lines changed: 168 additions & 3 deletions
Original file line number
Diff line number
Diff line change
@@ -11,7 +11,7 @@ The `syuzhet` package implements the [National Research Council Canada (NRC) Emo
11
11
12
12
This framework uses eight categories of emotions based on Robert Plutchik's theory of the emotional wheel, a foundational model that illustrates the relationships between human emotions from a psychological perspective. Plutchik’s wheel identifies eight primary emotions: anger, disgust, sadness, surprise, fear, trust, joy, and anticipation. As illustrated in Figure ? below, these emotions are organized into four pairs of opposites on the wheel. Emotions positioned diagonally across from each other represent opposites, while adjacent emotions share similarities, reflecting a positive correlation.
13
13
14
-
{fig-align="center" width="376"}
14
+
{fig-align="center" width="376"}
15
15
16
16
The NRC Emotion Lexicon was developed as part of research into affective computing and sentiment analysis using a combination of manual annotation and crowdsourcing. Human annotators evaluated thousands of words, indicating which emotions were commonly associated with each word. This method ensured that the lexicon captured human-perceived emotional associations, rather than relying solely on statistical co-occurrences in text.
17
17
@@ -22,9 +22,174 @@ You may explore NRC's lexicon Tableau dashboard to explore words associated with
Now that we have a better understanding of this package, let's get back to business and perform emotion detection to our data:
25
+
26
+
Now that we have a better understanding of this package, let's get back to business and perform emotion detection to our data.
27
+
28
+
#### Emotion Detection with Syuzhet's NRC Lexicon
29
+
30
+
##### Detecting Emotions per Comment/Sentence
31
+
32
+
```r
33
+
sentences<- get_sentences(comments$comments)
34
+
```
35
+
36
+
##### Compute Emotion Scores per Sentence
37
+
38
+
```r
39
+
emotion_score<- get_nrc_sentiment(sentences)
40
+
```
41
+
42
+
The `get_nrc_sentiment()` function assigns emotion and sentiment scores (based on the NRC lexicon) to each sentence. Each sentence gets numeric values (0 or 1) for the eight emotions to represent their absence or presence. The output also includes positive and negative sentiment scores:
43
+
44
+
45
+
46
+
##### Review Summary of Emotion Scores
47
+
48
+
Let's now compute basic statistics (min, max, mean, etc.) for each emotion column and get an overview of how frequent or strong each emotion is on our example dataset.
49
+
50
+
```r
51
+
summary(emotion_score)
52
+
```
53
+
54
+
This step should generate the following output:
55
+
56
+
57
+
58
+
Based on the results the overall emotion in these comments leans heavily toward **sadness**, which scored the highest average (1.236). It looks like **sadness** and **trust** are the most common feelings, since they're the only ones with a median score of 1.000, meaning at least half the comments contained words for them.
59
+
60
+
On the flip side, **Disgust** was the rarest emotion, with the lowest average (0.145). It's also worth noting that while Sadness and Trust are the most *common*, a few comments really went off the rails with **Trust (47.000), Anger (44.000)**, and **Fear (37.000)**, hitting the highest extreme scores.
61
+
62
+
##### Regroup with comments and IDs
63
+
64
+
After computing scores for emotions, we want to link them back to its **original comment and ID**.
65
+
66
+
```r
67
+
comments$comments<-sentences
68
+
emotion_data<- bind_cols(comments, emotion_score)
69
+
```
70
+
71
+
`bind_cols()` merges the original `comments` data frame with the new `emotion_score` table.
72
+
73
+
##### Summarize Emotion Counts Across All Sentences
74
+
75
+
Now, let's count **how many times each emotion appears** overall.
76
+
77
+
```r
78
+
emotion_summary<-emotion_data %>%
79
+
select(anger:trust) %>% # get only the emotion columns
80
+
summarise(across(everything(), sum)) %>% # sum counts
scale_fill_manual(values= brewer.pal(10, "Paired")) +# Color palette
98
+
theme_minimal(base_size=12) +# Clean theme
99
+
labs(title="Overall Emotion Distribution",
100
+
x="Emotion", y="Total Count") +# Titles and axis labels
101
+
coord_flip() # Flip axes for readability
28
102
```
29
103
30
-
You might be wondering: if the **`syuzhet`** package also computes polarity, why did we choose **`sentimentr`** in our pipeline? The reason is that syuzhet does not inherently account for valence shifters. In the original syuzhet implementation, words are scored in isolation—so “good” = +1, “bad” = −1—regardless of nearby negations or intensifiers. For example, “not good” would still be counted as +1. Because **`sentimentr`** adjusts sentiment scores for negators and amplifiers, polarity results are more nuanced, robust, and reliable.
104
+
105
+
106
+
##### Add a “Season” Variable (Grouping) and Summarize
107
+
108
+
Let's now add a new column called `season` by looking at the ID pattern — for example, `s1_` means season 1 and `s2_` means season 2. This makes it easy to compare the emotional tone across seasons.
109
+
110
+
```r
111
+
emotion_seasons<-emotion_data %>%
112
+
mutate(season= ifelse(grepl("^s1_", id), "s1",
113
+
ifelse(grepl("^s2_", id), "s2", NA)))
114
+
```
115
+
116
+
Time to aggregates the total count of each emotion within each season.
After running the script we should get the following heat map:
182
+
183
+
184
+
185
+
Based on these results, Overall, the emotional picture is pretty interconnected. It looks like the **negative emotions—Sadness, Fear, Anger, and Disgust—are more tightly linked**, meaning when people express one of these, they usually express the others too. In other words, they often show up together in the same comments.
186
+
187
+
While we've only scratched the surface of this particular dataset, the steps we've completed—from calculating basic sentiment scores to visualizing the co-occurrence of emotions—have demonstrated the **power of sentiment and emotion detection**. You now have the foundational skills to convert unstructured text into actionable data, allowing you to understand the **polarity (positive/negative)** and **specific emotional landscape** of any textual dataset.
188
+
189
+
##### Saving our work
190
+
191
+
After performing all the calculations and visualizations, it’s important to save the results so they can be reused or shared.
0 commit comments