You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Sentiment analysis, while a powerful method to extract insights from data, is far from perfect or straightforward. After all, it seeks to interpret natural language, which is constantly evolving. Speaking of evolution, did you know that the [Cambridge dictionary added 6,000 words](https://www.npr.org/2025/08/19/nx-s1-5506163/cambridge-dictionary-adds-more-than-6-000-words-including-skibidi-and-delulu) only this year, including "broligarchy" and "delulu", many of which are widely used by Gen Alpha? This constant expansion of language highlights just how dynamic the texts we analyze can be.
7
+
8
+
Language is also inherently rich, ambiguous, and culturally nuanced. Lexicon-based approaches, for instance, rely on predefined word lists and often struggle to capture subtleties in human expression.
9
+
10
+
In practice, sentiment analysis encounters issues like *code-switching*, where people mix languages in a single post, or compound sentences with mixed sentiments, such as “The movie had great acting, but the ending was lame,” which are difficult to score accurately. *Context dependence* further complicates interpretation: words can flip polarity depending on the domain, like “cheap,” which is positive when describing flights or monetary advantage in general, but negative when describing fabric or referring to quality.
11
+
12
+
Temporal dynamics also play a role, as *slang and cultural references* evolve rapidly, e.g., “bad” meaning “good” in some communities. Ambiguity adds another layer of difficulty: polysemous words like “sick” can mean either “ill” or “awesome”.
13
+
14
+
Another complication is to deal with *sarcasm* and irony which can completely invert the intended sentiment as in: “Oh great, another awesome Monday morning traffic jam!".
15
+
16
+
*Implicit sentiment* may be present even when emotional words are absent, as in “The waiter ignored us for 30 minutes before taking our order.” These factors collectively make sentiment analysis a useful but inherently imperfect tool for understanding human language and emotion.
17
+
18
+
However, it is important to emphasize that as described before, we have only explored sentiment analysis through a lexicon-based approach, and that, as illustrated in Figure ? below, there are other methods, including machine learning, deep learning and their combination (hybrid), that can be employed to extract emotions from text, including user generated content, all with their own limitations and challenges.
19
+
20
+
{fig-align="center" width="500"}
21
+
22
+
For example, Amazon relies on deep learning algorithms to determine the sentiment of customer reviews by identifying positive, negative, or neutral tones in the text. The models are trained on a vast dataset of Amazon’s product descriptions and reviews and are regularly updated with new information. This robust approach enables Amazon to efficiently analyze and interpret customer feedback on a large scale.
23
+
24
+
While there are more advanced approaches to sentiment analysis, including AI-assisted methods, these are discussion topics for future workshops!
Mao, Y., Liu, Q., & Zhang, Y. (2024). Sentiment analysis methods, applications, and challenges: A systematic literature review. *Journal of King Saud University - Computer and Information Sciences, 36*(4), 102048. <https://doi.org/10.1016/j.jksuci.2024.102048>
Now that we have a better understanding of this package, let's get back to business and perform emotion detection to our data.
26
27
27
28
#### Emotion Detection with Syuzhet's NRC Lexicon
28
29
29
-
##### Break Text into Sentences
30
+
##### Detecting Emotions per Comment/Sentence
30
31
31
32
```r
32
33
sentences<- get_sentences(comments$comments)
33
34
```
34
35
35
-
The `get_sentences()` function splits your text into individual sentences.
36
-
37
-
This allows us to analyze emotions at a finer level — rather than by entire comments, we examine each sentence separately. If a comment says: “I love the show. The ending made me sad.” It becomes two sentences.
38
-
39
36
##### Compute Emotion Scores per Sentence
40
37
41
38
```r
42
39
emotion_score<- get_nrc_sentiment(sentences)
43
40
```
44
41
45
-
The `get_nrc_sentiment()` function assigns emotion and sentiment scores (based on the NRC lexicon) to each sentence. Each sentence gets numeric values (0 or 1) for the eight emotions to represent their absence or presence. The output also includes positive and negative sentiment scores.
42
+
The `get_nrc_sentiment()` function assigns emotion and sentiment scores (based on the NRC lexicon) to each sentence. Each sentence gets numeric values (0 or 1) for the eight emotions to represent their absence or presence. The output also includes positive and negative sentiment scores:
43
+
44
+
46
45
47
46
##### Review Summary of Emotion Scores
48
47
@@ -52,9 +51,17 @@ Let's now compute basic statistics (min, max, mean, etc.) for each emotion colum
52
51
summary(emotion_score)
53
52
```
54
53
55
-
##### Rejoin Sentences
54
+
This step should generate the following output:
55
+
56
+
56
57
57
-
After sentence-level analysis, we want to link each emotion score back to its **original comment or ID**.
58
+
Based on the results the overall emotion in these comments leans heavily toward **sadness**, which scored the highest average (1.236). It looks like **sadness** and **trust** are the most common feelings, since they're the only ones with a median score of 1.000, meaning at least half the comments contained words for them.
59
+
60
+
On the flip side, **Disgust** was the rarest emotion, with the lowest average (0.145). It's also worth noting that while Sadness and Trust are the most *common*, a few comments really went off the rails with **Trust (47.000), Anger (44.000)**, and **Fear (37.000)**, hitting the highest extreme scores.
61
+
62
+
##### Regroup with comments and IDs
63
+
64
+
After computing scores for emotions, we want to link them back to its **original comment and ID**.
@@ -92,6 +101,8 @@ ggplot(emotion_summary, aes(x = emotion, y = count, fill = emotion)) +
92
101
coord_flip() # Flip axes for readability
93
102
```
94
103
104
+
105
+
95
106
##### Add a “Season” Variable (Grouping) and Summarize
96
107
97
108
Let's now add a new column called `season` by looking at the ID pattern — for example, `s1_` means season 1 and `s2_` means season 2. This makes it easy to compare the emotional tone across seasons.
@@ -126,7 +142,7 @@ ggplot(emotion_long, aes(x = reorder(emotion, -count), y = count, fill = season)
126
142
coord_flip()
127
143
```
128
144
129
-
##### Plotting the Data
145
+
130
146
131
147
Now, let's explore to see which emotions tend to occur together, revealing patterns of emotional co-occurrence in the text.
132
148
@@ -162,14 +178,18 @@ ggplot(co_occurrence_long, aes(x = emotion1, y = emotion2, fill = correlation))
162
178
)
163
179
```
164
180
181
+
After running the script we should get the following heat map:
182
+
183
+
184
+
185
+
Based on these results, Overall, the emotional picture is pretty interconnected. It looks like the **negative emotions—Sadness, Fear, Anger, and Disgust—are more tightly linked**, meaning when people express one of these, they usually express the others too. In other words, they often show up together in the same comments.
186
+
187
+
While we've only scratched the surface of this particular dataset, the steps we've completed—from calculating basic sentiment scores to visualizing the co-occurrence of emotions—have demonstrated the **power of sentiment and emotion detection**. You now have the foundational skills to convert unstructured text into actionable data, allowing you to understand the **polarity (positive/negative)** and **specific emotional landscape** of any textual dataset.
188
+
165
189
##### Saving our work
166
190
167
191
After performing all the calculations and visualizations, it’s important to save the results so they can be reused or shared.
You might be wondering: if the **`syuzhet`** package also computes polarity, why did we choose **`sentimentr`** in our pipeline? The reason is that syuzhet does not inherently account for valence shifters. In the original syuzhet implementation, words are scored in isolation—so “good” = +1, “bad” = −1—regardless of nearby negations or intensifiers. For example, “not good” would still be counted as +1. Because **`sentimentr`** adjusts sentiment scores for negators and amplifiers, polarity results are more nuanced, robust, and reliable.
0 commit comments