Topics and Sentiment Surrounding Vaping on Twitter and Reddit During the 2019 e-Cigarette and Vaping Use–Associated Lung Injury Outbreak: Comparative Study

Article type: Research Article

Keywords: vaping, e-cigarette, social media, Twitter, Reddit, e-cigarette and vaping use–associated lung injury, EVALI, sentiment analysis, topic analysis

License: ©Dezhi Wu, Erin Kasson, Avineet Kumar Singh, Yang Ren, Nina Kaiser, Ming Huang, Patricia A Cavazos-Rehg. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 13.12.2022. CC BY 4.0 This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

Article links: DOI: 10.2196/39460  |  PubMed: 36512403  |  PMC: PMC9795395

Relevance: Moderate: mentioned 3+ times in text

Introduction

Background

In the United States, vaping has become dramatically more popular in recent years, with 1 in every 20 American adults using vaping devices and >2 million middle- and high-school students in the United States using e-cigarettes in 2021 [ref. 1,ref. 2]. Vaping places individuals at risk for several negative health consequences including diminished lung function and cardiac performance, susceptibility to nicotine dependence, and impacted neurological development, particularly among youth [ref. 3,ref. 4]. However, despite these negative health consequences, youth and young adults have been found to report limited understanding of the dangers of vaping [ref. 5,ref. 6] and high perceived ability to quit vaping if desired [ref. 7]. Of further concern, e-cigarette and vaping use–associated lung injury (EVALI) resulted in hospitalizations and deaths in 2019, and many of these cases were later linked to vitamin E acetate (a filler substance in unregulated products) [ref. 8]. In the context of these risks and negative health outcomes, the United States Food and Drug Administration–labeled vaping among teens as a national epidemic in 2018 and continues to release policies to regulate vaping products more effectively [ref. 9]. Given the deleterious health effects of vaping and increased risks for EVALI, future research on publicly available, larger-scale data from sources such as social media are necessary to monitor this growing public health concern and to inform outreach interventions for vaping cessation. Previous literature has leveraged social media data for surveillance of health topics, including illicit drug use [ref. 10], mental well-being [ref. 11,ref. 12], public health [ref. 13,ref. 14], and other health-related experiences [ref. 15]. Twitter is a social media site that is used by approximately 22% (1/5) of American adults [ref. 16] as a source of information as well as information sharing [ref. 17]. Individuals on web-based platforms such as Twitter may be more willing to openly share experiences and personal stories about mental health or substance misuse with reduced fears of judgment or legal action, allowing them to access social support and advice and share this advice with others who are going through similar experiences [ref. 18]. For example, a study of 1200 tweets during mental health awareness week found that awareness, stigma, and personal experiences were central themes of discourse among Twitter users [ref. 19]. As such, Twitter has been used as a mass data source of information for public health monitoring and can be used to better understand attitudes and behaviors of individuals in relation to vaping [ref. 20–ref. 22]. For instance, during the COVID-19 pandemic, Twitter data were used to better understand sentiment and reactions to smoking in relation to the virus [ref. 23] as well as individual perspective of global-scale events and US-related lifestyle changes [ref. 24]. Although Twitter has several strengths related to surveillance and public health monitoring, other social media platforms such as Reddit may have complementary strengths to provide data on individual-level user vaping behaviors.

Reddit is a similar pseudonymous social media platform used by the public to discuss personal experiences that may be stigmatizing [ref. 25–ref. 27], including young adults who may disclose personal information with less fear of offline harm or consequences [ref. 28]. Reddit data have been used to investigate attitudes and behaviors of individuals who use illicit substances [ref. 29–ref. 31], and similar research has been conducted among those who vape. One analysis of Reddit threads indicated that primary motivations for vaping among individuals with mental illness include self-medication, freedom and control, vaping as a hobby, social connectedness, as well as vaping to quit smoking [ref. 32]. Other studies have used Reddit data to analyze public responses and concerns about vape bans [ref. 33], communities supporting e-cigarette cessation [ref. 34], and attitudes and reviews toward e-cigarette products [ref. 35].

Both Twitter and Reddit are popular social media platforms, but they differ in multiple ways that impact users’ posting behaviors and post content. Twitter, with >300 million monthly active users [ref. 36], only allows short 280-character tweets for breaking news, trends, and opinions, often leading to incomplete or misleading statements [ref. 37]. In contrast, Reddit, with >430 million actively monthly users, has no character posting limit, is anonymous, and comprises network of communities, namely, subreddits, dedicated to specific topics, allowing users to relate to other individuals with similar backgrounds, views, and lived experiences. With Reddit’s anonymity, people can honestly voice their own opinions with in-depth text and content to spread awareness and important news [ref. 38,ref. 39]. Thus, posts about the same topic during the same period (ie, posts about vaping in 2019) are expected to vary with regard to the type of content shared and the level of impact on public perception based on the platform on which they are shared.

Large-scale evaluations using computer science (CS) strategies, including those using natural language processing and machine learning for text mining, have been conducted previously on vaping content from social media [ref. 40–ref. 42]. For example, Visweswaran et al [ref. 41] developed machine learning classifiers to identify vaping-relevant tweets toward the development of a vaping surveillance system. Results demonstrate that social media content can be used for overall infoveillance, and such data could inform future, individual-level detection models to identify at-risk posts and users. A systematic review conducted by Kwon and Park [ref. 32] found that sentiment regarding vaping tended to be more positive across social media sites, and previous research on Twitter has demonstrated that those who smoke are more likely to engage with misinformation about vaping [ref. 43]. Studies conducted on Reddit posts have illustrated health symptoms associated with vaping [ref. 44] and highlighted communities aimed to support those wanting to quit vaping [ref. 45].

Objectives

Studying the EVALI public health crisis specifically could aid in the identification of content and keywords related to both acute and long-term health outcomes associated with vaping shared on social media, as such signals of vaping risk may have been amplified during this period. By leveraging the strengths of both Twitter and Reddit as publicly available data sources as well as using an interdisciplinary approach to analyze complex social media content, technical and clinical insights may be garnered to inform the future development of an automatic detection model to connect with vaping users who may benefit from digital intervention on social media platforms. However, to date, there are few studies comparing insights from both Twitter and Reddit for substance misuse within the same time frame [ref. 46,ref. 47], and no known studies related to vaping have been conducted to analyze the 2019 EVALI outbreak at both the individual user level, and population level. As such, this paper examined vaping-related content on Twitter and Reddit to better understand the (1) sentiment and keywords associated with vaping-related content during the 2019 EVALI time frame, (2) differences in sentiment and keywords between content on Twitter and Reddit, and (3) similarities or differences between statistical analyses and clinical coding of vaping-related content.

Methods

Data Collection

In this study, we focused on comparing vaping-related keyword frequencies and sentiment on Twitter and Reddit during the EVALI outbreak period using data from both platforms from July 2019 to September 2019, as our previous work had identified this as a time frame during which vaping-related social media content increased [ref. 48]. To define the criteria for large-scale data extraction, our team first conducted a manual analysis of 200 randomly selected vaping-related tweets across the 2019 time line to generate a list of clinically relevant keywords. Our primary research questions guided the creation of this keyword list, which included vaping, vape, and 60 other specifying terms (Multimedia Appendix 1). Using this set of keywords, a random sample of 316,620 vaping-related tweets with an average of 27 words per tweet was extracted during the EVALI outbreak period (July, August, and September 2019). For comparison purposes, we used the same set of keywords to randomly extract Reddit data, resulting in 17,320 Reddit posts with an average of 211 words per post associated with vaping during the EVALI outbreak.

GetOldTweets [ref. 49] is an open-source python library that allowed our team to extract a random sample of tweets with our identified vaping keywords. This module permitted access to and extraction of historical tweets of any date and topic. The benefit of using this application program interface (API) is that it had no restrictions on size and provided access to historical tweets [ref. 49,ref. 50]. We used Pushshift Reddit API (version 4.0) [ref. 51], which provided rich features for searching and extraction and flexible ways to aggregate publicly available Reddit posts and comments.

Data Cleaning

After we extracted posts from Twitter and Reddit based on the keywords, we cleaned our data sets before further analysis. As we only focused on English-language posts in this study, we first removed the posts that contain non-English languages. We also removed invalid Reddit posts marked as “removed” or “deleted.” After that, the number of Twitter posts reduced from 316,620 to 286,703, and the number of Reddit posts reduced from 17,320 to 12,069.

For the text in the posts, we first converted all the characters to lowercases to avoid the case-sensitive process. Then, we removed all special characters non–American Standard Code for Information Interchange from the text. For text contractions, we expanded them into multiple individual words. Next, we removed the stop words that have no significant contributions to the meaning of the text from the text (eg, is, a, the, and of). After that, we removed the special terms from the tweet text, including mentions, hashtags, links, ticks, punctuations, numbers, and over spaces. Then, we applied the word lemmatization function to convert the words to their base forms.

Sentiment Analysis

Sentiment analysis is a common computer technique to measure the subjectivity, opinions, attitudes, and emotions in texts [ref. 52]. Sentiment analysis quantifies the sentiment contents in a given text along a continuum scale, for example, from −1 to 1 [ref. 41,ref. 53]. We applied Valence Aware Dictionary and sEntiment Reasoner (VADER) as the tool to analyze the sentiment of tweets and Reddit posts, as VADER is a lexicon and rule-based sentiment analysis tool [ref. 54] that recent studies [ref. 24,ref. 55,ref. 56] have found to effectively calculate sentiment social media analysis. More specifically, VADER has been attuned to social media sentiments and pretrained by a gold standard sentiment lexicon, which was developed based on mature sentiment word-banks, popular sentiment expression, and common slang with sentiment value in social media. To determine the sentiment, VADER maps lexical features to emotion intensities known as a sentiment score, which can be obtained by summing up the intensity of each word in the text. The score is then normalized to −1 (most extreme negative) and +1 (most extreme positive). In our study, if the text sentiment score was >0, then the text was classified as positive. The text was classified as negative if the sentiment score was <0. The neutral text’s sentiment score was 0. Our study further classified posts into positive, negative, and neutral sentiment toward vaping using this sentiment score, calculating the distribution of the posts in terms of the 3 sentiment types per month.

Keyword Analysis

In addition to the sentiment analysis described earlier, we used chi-square tests to compare differences between the frequency of keywords in Twitter and Reddit posts during each month across the following topics: (1) sentiment, (2) emotion-related keywords, (3) health-related keywords, (4) age-related keywords, (5) marketing-related keywords, (6) product-related keywords, (7) addiction-related keywords, and (8) quitting-related keywords.

Term Frequency–Inverse Document Frequency

Term frequency–inverse document frequency (TF-IDF) is a statistical measurement that can represent the word relevant in a corpus [ref. 57]. The TF-IDF score is calculated based on the term frequency and inverse document frequency. Using this method helps us find the common words on Twitter and on Reddit. On the basis of the TF-IDF scores, we can identify the most important words on both the platforms. The formulas are as follows:

Clinical Coding Comparison

During the EVALI outbreak, July, August, and September 2019 were identified as months during and just before the dramatic increase in vaping-related discussions on Twitter based on both the content and sentiment analyses outlined earlier. As such, a random sample of 200 posts per month from the Twitter and Reddit data sets described earlier were extracted for in-depth human coding toward contextual content analysis. Specifically, members of our clinical team with experience in substance use research (students in psychology, social work, or public health at the graduate level and with relevant experience coding qualitative social media data led by author PCR, a clinical psychologist) used inductive and deductive methods to construct a codebook based on a review of sample tweets and informed by previous literature [ref. 58,ref. 59]. Three primary coding categories were used: (1) type of post, including personal, marketing, or media or news or other [ref. 60]; (2) sentiment toward vaping [ref. 61]; and (3) health outcomes mentioned, including both positive (eg, quitting combustible smoking) and negative (eg, lung injury, death, and addiction or dependence) [ref. 60,ref. 62]. Secondary concepts that were coded as either present or not present included (1) mentions teens or adolescents or young adults [ref. 63] and (2) mentions marijuana or weed or cannabidiol or tetrahydrocannabinol [ref. 64,ref. 65]. Two independent human coders reviewed each post and assigned applicable codes based on text content, and agreement among coders was substantial as reflected by an average κ score of 0.62 [ref. 66]. A third coder then reviewed the coding from each preliminary coder and provided final codes for those tweets on which there was disagreement [ref. 67], which is a third-party resolution method used in previous qualitative analysis literature [ref. 68]. Both frequency and qualitative themes were then compared with the preliminary results from the CS analyses to aid in the conceptualization of the clinical themes reflected in the data set.

Total frequency of each theme mentioned on both Twitter and Reddit was compared across the months of July, August, and September 2019 (sum of 3 months) to demonstrate relative weight of each topic on the respective platforms.

Ethics Approval

The Washington University Institutional Review Board (202101009) reviewed the methods of data extraction and analysis for this study. Given that the data are publicly available on social media, the study was determined to be nonhuman subjects research and exempt from review.

Results

Data Set Summary and Unique Users

This section presents the results from the high-throughput computational analyses. In total, we collected 286,703 tweets and 12,096 posts on Reddit. The sample size differences between Twitter and Reddit were related to the amount of information included in each Reddit post and in a tweet. The word limit for each tweet is 280 characters, whereas the word limit for each Reddit post is 40,000 characters. Thus, each Reddit post included much richer information than a tweet. To analyze the data set at the word level and further content analysis, the number of extracted Reddit posts was significantly smaller than the number of tweets. Table 1 presents the number of unique users and posts per user on both platforms. Overall, the number of vaping-related posts and unique users on Twitter and Reddit had an increasing trend from July 2019 to September 2019. In particular, the number of posts and unique users on Twitter increased by approximately 4 times from August 2019 to September 2019. The number of posts per user on Twitter and Reddit increased from 1.68 to 1.81 and 1.19 to 1.21, respectively.

Sentiment Analysis Results

CS pattern analysis of sentiment found that overall posts with positive sentiment about vaping were more common than negative posts on Reddit (8905/12,096, 73.62%), and negative sentiment was dominant on Twitter (174,448/286,703, 60.86%) during the EVALI period (Table 2). Clinical results based on a small random sample during this period were similar to the results using CS methods, still demonstrating that Reddit had a higher number of positive sentiment posts and also reflecting that Twitter had a higher number of negative sentiment posts based on manual review of post content.

The results of monthly sentiment trends indicated that the percentage of posts with positive sentiment was higher than that with negative sentiment in July both on Twitter and on Reddit. In August and September, the percentage of negative posts was higher than that of the positive ones on Twitter. Moreover, there was a significant decrease in the percentage of positive sentiment from July to September on Twitter, whereas positive posts were dominant on Reddit in August and September.

The chi-square tests (Table S1 in Multimedia Appendix 2) found an overall significant difference in sentiment between platforms. Twitter contained significantly more negative postings (174,488/286,703, 60.86%) than Reddit (2281/12,096, 18.86%), and Reddit contained significantly more positive posts (8905/12,095, 73.62%) than Twitter (85,209/286,703, 29.72%).

In addition to the sentiment analysis and trends, we also ran chi-square tests to compare emotion expression–related posting differences on Twitter and Reddit. We selected common emotional words from the list of most frequent words on both Twitter and Reddit. Positive keywords included safe, good, and love, and negative keywords included kill, bad, dangerous, concern, and serious. The statistical results indicated significant posting differences between the 2 platforms as a whole, based on their frequency percentages. We found that positive emotion expressions were much more significant on Reddit than on Twitter in all 3 months during the EVALI outbreak period (Table S2 in Multimedia Appendix 2).

Keyword Analysis by Topic

Health-Related Keyword Analysis

The distributions and percentages of the posts that contained vaping health-related keywords are shown in Table 3. Figure 1 presents the frequency of the top 6 words associated with health issues in July, August, and September 2019. The top 6 words were commonly shared between Twitter and Reddit. On the basis of the TF-IDF scores as shown in Multimedia Appendix 3, we found that the most important health-related keywords often mentioned on Twitter included death, lung, quit, smoking, disease, and harm, whereas the most important words in the Reddit posts included death, lung, quit, smoking, cough, and doctor.

We performed a chi-square test to compare health-related keywords, including death, lung, disease, risk, crisis, sick, doctor, cancer, injury, epidemic, research, damage, harm, harmful, patient, cough, chest, prevention, smoking, and quit based on the posts in July, August, and September 2019. The chi-square test results (Table S3 in Multimedia Appendix 2) showed significant differences between health-related keywords posting on Twitter and Reddit for each of the 3 months and as a whole. However, owing to the significant differences between the size of posts on Twitter and Reddit, the overall effect size was small. On the basis of the percentages, more health-related keywords were discussed on Twitter than on Reddit, and negative health outcomes were highly discussed on both Reddit and Twitter.

In addition to investigating the sentiment of health-related keywords, chi-square tests associated with addiction-related keywords (Table S4 in Multimedia Appendix 2) showed significant differences and small effect sizes between platforms in each month and the entire EVALI outbreak period. On the basis of percentages, the addiction-related keywords were mentioned more significantly on Twitter than on Reddit.

Within the in-depth clinical coding, negative health outcomes were mentioned much more frequently on both the platforms (Twitter: 230/577, 39.9% and Reddit: 227/578, 39.3%) than positive health outcomes (Twitter: 134/577, 23.2% and Reddit: 182/578, 31.5%). Additional topic mentioned within these negative health outcomes included EVALI/hospitalization, which was more prevalent on Twitter (Twitter: 176/577, 30.5% and Reddit: 146/578, 25.3%), whereas addiction or dependence on vaping products was mentioned more often on Reddit (Twitter: 57/577, 9.9% and Reddit: 123/578, 21.3%). Those mentioning positive health outcomes related to vaping were more common on Reddit, consistent with the keyword analysis described earlier; further, clinical coding found that vaping as a means of quitting combustible smoking was more often mentioned on Reddit than on Twitter (Twitter: 118/577, 20.5% and Reddit: 177/578, 30.6%).

Age-Related Keyword Analysis

The top 6 words related to age groups in July, August, and September 2019 are presented in Figure 2 for Twitter and Reddit based on frequency. Among the age-related keywords, kids was the most used word on Twitter and Reddit after August 2019. Other frequently used words on Twitter included youth, young, child, and teenager. Reddit posts more often contained words such as parent, school, and family.

Age-related keywords in our data set included kid, adult, child, young, old, youth, parent, school, age, student, family, teenager, minor, mother, husband, wife, adolescent, father, and aunt in July, August, and September 2019 separately. The chi-square test results (Table S5 in Multimedia Appendix 2) showed significant differences and small effect sizes between age-related keywords on Twitter and Reddit for each of the 3 months and as a whole and indicated that age-related keywords were more frequently mentioned on Twitter than on Reddit. Clinical review of post content focused only on mentions of youth and young adults and found differing results, showing that Twitter had 22.9% (132/577) of tweets mentioning youth and Reddit had 28.5% (165/578) of posts mentioning this group.

Marketing-Related Keyword Analysis

Frequently used words about vaping marketing were highly similar between Twitter and Reddit, including sale, commercial, market, black market, and promote. The trends of top 5 marketing-related words on Twitter and Reddit during the 3 months of EVALI outbreak are illustrated in Figure 3. Mentions of sale, black market, and commercial increased on Twitter and Reddit from July 2019 to September 2019.

Marketing-related keywords in our data set included black market, black market, market, sale, news, promote, marketing, commercial, blackmarket, and media for July, August, and September 2019. The chi-square test results (Table S6 in Multimedia Appendix 2) showed significant differences with small effect sizes between marketing-related keywords posting on Twitter and Reddit for all the 3 months, indicating that they were discussed more frequently on Twitter than on Reddit. Clinical analyses focused on marketing regulation and policies and had different results, showing that Twitter discussed policies 8.3% (48/577) of the time and Reddit discussed policies 20% (116/578) of the time.

Vaping Product Keyword Analysis

The detailed distributions and percentages of the vaping product keywords are listed in Multimedia Appendix 3, and the top words related to vaping substances on Twitter and Reddit are illustrated in Figure 4. On both platforms, the most frequent word about vaping ingredients or products was cigarette, and mentions of marijuana-related keywords (weed, CBD, THC, and cannabis) and alcohol were also prevalent. On Reddit, specific keywords about product components, such as juice, cartridge, and liquid, were slightly more common. The most common words on Reddit included cigarette, product, and juice, which varied across months. The most common words on Twitter included cigarette, tobacco, and product and stayed consistent across August 2019 to September 2019. On the basis of the TF-IDF scores as shown in Multimedia Appendix 3, we found that the most important words in the posts from Twitter included cig, cigarette, tobacco, product, thc, and nicotine, whereas the most important words in the Reddit posts included nicotine, cigarette, juice, and weed.Vaping product-related keywords in our data set included cigarette, tobacco, product, thc, cig, nicotine, juice, juul, cartridge, liquid, cannabis, chemical, alcohol, ecigarette, weed, cbd, flavour, and ingredient based on the data sets in July, August, and September 2019. The chi-square test results (Table S7 in Multimedia Appendix 2) showed significant differences with small effect sizes between vaping product-related keywords posting on Twitter and Reddit for all the 3 months, finding that more vaping product-related keywords were mentioned on Twitter based on percentages. Clinical analyses found different results, showing that marijuana-related keywords were mentioned more than twice as often on Reddit (208/578, 35.9%) than on Twitter (77/577, 13.3%).

Quitting Vaping

In addition, quit-related keywords in our data set included quit, quitting, stop, and stopper to compare pattern differences on Twitter and Reddit. The chi-square test results indicated significant posting differences with small effect sizes between the 2 platforms for all the 3 months and as a whole, showing quit-related words mentioned on Twitter more often based on percentages (Table S8 in Multimedia Appendix 2). Clinical analyses showed different results, with Reddit having 21.8% (126/578) of posts related to quitting and Twitter having 6.4% (37/577) of tweets related to quitting.

Discussion

Principal Findings and Implications

As vaping has become more popular in recent years, so have discussions about its direction, policies, and health connotations on social media platforms, and this study illustrated differences in sentiment and keyword content on Twitter and Reddit during the EVALI outbreak in 2019. According to the trends in the frequency of vaping-related posts during this time frame, vaping-related content increased slowly between July and August, with a dramatic spike from August to September. Moreover, there was a significant increase in the number of unique Twitter and Reddit users who participated in these discussions during the EVALI outbreak. The fact that increasing trends in the frequency of social media vaping-related content peaked in parallel with the EVALI outbreak and across both popular social media platforms supported the utility of social media as a surveillance system for exploring naturally occurring, real-time reactions and communications during a public health vaping-associated crisis.

Importantly and based on our content analysis, Twitter and Reddit content within posts about vaping were found to contain primarily positive sentiment about vaping. However, the 2 platforms were notably different based on the most prevalent type of content identified. Specifically, Reddit users tended to reveal personal vaping experiences and opinions about vaping benefits, policies, and products, including how potential restrictive vaping policies may have negative impacts on users who vape (ie, less access to vaping products that aid cigarette smoking cessation). Mentions of marijuana were also >2 times as high on Reddit as on Twitter and often included queries to other Reddit users about the safety of specific vaping products and which symptoms, if any, should warrant concern or medical care. In contrast, Twitter included more mainstream media content surrounding vaping, specifically related to the rise in EVALI cases. We also observed that Twitter feeds contained attention-grabbing negative sentiment and higher use of negative emotional expressions, including kill, bad, dangerous, concern, and serious, as well as increased content on possible negative health outcomes of vaping, including addiction. Although both platforms had mentions of youth, Twitter highlighted headlines about the youth vaping epidemic and EVALI among teens and ways to limit vaping products for adults who use them as smoking cessation aids, whereas on Reddit, mentions related to youth mostly were individuals describing their own vaping behaviors, including initiating vaping behaviors as a teen.

In summary, we observed numerous and meaningful distinctions in the frequencies of content topics across both social media outlets. These differences may be owing to the way individuals socially network as well as their motive for discussion on each platform. For instance, information on Twitter is known as “the” social media platform for news coverage, and it is most often used by journalists and major news providers to broadcast news and update the public in real time as important events transpire [ref. 69]. This may explain why Twitter had a higher frequency of negative posts related to vaping, as journalists and their audience leveraged this platform for updates and interactions throughout the unfolding of the EVALI outbreak, especially as it evolved into a crisis that resulted in many hospitalizations and several deaths. In contrast, Reddit distinguishes itself from other social media platforms by facilitating more candid discussions, including exchanges about substance use behaviors, given its pseudonymous user system and generous character limit restrictions; this may be why we found a higher prevalence of content describing one’s personal experiences with vaping.

Comparison With Previous Work

It might also be that the differences we found were owing to the distinctions between the users themselves. For instance, the demographic user base of Twitter is predominantly White adults, who have a higher degree of education and are more likely to be identified as Democrat than the general public, with 10% of users creating 80% of the tweets [ref. 70]. In contrast, Reddit users tend to span degrees of education attainment and live in urban or suburban areas [ref. 71]. The Centers for Disease Control and Prevention finds that within people of color, there are higher percentages of individuals who vape compared with the percentage of White people who vape [ref. 72], and another study shows that higher level of education attainment was linked to lower odds of e-cigarette use [ref. 73]. This suggests that users on Reddit may be more likely to vape than users on Twitter, explaining their different sharing patterns of personal vaping-related experiences and concerns over restrictive policy.

Limitations

The findings of this paper should be considered within the context of its limitations. First, we analyzed only text-based posts or messages on these platforms. Although this provided us with data-heavy information from each social media site, it did not include the multitude of multimedia content including photos, videos, and links that are available for further analysis. Second, owing to the character limits on Twitter and the unlimited length of Reddit posts, the differences between the number of words in each post could have impacted both the sentiment and keyword analyses in this study. Third, because of the timing of our data gathering, we did not garner information related to COVID-19 and its implications on those who vape and vaping policies, leaving us unable to discern more recent implications. Fourth, our original keyword list used to extract the vaping-related data sets from Twitter and Reddit may have contained more negative health-related keywords, and this could have impacted the results with regard to sentiment and health outcomes, causing a potential selection bias in our keyword list. As this study focused on the health issue regarding vaping-related topics on Twitter and Reddit during the outbreak period, the keyword list included multiple sentiment-related words. It will bring bias to our sentiment analysis results, but these words were the key to selecting the related posts and addressing our research questions. In addition, we applied the GetOldTweets and Pushshift APIs to extract the data based on the keyword list. As the extraction mechanism of these APIs is to find the posts with the same field as one of the keywords without further filtration to matched posts, the extracted data set might include the posts from bots instead of real users, which may introduce bias to our sentiment results. We plan to apply different methods to clean the posts generated by bots in our future studies. However, the use of this data set was in line with the larger aims of this study, which were to better understand the content and sentiment surrounding vaping on Twitter and Reddit to inform the development of potential identification and outreach methods on social media to those at risk of negative health outcomes to improve public health. The fifth limitation was that we applied an existing tool VADER to analyze the sentiment of the posts, and thus, it could bias to our sentiment analysis results, which are common issues for any sentiment analysis tool owing to the complex dynamics of human expressions, emotions, and contexts. In the future, we will also consider creating a sentiment analysis model optimization with social media posts to overcome the current disadvantages of not effectively identifying sarcastic sentences.

Future Directions

Overall, the results of this study revealed the strengths of both Twitter and Reddit as publicly available social media data sources as a public health crisis transpired and evolved. Health practitioners working with those who vape or who have interest in quitting vaping should be aware of the information and possible misinformation related to vaping and work to assess whether social media engagement on various platforms could impact continued use or be a barrier to cessation. The results shared in this manuscript could also inform social media companies and public health officials by alerting them to the marketing of vaping products on these sites and encouraging protections for communities such as those on Reddit aimed to support vaping cessation. In addition, to improve public health reach, future research could explore automatic detection mechanisms that leverage each platform’s content and type of networking identified here, especially to study the potential for identifying users that are vaping and may want information and support to quit. This could help lead to efficient and timely social media informed proactive outreach strategies to distribute health education about vaping, including strategies for vaping cessation.

References

1. M Mirbolouk, P Charkhchi, S Kianoush, SM Uddin, OA Orimoloye, R Jaber, A Bhatnagar, EJ Benjamin, ME Hall, AP DeFilippis, W Maziak, K Nasir, MJ Blaha. Prevalence and distribution of e-cigarette use among U.S. adults: behavioral risk factor surveillance system, 2016. Ann Intern Med, 2018. [DOI | PubMed]
2. Youth E-Cigarette Use Remains Serious Public Health Concern Amid COVID-19 Pandemic. Centers for Disease Control and Prevention, 2021
3. S Singh, SB Windle, KB Filion, BD Thombs, JL O’Loughlin, R Grad, MJ Eisenberg. E-cigarettes and youth: patterns of use, potential harms, and recommendations. Prev Med, 2020. [DOI | PubMed]
4. MA El-Mahdy, EM Mahgoup, MG Ewees, MS Eid, TM Abdelghany, JL Zweier. Long-term electronic cigarette exposure induces cardiovascular dysfunction similar to tobacco cigarettes: role of nicotine and exposure duration. Am J Physiol Heart Circ Physiol, 2021. [DOI | PubMed]
5. E Park, M Kwon, MR Gaughan, JA Livingston, Y Chang. Listening to adolescents: their perceptions and information sources about e-cigarettes. J Pediatr Nurs, 2019. [DOI | PubMed]
6. ME Morean, KW Bold, G Kong, R Gueorguieva, DR Camenga, P Simon, A Jackson, DA Cavallo, S Krishnan-Sarin. Adolescents’ awareness of the nicotine strength and e-cigarette status of JUUL e-cigarettes. Drug Alcohol Depend, 2019. [DOI | PubMed]
7. K Strombotne, J Sindelar, J Buckell. Who, me? Optimism bias about US teenagers’ ability to quit vaping. Addiction, 2021. [DOI | PubMed]
8. Outbreak of Lung Injury Associated with the Use of E-Cigarette, or Vaping, Products. Centers for Disease Control and Prevention, 2020
9. Vaping Illness Update: FDA Warns Public to Stop Using Tetrahydrocannabinol (THC)-Containing Vaping Products and Any Vaping Products Obtained Off the Street. U.S. Food & Drug Administration, 2021
10. DM Kazemi, B Borsari, MJ Levine, B Dooley. Systematic review of surveillance by social media platforms for illicit drug use. J Public Health (Oxf), 2017. [DOI | PubMed]
11. R Skaik, D Inkpen. Using social media for mental health surveillance: a review. ACM Comput Surv, 2021. [DOI]
12. S Amir, M Dredze, JW Ayers. Mental health surveillance over social media with digital cohorts. Proceedings of the 6th Workshop on Computational Linguistics and Clinical Psychology, 2019. [DOI]
13. MJ Paul, A Sarker, JS Brownstein, A Nikfarjam, M Scotch, K Smith, G Gonzalez. Social media mining for public health monitoring and surveillance. Proceedings of the 2016 Pacific Symposium on Biocomputing, 2016. [DOI]
14. AE Aiello, A Renson, PN Zivich. Social media- and Internet-based disease surveillance for public health. Annu Rev Public Health, 2020. [DOI | PubMed]
15. K Jiang, S Feng, Q Song, RA Calix, M Gupta, GR Bernard. Identifying tweets of personal health experience through word embedding and LSTM neural network. BMC Bioinformatics, 2018. [DOI | PubMed]
16. M Odabaş. 10 facts about Americans and Twitter. Pew Research Center, 2022
17. J Phua, SV Jin, JJ Kim. Uses and gratifications of social networking sites for bridging and bonding social capital: a comparison of Facebook, Twitter, Instagram, and Snapchat. Comput Human Behav, 2017. [DOI]
18. N Berry, F Lobban, M Belousov, R Emsley, G Nenadic, S Bucci. #WhyWeTweetMH: understanding why people use Twitter to discuss mental health problems. J Med Internet Res, 2017. [DOI | PubMed]
19. M Makita, A Mas-Bleda, S Morris, M Thelwall. Mental health discourses on Twitter during mental health awareness week. Issues Ment Health Nurs, 2021. [DOI | PubMed]
20. SE Jordan, SE Hovet, IC Fung, H Liang, KW Fu, ZT Tse. Using Twitter for public health surveillance from monitoring and prediction to public response. Data, 2019. [DOI]
21. X Dai, M Bikdash, B Meyer. From social media to public health surveillance: word embedding based clustering method for Twitter classification. Proceedings of the 2017 SoutheastCon, 2017. [DOI]
22. JP Allem, A Majmundar, L Dharmapuri, TB Cruz, JB Unger. E-liquid-related posts to Twitter in 2018: thematic analysis. Addict Behav Rep, 2019. [DOI | PubMed]
23. K McCausland, B Maycock, T Leaver, K Wolf, B Freeman, J Jancey. E-cigarette advocates on Twitter: content analysis of vaping-related tweets. JMIR Public Health Surveill, 2020. [DOI | PubMed]
24. D Valdez, M Ten Thij, K Bathina, LA Rutter, J Bollen. Social media insights into US mental health during the COVID-19 pandemic: longitudinal analysis of Twitter data. J Med Internet Res, 2020. [DOI | PubMed]
25. KB Rhidenour, K Blackburn, AK Barrett, S Taylor. Mediating medical marijuana: exploring how veterans discuss their stigmatized substance use on Reddit. Health Commun, 2022. [DOI | PubMed]
26. J Pestana, F Beccaria, E Petrilli. Psychedelic substance use in the Reddit psychonaut community. A qualitative study on motives and modalities. Drugs Alcohol Today, 2021. [DOI]
27. JL Green, SK Vosburg, R Robbins, SV Faraone, KM Antshel. Substance use trajectories: nonmedical use (NMU) of prescription stimulants via non-oral routes of administration among adults recruited from Reddit. CNS Spectr, 2021. [DOI]
28. N Andalibi, OL Haimson, M De Choudhury, A Forte. Understanding social media disclosures of sexual abuse through the lenses of support seeking and anonymity. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 2016. [DOI]
29. MR Kaufman, AT Bazell, A Collaco, J Sedoc. "This show hits really close to home on so many levels": an analysis of Reddit comments about HBO’s Euphoria to understand viewers’ experiences of and reactions to substance use and mental illness. Drug Alcohol Depend, 2021. [DOI | PubMed]
30. SK Vosburg, RS Robbins, KM Antshel, SV Faraone, JL Green. Characterizing pathways of non-oral prescription stimulant non-medical use among adults recruited from Reddit. Front Psychiatry, 2021. [DOI | PubMed]
31. J Lu, S Sridhar, R Pandey, M Al Hasan, G Mohler. Investigate transitions into drug addiction through text mining of Reddit data. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019. [DOI]
32. M Kwon, E Park. Perceptions and sentiments about electronic cigarettes on social media platforms: systematic review. JMIR Public Health Surveill, 2020. [DOI | PubMed]
33. YA Xu, HM Kim, Y Wang, J Wu, T Hong, M McLaughlin. Public responses and concerns regarding vape bans on Reddit: a longitudinal topic modeling approach. Proceedings of the 13th International Conference on Social Computing and Social Media: Experience Design and Social Network Analysis, 2021. [DOI]
34. K Shaw, SL Watkins. E-cigarette Cessation on Reddit:​ How users discuss their cessation experience, social support, and recommendations to quit. College of Public Health, The University of Iowa, 2020
35. JP Allem, A Majmundar, L Dharmapuri, JB Unger, TB Cruz. Insights on electronic cigarette products from reviews on the Reddit forum. Tob Prev Cessat, 2019. [DOI | PubMed]
36. M Iqbal. Twitter revenue and usage statistics. Business of Apps, 2022
37. 3 reasons why Twitter is so toxic – and how you can protect yourself. Notifier, 2021
38. W Sattleberg. The Demographics Of Reddit: Who Uses The Site?. Alphr, 2021
39. M Rutnik. Change my mind: Reddit is a social network and it’s the best one out there. Android Authority, 2021
40. M Myslín, SH Zhu, W Chapman, M Conway. Using twitter to examine smoking behavior and perceptions of emerging tobacco products. J Med Internet Res, 2013. [DOI | PubMed]
41. S Visweswaran, JB Colditz, P O’Halloran, NR Han, SB Taneja, J Welling, KH Chu, JE Sidani, BA Primack. Machine learning classifiers for Twitter surveillance of vaping: comparative machine learning study. J Med Internet Res, 2020. [DOI | PubMed]
42. K Janmohamed, AN Soale, L Forastiere, W Tang, Y Sha, J Demant, E Airoldi, N Kumar. Intersection of the web-based vaping narrative with COVID-19: topic modeling study. J Med Internet Res, 2020. [DOI | PubMed]
43. J Liu, C Wright, P Williams, O Elizarova, J Dahne, J Bian, Y Zhao, AS Tan. Smokers’ likelihood to engage with information and misinformation on twitter about the relative harms of e-cigarette use: results from a randomized controlled trial. JMIR Public Health Surveill, 2021. [DOI | PubMed]
44. L Chen, X Lu, J Yuan, J Luo, J Luo, Z Xie, D Li. A social media study on the associations of flavored electronic cigarettes with health symptoms: observational study. J Med Internet Res, 2020. [DOI | PubMed]
45. L Struik, Y Yang. e-Cigarette cessation: content analysis of a quit vaping community on Reddit. J Med Internet Res, 2021. [DOI | PubMed]
46. S Priya, R Sequeira, J Chandra, SK Dandapat. Where should one get news updates: Twitter or Reddit. Online Soc Netw Media, 2019. [DOI]
47. RV Dulam, M Murthy, J Luo. Seeing through the smoke: a world-wide comparative study of e-cigarette flavors, brands and markets using data from Reddit and Twitter. arXiv, 2020
48. E Kasson, AK Singh, M Huang, D Wu, P Cavazos-Rehg. Using a mixed methods approach to identify public perception of vaping risks and overall health outcomes on Twitter during the 2019 EVALI outbreak. Int J Med Inform, 2021. [DOI | PubMed]
49. GetOldTweets3 0.0.11. Python Package Index, 2019
50. R Chandrasekaran, V Mehta, T Valkunde, E Moustakas. Topics, trends, and sentiments of Tweets about the COVID-19 pandemic: temporal infoveillance study. J Med Internet Res, 2020. [DOI | PubMed]
51. Pushshift Reddit API v4.0 Documentation. Pushshift, 2018
52. E Cambria, D Das, S Bandyopadhyay, A Feraco. A Practical Guide to Sentiment Analysis, 2017
53. M Huang, O ElTayeby, M Zolnoori, L Yao. Public opinions toward diseases: infodemiological study on news media data. J Med Internet Res, 2018. [DOI | PubMed]
54. C Hutto, E Gilbert. Vader: a parsimonious rule-based model for sentiment analysis of social media text. Proceedings of the 8th International AAAI Conference on Weblogs and Social Media, 2014. [DOI]
55. J Turner, M Kantardzic, R Vickers-Smith. Infodemiological examination of personal and commercial Tweets about cannabidiol: term and sentiment analysis. J Med Internet Res, 2021. [DOI | PubMed]
56. K Krawczyk, T Chelkowski, DJ Laydon, S Mishra, D Xifara, B Gibert, S Flaxman, T Mellan, V Schwämmle, R Röttger, JT Hadsund, S Bhatt. Quantifying online news media coverage of the COVID-19 pandemic: text mining study and resource. J Med Internet Res, 2021. [DOI | PubMed]
57. S Qaiser, R Ali. Text mining: use of TF-IDF to examine the relevance of words to documents. Int J Comput Appl, 2018. [DOI]
58. BA Lienemann, JB Unger, TB Cruz, K Chu. Methods for coding tobacco-related Twitter data: a systematic review. J Med Internet Res, 2017. [DOI | PubMed]
59. S Elo, H Kyngäs. The qualitative content analysis process. J Adv Nurs, 2008. [DOI | PubMed]
60. J van der Tempel, A Noormohamed, R Schwartz, C Norman, M Malas, L Zawertailo. Vape, quit, tweet? Electronic cigarettes and smoking cessation on Twitter. Int J Public Health, 2016. [DOI | PubMed]
61. LS Martinez, S Hughes, ER Walsh-Buhi, MH Tsou. “Okay, we get it. You vape”: an analysis of geocoded content, context, and sentiment regarding e-cigarettes on Twitter. J Health Commun, 2018. [DOI | PubMed]
62. K McCausland, B Maycock, T Leaver, J Jancey. The messages presented in electronic cigarette-related social media promotions and discussion: scoping review. J Med Internet Res, 2019. [DOI | PubMed]
63. R Miech, ME Patrick, PM O’Malley, LD Johnston. What are kids vaping? Results from a national survey of US adolescents. Tob Control, 2017. [DOI | PubMed]
64. AJ Budney, JD Sargent, DC Lee. Vaping cannabis (marijuana): parallel concerns to e-cigs?. Addiction, 2015. [DOI | PubMed]
65. SJ Sowles, MJ Krauss, S Connolly, PA Cavazos-Rehg. A content analysis of vaping advertisements on Twitter, November 2014. Prev Chronic Dis, 2016. [DOI | PubMed]
66. ML McHugh. Interrater reliability: the kappa statistic. Biochem Med (Zagreb), 2012. [DOI | PubMed]
67. M Syed, SC Nelson. Guidelines for establishing reliability when coding narrative data. Emerg Adulthood, 2015. [DOI]
68. JL Lee, M DeCamp, M Dredze, MS Chisolm, ZD Berger. What are health-related users tweeting? A qualitative content analysis of health-related users and their messages on Twitter. J Med Internet Res, 2014. [DOI | PubMed]
69. C Orellana-Rodriguez, MT Keane. Attention to news and its dissemination on Twitter: a survey. Comput Sci Rev, 2018. [DOI]
70. J Wojcik, A Hughes. Sizing up Twitter Users. Pew Research Center, 2019
71. M Duggan, A Smith. 6% of Online Adults are reddit Users. Pew Research Center, 2013
72. Characteristics of Electronic Cigarette Use Among Middle and High School Students—United States, 2015. Centers for Disease Control and Prevention, 2016
73. S Assari, R Mistry, M Bazargan. Race, educational attainment, and e-cigarette use. J Med Res Innov, 2020. [DOI]