Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Free AccessOriginal Article

Exaggerated and Questioning Clickbait Headlines and Their Influence on Media Learning

Published Online:May 28, 2021https://doi.org/10.1027/1864-1105/a000298

Abstract

Abstract. Headlines that are incongruous with article content can negatively impact media learning outcomes. Clickbait headlines intentionally misrepresent news content, often in sensational ways to increase click-throughs and ad revenue. To evaluate the impact of clickbait headlines on media learning and article-related beliefs, we conducted two online experiments, each testing a 3 (headline-type: accurate, clickbait-question, clickbait-exaggerated) × 2 (exposure: headline-only, full article) factorial. In Study 1, an online sample of US adults (N = 629) was randomly assigned to one of six news message conditions. Study 2 (N = 1,674) was a replication study across three news contexts and testing a mediator to explain how exposure to a clickbait headline can influence learning. Key results suggest that reading the full article with an accurate headline resulted in the highest recognition and comprehension, and reading correcting information within an article is likely not enough to overcome the deleterious impact of a clickbait headline. Theoretical and practical recommendations are discussed.

Headlines that are incongruent with article can increase perceptions of scientific uncertainty and result in the formation of attitudes and beliefs that are inconsistent with article content (Ecker et al., 2014; Geer & Kahn, 1993; Pfau, 1995). Unfortunately, incongruous headlines are commonly utilized in online news coverage, a practice known as “click baiting.” Clickbait is colloquially defined as imprecise or sensationalist headlines to attract an audience. Most clickbait headlines are irksome but innocuous, however, there are at least two potentially problematic forms that warrant further research: (1) questioning headlines, which pose a question subsequently answered in the article (e.g., “Does green tea extract cause liver damage?”); and (2) exaggerated headlines, which overstate scientific findings through embellishment, hyperbole, or a lack of qualifying information, such as, “White wine has a scary link to skin cancer.” Clickbait headlines are one principal driver of fake news online, they are intentionally misleading and often not fully corrected within article content (Silverman, 2015). Indeed, even when incongruous headlines are corrected within the article, lingering effects of misinformation remain (Ecker et al., 2014). This is due in large part to the advance organizer effect, which posits that content presented in advance of a presentation (e.g., a headline or a news tease) provides an ideational scaffolding or a general structure that guides the retention of later information (Ausubel et al., 1978). The purposeful obfuscation or misrepresentation of news content can yield problematic outcomes in a variety of contexts, including, but not limited to science communication, where an incongruous headline may result in health-related beliefs that are unsupported by scientific findings; risk communication, where incongruous headlines lead to inaccurate risk perceptions; and of course, journalism, where clickbait headlines can have problematic effects on article understanding and publication reputation.

In the present investigations, two studies are conducted to identify and verify the influence of clickbait in multiple contexts, including environmental news, science news, and two different types of health news: cancer and HIV. Although incongruent headlines can impact attitude formation (Andrew, 2007), it seems plausible that clickbait headlines could have either beneficial or detrimental effects on both proximal and distal attitudes and beliefs that may inform risk perceptions and subsequent decision-making. For instance, a headline that dramatically overstates cancer risk may increase perceived susceptibility to cancer, ultimately leading to the adoption of prevention behaviors. Conversely, a dramatic overstatement of cancer risk may conflict with other news content, a known indicator of information overload that reduces the likelihood of taking preventive actions. Considering this, the overarching research questions for this study are as follows: (1) How do clickbait headlines influence understanding?; (2) As most readers spend more time reading headlines than articles (Dor, 2003), does exposure to only the headline or full article impact understanding?; and (3) Is there an interaction between headline-type and article exposure on understanding? We use the cognitive mediation model as a guiding theoretical framework to design and test a 3 (headline type: clickbait-questioning, clickbait-exaggerated, accurate) × 2 (article length: headline-only, full article) factorial design across two online experimental studies assessing how clickbait news content influences two indicators of media learning: recognition and comprehension. We also assess two secondary outcomes, perceptions of susceptibility and information overload, as well as elaboration as a potential mediator in Study 2.

Literature Review

The purpose of a headline is to summarize the main idea of an article, allowing readers to choose which articles to read among a large number of choices (Ecker et al., 2014). Effective headlines provide accurate article summaries while minimizing processing effort (Dor, 2003). Clickbait headlines are not accurate article summaries. While there is no scholarly consensus on a conceptual definition of clickbait, most agree that clickbait headlines are short, often sensationalist content that entices readers to click the article link (Potthast et al., 2016; Shire, 2014).

Clickbait headlines are often modified to appear more negative than the actual article, exaggerate main points, or over-emphasize conflicts to attract more readers (Ecker et al., 2014). This strategy has been used profitably for decades by publications like the National Enquirer, and more recently online through outlets like Buzzfeed, IFLScience, and Lifehacker, where journalists have a financial imperative to increase online readership, as advertising revenue is based explicitly on page views (Farhi, 2008). This financial pressure increases the likelihood that a website would distort the veracity of a headline to increase ad revenue. When journalists, bloggers, and other web content creators misrepresent message content, it can dramatically influence learning outcomes of message exposure.

Cognitive Mediation Model

The cognitive mediation model describes the process through which media learning occurs (Ho et al., 2013; Jensen, 2011). The model proposes that we learn from news through a process known as surveillance motivation, or an intrinsic ambition to learn, which happens through increased attention to news and elaboration on content (Eveland, 2001). One obvious problem of clickbait headlines arises for those who read only the headline, making it impossible to elaborate on any correcting information within the article. Since most readers spend more time reading headlines than full articles (Dor, 2003), they may not read through article content to correct exaggerated headlines, and even if they did, incongruent headlines hold lingering effects. As an example, a recent investigation explored how altering headlines to focus on secondary content stated within the article rather than the overall thrust of the article impacted message recall, finding that participants exposed to an article with an incongruent headline were less likely to remember key facts about that article than those exposed to an article with an accurate headline (Ecker et al., 2014). In short, clickbait headlines can present distorted versions of the truth that limit one’s ability to process and contextualize a news story. In the present study, attention is manipulated through exposure to the headline-only (low attention) or exposure to the full article (high attention).

The principal outcome variable of the cognitive mediation model is learning. In past studies, this has been commonly operationalized as knowledge (e.g., Eveland, 2001). However, measures of knowledge may be inappropriate in the context of clickbait headlines, where the truth is often stretched or distorted. Other cognitive mediation model studies have operationalized learning through measures of recognition and comprehension (e.g., Jensen, 2011). These measures may be more appropriate in the context of clickbait, as they distinguish between being able to recall article content (recognition) and applying that content to make accurate attributions in other contexts (comprehension). In traditional journalistic practices, headlines and article content are congruent, meaning that both recognition and comprehension can be achieved solely through engaging with the article. However, in clickbait articles, headline and article content are incongruent, where headline recognition may lead to faulty comprehension, as headline content informs processing over any correcting information found in the article (Van Dijk, 1988).

Considering the uncertainty, confusion, and misinformation caused by clickbait headlines, the following hypotheses are proposed:

Hypothesis 1 (H1):

Clickbait headline framing (exaggeration, questioning) will yield lower scores on recognition (H1a) and comprehension (H1b) than reading accurate headline framing.

Hypothesis 2 (H2):

Reading only the headline will yield lower scores on recognition (H2a) and comprehension (H2b) than reading the entire article.

Hypothesis 3 (H3):

The effect of clickbait headlines on recognition (H3a) and comprehension (H3b) will be stronger when participants only read the headline as opposed to the full article.

Clickbait Headlines and Health-Related Beliefs

Two types of clickbait that can have deleterious outcomes for disease prevention and screening are posing questions and exaggeration. Questioning headlines are perceived as confusing and less informative (Kuiken et al., 2017). For instance, “Is mammography the best screening test to identify breast cancer?” can elicit doubt, regardless of the answer to this question. Exaggerated headlines can be similarly problematic. They often contain provocative or misleading words that overstate scientific findings and elicit confusion, anxiety, and in some cases medical non-adherence (Biyani et al., 2016; Brunt et al., 2003; Schwartz & Woloshin, 2003). Specifically, exaggeration may contribute to one of two outcomes: (1) a gross overestimation of risk leading to risk perceptions in excess of actual risk probabilities; or perhaps worse, (2) disbelief in the sheer extremity of the exaggeration that leads one to reappraise their beliefs and reduce risk perceptions (Adams et al., 2017). The first case, although yielding potentially inflated risk perceptions, may be helpful as exaggeration can increase perceptions of susceptibility, which is associated with the adoption of preventive health behaviors (Witte & Allen, 2000). In contrast, the second case demonstrates the negative effects of exaggeration, where exaggerated findings may reduce risk perceptions through disbelief.

Exaggerated headlines may also increase perceptions of information overload, especially if the exaggerated claim conflicts with existing cognitive representations. For instance, it is not uncommon to see news articles portraying either the cancer-preventive or cancer-causing influence of red wine. Headlines exaggerating the connection between a common food and cancer incidence or cancer prevention may increase perceptions of information overload by eliciting confusion and uncertainty.

Given that clickbait may have either positive or negative impacts on the enactment of cancer prevention outcomes, the following research questions are advanced:

Research Question 1 (RQ1):

Are there relationships between headline framing and message length on perceptions of disease susceptibility?

Research Question 2 (RQ2):

Are there relationships between headline framing and message length on perceptions of information overload?

Study 1

Method

A 3 × 2 factorial experiment with random assignment to condition was employed to assess the impact of headline framing (accurate, questioning, exaggerated) and message length (headline-only, full article) on perceptions of disease susceptibility and information overload as well as two dimensions of knowledge: recognition and comprehension. Specifically, we focus on whether and how cancer-related clickbait headlines affect cancer information overload.

Participants and Procedure

Eligibility criteria were restricted to US residents 18 years or older who read online news. Participants (N = 630) were recruited using Qualtrics’ panel service, weighting responses on US census data for age and race/ethnicity. On average, participants were 46.57 years (SD = 16.32; range 18–85), female (77%), and Caucasian/White (62%). Other races/ethnicities included in the sample were Latino/Hispanic (16.8%), African American/Black (13.2%), Asian/Pacific Islander (5.4%), Native American/American Indian (0.6%), and those marked “other” (1.1%).

Data were collected using Qualtrics’ panel service, employing a variety of marketing survey research outlets to reach a broad cross-section of the US online population. Eligible participants were directed to the survey where they completed informed consent, were randomized to one of the six study conditions, read through the stimulus message, and completed a posttest survey measuring attitudes, beliefs, as well as knowledge measures about the article.

Stimuli

Six intervention conditions were created from an original article published on the Cosmopolitan magazine website (Narins, 2016). The article was shared over 400 times directly from the Cosmopolitan website, likely more when taking into account secondary shares, including Facebook shares, tweets, and retweets. The article features a clickbait headline about the relationship between white wine and skin cancer: “White wine has a scary link to skin cancer.” In truth, the research findings upon which this article is based report a modest increase in skin cancer incidence among those who drink more white wine than other types of spirits (Rivera et al., 2016). The original article headline was used as the exaggeration condition headline. We slightly edited the article content to provide an accurate summary of the original research article, which was omitted from the news story. Headlines for the questioning and accurate condition were modified from this headline (see online supplementary content for the stimulus message at https://drive.google.com/file/d/137z3gMTMgTcUwVuK30PpF036P3jwcL8Y/view?usp=sharing). Article length was manipulated by showing participants the headline-only or the full article.

Measures

Four dependent variables were measured in this study: recognition, comprehension, information overload, and perceived susceptibility. Recognition and comprehension items were modified from a previous study (Jensen, 2011). Recognition items measured rote memorization and comprehension items measured participants’ ability to apply knowledge from the article to another situation. An index was created for both variables by summing correct responses. Both indexes contained three questions measuring knowledge about the article, each with four response options, one of which was correct. Scores on both indexes were on a 4-point scale ranging from 0 to 3, representing zero correct answers through 3 correct answers (Mrec = 1.68; SDrec = 1.10; Mcomp = 1.60; SDcomp = 0.98). An example of a recognition item is: “In general, drinking any type of alcohol is associated with a _______ increase in skin cancer risk compared to non-drinkers.” An example of a comprehension question is: “David drinks a glass of white wine every day with dinner. David is at __________ risk of getting skin cancer compared to people who drink other types of alcohol.” Cancer information overload was conceptualized as feeling overwhelmed by the sheer amount of cancer information and was taken from an 8-item, reliable, and validated scale with seven response options ranging from strongly disagree to strongly agree (Jensen et al., 2014; M = 4.02, SD = 1.14; α = .86). A sample item from the cancer information overload scale is: “There are so many different recommendations about preventing cancer, it’s hard to know which ones to follow.” Perceived susceptibility to skin cancer was modified to ask specifically about perceived susceptibility related to one’s own white wine consumption and was measured by modifying a 3-item, previously validated scale measured on a scale with seven response options ranging from 1 = strongly disagree to 7 = strongly agree (Witte, 1996; M = 2.21, SD = 1.55; α = .93). A sample item is: “I am at risk for getting skin cancer because I drink a lot of white wine.”

Results

Data were analyzed using two-way analysis of variance (ANOVA) in SPSS. Headline framing and message length were entered as the independent variables. Four models were specified, one for each dependent variable including recognition, comprehension, information overload, and susceptibility. H1a, H2a, and H3a were tested within the same model looking at the main effects of headline framing (H1a) and message length (H2a), as well as their interaction (H3a) on recognition. There was a significant main effect of message length on recognition, F(1, 624) = 30.39; p < .001, η2 = .05, such that those who read the entire article had higher recognition scores, in support of H2a. Although the main effect for headline framing was nonsignificant, F(2, 624) = 1.47; p = .230, η2 = .01, this finding was qualified by a significant interaction effect between headline framing and message length on recognition, F(2, 624) = 7.44; p = .001, η2 = .02. Specifically, the question/headline-only condition generated significantly lower recognition scores than the question/full article condition (Mdiff = −0.67, SE = .15, p < .001). That is, the negative impact of questioning headlines was stronger when participants only read the headlines than the full article. Interestingly, this effect was not observed among exaggerated headlines; the exaggeration/headline-only condition generated similar recognition scores to the exaggeration/full article condition (Mdiff = −0.01, SE = .15, p = .942). Additionally, the accurate/full article condition generated significantly higher recognition scores than the accurate/headline-only condition (Mdiff = 0.73, SE = .15, p < .001), and the accurate/full article condition resulted in the highest levels of recognition. In other words, when the headlines were accurate or in questioning-clickbait formats, the effect was stronger when participants only read the headlines. The effect of exaggerated headlines did not differ by message length. Taken together, H3a was partially supported. All conditional means and standard deviations can be seen in Table 1.

Table 1 Conditional means for all dependent variables (DV) in Study 1

H1b, H2b, and H3b were tested within the same model looking at the main effects of headline framing (H1b) and message length (H2b), as well as their interaction (H3b) on comprehension. There was a significant main effect of message length on comprehension, F(1, 623) = 23.94; p < .001, η2 = .04, such that those who read the entire article had higher comprehension scores, in support of H2b. The main effect for headline framing was marginally significant, F(2, 623) = 2.79; p = .06, η2 = .01. Post hoc comparisons were assessed using Tukey’s HSD test, revealing significant mean differences between the exaggerated and questioning headline conditions, Mdiff = 0.22, SE = 0.09, p = .04, and a marginally significant difference between the accurate and questioning headline conditions, Mdiff = 0.21, SE = 0.10, p = .07. These results demonstrate that the questioning headline generally resulted in the lowest comprehension. However, as H1b hypothesized that the accurate framing condition would result in the highest comprehension, this hypothesis was not supported. There was not a significant interaction effect, F(2, 623) = 0.06; p = .94, η2 = .00.

Additionally, we explored contrasts in ANOVA to compare the mean differences between (a) accurate headline condition versus the clickbait headline conditions and (b) the question headline versus the exaggerated headline among those who were assigned to read the full article. This analysis investigated whether correcting information within the article could ameliorate the negative influence of clickbait headlines. In the first contrast, the clickbait headlines elicited significantly lower recognition scores than the accurate headline, t(310) = 2.63, p = .009, which further supports H3a. No significant differences between exaggerated and questioning clickbait headlines was found, t(310) = 1.43, p = .15. Clickbait and accurate headlines elicited statistically similar comprehension scores, t(310) = 0.40, p = .69, hence H3b was not supported. There was no significant difference between the two types of clickbait headlines on comprehension, t(310) = −1.61, p = .11. Taken together, the contrasting results indicate that those who read the full article with clickbait headlines had lower recognition and but similar comprehension scores as those who read an accurate headline.

RQ1 asked about the interaction between headline framing and message length on perceived disease susceptibility. There was no significant interaction effect observed, F(2, 624) = 1.02; p = .361, η2 = .00, and no main effect for headline framing, F(2, 624) = 0.77; p = .92, η2 = .00, or for message length, F(1, 624) = 1.23; p = .27, η2 = .00. Thus, no support for RQ2 was found.

The second research question asked about the interaction between headline framing and message length on perceptions of information overload. There was no significant interaction effect observed, F(2, 624) = 1.28; p = .28, η2 = .00, and no main effect for headline framing, F(2, 624) = 1.59; p = .20, η2 = .01. However, there was a significant main effect for message length, F(1, 624) = 5.53; p = .02, η2 = .01. In general, information overload scores were higher when participants read only the headline and not the entire message.

Discussion

There was a significant interaction between headline framing and message length on recognition such that recognition was highest among people who read the accurate headline and the full article, as predicted. Of note, among those who read the exaggerated headline, message length proved inconsequential. That is, even those who read the full article had virtually the same recognition scores as those who read only the headline. It appears that clickbait headlines employing exaggeration tactics leave such an effect on readers that even corrective information in the text cannot change initial impressions. The findings for comprehension were similar, but the interaction effect was only significantly different between the exaggerated and questioning clickbait conditions. In short, the lowest levels of comprehension occurred with exposure to the questioning headline without reading the full article. Although the effect size was small, it can nevertheless still be important (Durlak, 2009; Lakens, 2013); Cosmopolitan is published in 34 languages and more than 100 countries, making it one of the most widely recognized and distributed brands globally (Hearst, 2013), and online articles serve only to broaden that reach.

Concerning beliefs, message length was a significant predictor of information overload, where overload was higher among people who only read the headline. This has troubling practical implications for the saturated online media environment. It seems that people who encounter headlines about cancer news, regardless of clickbait strategies or headline congruency, may experience higher perceptions of information overload. There were no significant main effects or interaction effects witnessed for susceptibility. Taken together, it appears that clickbait headlines themselves may not influence protective or avoidance motivations, at least in the context of the two beliefs measured in the present study: information overload and susceptibility.

Study 2

The findings from Study 1 are compelling but preliminary and limited in scope, focusing only on a cancer-related news article. To increase certainty in these findings and provide a broader argument for external validity, we propose a second study to assess the impact of clickbait headlines on susceptibility, recognition, and comprehension across three additional news contexts: science news, health news, and environmental news. In this study, we attempt to replicate Hypotheses 1–3 and RQ1 in Study 1. RQ2 is not relevant, as these news stories are not related to cancer.

An additional research question is posed to assess the process through which clickbait headlines impact news learning outcomes. The cognitive mediation model suggests two pathways to media learning: attention and elaboration. In each of the present studies, attention is manipulated through either exposure to a headline-only or the full article. The present study will assess elaboration on message content as a mediator of media learning outcomes, as the cognitive mediation model posits that increased elaboration facilitates media learning (Eveland, 2001). As such, the following research question is proposed:

Research Question 3 (RQ3):

Will elaboration mediate the relationship between the Headline × Length interaction and (RQ3a) recognition, (RQ3b) comprehension, and (RQ3c) susceptibility?

Method

A similar design to Study 1 was employed, a 3 × 2 factorial experiment with random assignment to condition to assess the impact of headline framing (accurate, questioning, exaggerated) and message length (headline-only, full article) on perceptions of susceptibility and two dimensions of knowledge: recognition and comprehension. Three different news contexts (articles on science, health, and the environment) were used to assess the effect of clickbait headlines across various news contexts. Message context was entered as a covariate for all analyses. This study was preregistered prior to data collection on the Open Science Foundation website (see https://osf.io/5fev8).

Participants and Procedure

Eligibility criteria were restricted to US residents 18 years or older. Participants (N = 1,674) were recruited through Amazon’s Mechanical Turk (mTurk). Although multiple investigations have demonstrated the utility and representativeness of mTurk samples (Berinsky et al., 2012; Casler et al., 2013; Hauser & Schwarz, 2016), there are nevertheless concerns when collecting data online that should be addressed to ensure data fidelity. To address these concerns, we took multiple measures. In total, we collected 2,094 responses. A pretest survey was conducted to assess eligibility criteria (ages 18+). Eighty-seven people were excluded from proceeding for failing to meet this age requirement. Next, to make sure people were investing adequate effort, we removed 59 cases from analysis for a response time that was less than ⅓ of the median response time. We found no evidence of straight-liners or patterned responses. One hundred thirty cases were removed for failing the attention check item, and an additional 133 cases were removed for failing to provide an adequate answer for what the article/headline was about. Eleven cases were removed as univariate outliers. A total of 1,674 cases were used for data analysis. On average, participants were 40.16 years (SD = 13.06; range 18–83), female (54%; n = 920), and White (76.3%; n = 1278). Other races/ethnicities included in the sample were Latino/Hispanic (5.3%), African American/Black (9.6%), Asian/Pacific Islander (6.8%), Native American/American Indian (0.8%), and those who marked “other” (1.3%).

Stimuli

Three news articles were collected from Time Magazine, Huffington Post, and The Independent published between 2015 and 2019. Three different news contexts were chosen: (1) an environmental news article reporting on pollution in London and its health effects; (2) a science news article reporting on findings in psychology linking bitter taste preferences to antisocial personality traits; and (3) a health news article reporting on a recent study eliminating HIV in a test group of mice. The original article headline for each article was maintained as the exaggeration condition headline. We also generated two additional headlines (a questioning headline and an accurate headline) based on the article context to test the headline framing effect. The headlines used in the environmental news article were:

  • •
    “London air pollution cancels positive health effects of exercise in those over 60” [Accurate headline];
  • •
    “Pollution wipes out benefits of exercise, study suggests” [Exaggeration headline];
  • •
    “Does air pollution negate the health benefits of a long walk?” [Question headline].”

The exaggeration headline overstates scientific findings by suggesting that pollution counteracts the benefits of all exercise for all people, while the study found that pollution impacts elderly people who take long walks outside. The headlines used in the science news article were:

  • •
    “A recent study found bitter taste preferences as a predictor for Machiavellianism, psychopathy, narcissism, and everyday sadism” [Accurate headline];
  • •
    “How you drink your coffee could point to psychopathic tendencies” [Exaggeration headline];
  • •
    “Are bitter taste preferences linked to antisocial personality traits?” [Question headline].”

The exaggeration headline is a clear embellishment as the study found no significant link between any specific bitter food, such as coffee, beer, or grapefruit, and antisocial personality traits. The headlines used in the health news article were:

  • •
    “Researchers eliminated HIV in 30% of test mice” [Accurate headline];
  • •
    “For the first time, researchers eliminated HIV from the genomes of living animals” [Exaggeration headline];
  • •
    “Did researchers just eliminate HIV from the genomes of living animals?” [Question headline].”

The exaggeration headline overstates scientific findings by neglecting to report this finding was for a small percentage of a test case in mouse models. Message length was manipulated as headline-only or full article.

Measures

Three dependent variables were measured in Study 2: recognition, comprehension, and perceived susceptibility. Recognition and comprehension items were operationalized using three true or false questions for each construct. A sample recognition item is “Exposure to air pollution on city streets is enough to wipe out the benefits of exercise for most people.” A sample comprehension item is “David enjoys bitter flavors and his brother Charles prefers sweets. David is more likely to display antisocial personality traits than Charles.” Response options were true, false, and I do not know. Participants were given an I do not know option to reduce the likelihood of selecting the correct answer by chance. All responses of I do not know were coded as incorrect. An index was created for both variables by summing correct responses ranging from 0 to 3 (Mrec = 1.61; SDrec = 1.03; Mcomp = 1.49; SDcomp = 1.00). Similar to Study 1, perceived susceptibility was modified for each of the three context conditions (α = .92), such as “My health is at risk because I [drink a lot of coffee/spend a lot of time outdoors].” Elaboration is conceptually defined as the number of unique thoughts that arise after exposure to the message. Following previous research, we operationally defined elaboration using 20 text entry boxes and asked participants to list any thoughts they had about the news article, limiting each entry to one thought (Cacioppo et al., 1979; M = 8.26, SD = 6.23). Need for cognition represents one’s tendency to engage in and enjoy thinking, operationally defined using Cacioppo and colleagues (1984) 18-item scale (α = .86). The need for cognition was included as a covariate to account for any individual differences.

Results

H1, H2, and H3 addressed the main effect of headline framing (H1), the main effect of message length (H2), and their interaction effect (H3) on recognition (a) and comprehension (b). Two analyses of covariance (ANCOVAs) were conducted with recognition and comprehension as the dependent variables in each. Need for cognition and message context were included as planned covariates in our preregistered study. However, the need for cognition was not significantly related to any outcome variable and thus was dropped from all analyses. The results revealed a significant difference between headline types among recognition, F(2, 1,668) = 32.64, p < .001, η2 = .04, where participants who were assigned to accurate condition had significantly higher recognition scores than those who were assigned to questioning (Mdiff = 0.42, SE = 0.06, p < .001) and exaggeration conditions (Mdiff = 0.37, SE = 0.06, p < .001). All conditional means and standard deviations can be seen in Table 2. Thus, H1a was supported. There was also a significant main effect for length, F(1, 1,668) = 170.10, p < .001, η2 = .09, where those who read the full article reported higher recognition scores, in support of H2a. H3a specified that the effect of clickbait headlines on recognition will be stronger when participants only read the headline as opposed to the full article. The interaction effect between headline type and length was significant, F(2, 1,668) = 6.36, p = .002, η2 = .01. A look at the simple effects confirm that recognition was highest in the full article/accurate headline condition compared to the full article/question condition (Mdiff = 0.26, SE = 0.08, p = .002), full article/exaggeration condition (Mdiff = 0.19, SE = 0.08, p = .024), and headline-only/accurate condition (Mdiff = 0.38, SE = 0.08, p < .001). The lowest recognition scores were found in the headline-only exaggeration (M = 1.13, SD = 0.06) and question (M = 1.10, SD = 0.06) conditions. These findings lend support for H3a.

Table 2 Conditional means for all dependent variables (DV) in Study 2

Consistent with H1b, accurate headlines also had a significant effect on comprehension, F(2, 1,668) = 29.97, p < .001, η2 = .04, where participants who were assigned to accurate condition had significantly higher comprehension scores than those who were assigned to the questioning (Mdiff = 0.36, SE = 0.06, p < .001) and exaggeration conditions (Mdiff = 0.39, SE = 0.06, p < .001). Thus, H1b was supported. There was also a significant main effect for headline length, F(1, 1,668) = 122.19, p < .001, η2 = .07, where those who read the full article reported higher comprehension scores (Mdiff = 0.51, SE = 0.05, p < .001), in support of H2b. H3b specified that the effect of clickbait headlines on comprehension will be stronger when participants only read the headline as opposed to the full article. The interaction effect between headline type and length was significant, F(2, 1,668) = 11.34, p < .001, η2 = .01. A look at the simple effects confirmed that although comprehension was highest in the full article/accurate headline condition, it was statistically equivalent to the full-article/question condition (Mdiff = 0.13, SE = 0.08, p = .12). However, the full article/accurate headline condition was significantly more effective than the full-article/exaggeration condition (Mdiff = 0.17, SE = 0.08, p = .042), and all headline-only messages (e.g., headline-only/accurate condition: Mdiff = 0.20, SE = 0.08, p < .012). The lowest comprehension scores were found in the headline-only exaggeration (M = 1.02, SD = 0.06) and question (M = 1.04, SD = 0.06) conditions. These findings lend partial support for H3b.

To further probe this relationship, we conducted contrasts in ANOVA to compare only the mean differences in recognition and comprehension among those who were assigned to read the full article for (a) the accurate headline condition versus the clickbait headline conditions, as well as (b) the question headline versus the exaggerated headline. These contracts were not included in our preregistration. This analysis explored whether correcting information within the article could ameliorate the negative influences of clickbait headlines. In the first contrast, participants in the clickbait headline conditions reported significantly lower recognition than the accurate headline condition, t(820) = 3.20, p = .001, while no significant differences were observed between exaggerated and questioning clickbait headlines, t(820) = −0.84, p = .402. This pattern was also observed for comprehension scores. Clickbait headlines elicited lower comprehension scores than the accurate headline, t(820) = 2.11, p = .035, and there was no difference between the two clickbait headline conditions, t(820) = 0.51, p = .614. Taken together, the results indicate that when reading the full article, clickbait headlines generally reduce message recognition and comprehension compared to the accurate headline. This finding lends support for H3a and H3b.

RQ1 examined the interaction effect between headline framing and message length on perceived susceptibility. Two-way ANCOVA was conducted with perceived susceptibility as the outcome and need for cognition and message context as covariates. There was no significant main effects for headline framing, F(2, 1,668) = 1.87, p = .154, η2 = .00, message length, F(1, 1,668) = 0.69, p = .407, η2 = .00, and the interaction effect was not significant, F(2, 1,668) = 0.38, p = .682, η2 = .00. Thus, no evidence was found in support of RQ1.

RQ3 asked if elaboration would mediate the relationship between the Headline × Length interaction and (RQ3a) recognition, (RQ3b) comprehension, and (RQ3c) susceptibility. To address this Hayes’ (2018) PROCESS (v3.4) custom dialog for SPSS was utilized specifying a linear regression equation with headline type entered as the primary predictor variable, length as the moderator, elaboration specified as a mediator of the interaction effect of headline Type × Length, and recognition, comprehension, and susceptibility as outcome variables in three separate analyses. Bootstrapping procedures were utilized to estimate a 95% confidence interval around the mediation coefficient with 10,000 resamples with replacement. There was no relationship between predictor variables and elaboration, no relationship between elaboration and any outcome (recognition, comprehension, susceptibility), and no evidence of moderated mediation in any analysis, Thus, we found no evidence to substantiate RQ3.

Discussion

In general, a similar pattern of results found in Study 1 was seen in Study 2 across three different message contexts: exposure to the accurate headline and reading the full article resulted in the most recognition and comprehension. Similar to Study 1, no relationship was observed between exposure to clickbait headlines and susceptibility. However, two important distinctions emerged: (1) when exposed to the full article, comprehension was higher across headline types than recognition, and (2) an investigation of simple effects revealed that among those who read the full article, recognition, and comprehension were significantly lower for those that read a clickbait headline than for those who read the accurate headline. These findings have important theoretical implications, addressed shortly in the general discussion. No evidence was found to support elaboration as a mediator in this process.

General Discussion

The present experimental study was conducted to understand the effect of clickbait headlines on learning outcomes in three different news contexts: science news, environmental news, and two different types of health news: cancer and HIV. These findings support previous research stating that headlines inconsistent with article content can result in reduced understanding (Ecker et al., 2014) and have multiple salient practical and theoretical implications. First, these results highlight the importance of message length on understanding. We live in an increasingly saturated digital information environment. News content is available and consumed across a variety of digital platforms, including news aggregator apps such as Feedly and Flipboard, link aggregator sites like Reddit, and social media sites including Facebook and Twitter. Exposure to hundreds of news headlines a day while using these services is not uncommon. Problems may arise when clickbait headlines are encountered; people who read just the clickbait headline likely believe it offers an adequate summary of the article content, and may consequently believe they understand article content. The results provided here suggest they generally have lower recognition and comprehension scores than those who receive accurate headlines.

These findings also suggest that clickbait journalistic practices have a tangible effect on consumers’ ability to understand news content, which is critical for making informed decisions in a variety of contexts. Reading only clickbait headlines resulted in reduced ability to recognize the correct focus of a news article (recognition) and also the ability to apply the information in the article to a real-world setting (comprehension). This failure of understanding may contribute to decision-making that has no grounding in scientific evidence. In line with this, Study 1 found that reading only a clickbait headline on cancer news can increase cancer information overload, which is associated with reduced attentiveness to health news, lower trust in health news, and reduced likelihood of adopting preventive health behaviors (Gurmankin & Viswanath, 2005; Han et al., 2007; Jensen et al., 2014; Kim et al., 2007; Niederdeppe & Gurmankin Levy, 2007). Further, in Study 2, we found that even among those who read the full article, recognition and comprehension scores were lower for those who received a clickbait headline than those who received an accurate headline. This suggests that reading the full article may not fully ameliorate the negative effects of clickbait. Recent work has identified a backfire effect of exposure to some clickbait headlines, finding that headlines presented in a questioning format negatively impact information seeking (Scacco & Muddiman, 2020). Coupled with our findings, this suggests that exposure to questioning headlines may reduce comprehension compared to accurate headlines and may reduce motivation to seek out further information on the topic.

There are also relevant theoretical applications. First, this investigation takes a novel approach in assessing the cognitive mediation model. Clickbait headlines do not conform to traditional models of media learning, which presuppose veracity of content and an implied relationship between recognition and comprehension where the ability to recall article content yields accurate comprehension of an issue. Instead, they introduce confusion and stretch the truth, reducing the likelihood that any correcting information found within the article is internalized. Counter to expectations results in Study 2 revealed that exposure to the full article led to higher comprehension scores than recognition scores. In opposition to this, much literature on learning posits accurate recognition as a precursor to comprehension (Gillund & Shiffrin, 1984). This suggests an alternative, unmeasured constructs may be influencing learning outcomes beyond the variables included in the cognitive mediation model. For instance, constructs such as prior clickbait experience, media literacy, and skepticism may all impact accurate comprehension in response to clickbait headlines. Given this, it is likely that the mechanism of effect that explains the process through which news media influences learning outcomes is different for clickbait content than for more traditional and truthful media types.

Future research should build on this model to test mediational and moderation effects of clickbait headlines on knowledge, attitudes, beliefs, and behavior across a variety of communication contexts. One important next step in this research is to identify the specific process of influence through which exposure to incongruent headlines influences news-relevant outcomes. The current study provides compelling evidence that clickbait headlines have a distinct influence on resultant knowledge and beliefs, but the exact mechanism remains unclear as our operationalization of elaboration did function as a mediator. Future research in this area should consider a valenced operationalization that looks at the extent of agreement or disagreement with article content; perhaps elaboration can function as a mediator if research can isolate whether people were agreeing with the message or counter-arguing against it.

Limitations and Future Research

One limitation of the present investigation is that the experiment was conducted online and can only represent members of the population who read and consume news digitally. Future research should take print media under consideration and focus on validating the findings of the present investigation in alternative contexts. One area of improvement for future research is a measurement of the dependent variable. In our study, reading the entire article resulted in a modest increase in recognition, suggesting either the questions themselves or the multiple-choice response options provided were too simple. Future research should explore different types of measurement for recognition and comprehension, including writing more difficult items and considering open response options for answers. In addition, it would be worthwhile to study these relationships using a random-effects model to test multiple and varied types of clickbait headlines to see the extent to which these findings hold up across variations in headline style and writing. Another limitation was that in Study 1, quota-sampling was used to result in a sample nationally representative (in the US) on age and race, but not gender, resulting in a 77% female sample. Finally, future research would benefit from exploring more distal outcomes of media learning, including how knowledge and beliefs translate to behavioral intentions and ultimately, behavior.

Conclusion

This article assessed the effect of clickbait headlines or those that are incongruent with article content, on media learning. Results suggest that clickbait headlines can reduce recognition and comprehension, and this can endure despite reading corrective content within the article. Study 2 revealed that exposure to the full article, as opposed to only the headline, unexpectedly led to higher comprehension scores than recognition scores. This suggests alternative constructs may be influencing learning outcomes beyond the variables included in the cognitive mediation model. Perhaps the mechanism of effect through which news media influences learning outcomes is different for clickbait content than for more traditional media types, a notion that should be prioritized in future theoretical work in this area.

Nick Carcioppolo (PhD) is an Associate Professor of Communication Studies and Affiliate Faculty at the Sylvester Comprehensive Cancer Center at the University of Miami. His research primarily focuses on cancer communication interventions for primary and secondary cancer prevention.

Di Lun is a U-LINK predoctoral research fellow and a PhD candidate in the School of Communication at University of Miami. Her research involves the development and assessment of both interactive and non-interactive educational materials for health communication campaigns and interventions with an emphasis on cancer prevention.

Soroya Julian McFarlane (PhD) is an Assistant Professor in the Department of Communication Studies at the University of Georgia. Her research focuses on designing and evaluating communication interventions that address health disparities at the community level. Research interests: health disparities, message design, community-engaged research; campaigns and interventions – design and evaluation; clinical trials; new biomedical preventive technologies.

References