EASP Seedcorn Grant Report by Alexandra Lux¹, Julia Schnep², Zixi Jin³, and Magdalena Formanowicz⁴
07.09.2021, by Tina Keil in grant report
¹ University of Leuven, Belgium; ² University of Koblenz-Landau, Germany; ³ Cardiff University, UK; ⁴ SWPS University of Social Sciences and Humanities, Poland
The influence of text complexity on trust and compliance following health-related messages
It is the nature of science to explore objects and the relationships between objects, and thus to come closer to what is commonly called “reality”. However, reality does not claim to be simple at all. Rather, the relationships between research objects are often complex and cannot be expressed by simple laws. For many people, this makes science per se difficult to understand. In addition, from a historical perspective, science has been set apart from the general public and access to science has been reserved for privileged groups across the globe for centuries. This elitism has also been ingrained in academic language. The movement that science is a common good which should be accessible to all citizens is relatively young (Vicente-Saez & Martinez-Fuentes, 2018). In psychology, for example, this demand is significantly driven by the current open science movement and also by the fact that more and more research is published through open access journals (Samarrai, 2013).
However, a key variable for the public dissemination of scientific findings is the language in which they are expressed. Scientific language typically exceeds the complexity level of everyday language and thus makes science inaccessible to many lay people (e.g., Plavén-Sigray et al., 2017; Zabal et al., 2013). In many cases, this cannot be avoided, as scientific jargon follows its own laws and is often formally necessary to precisely and correctly express scientific results (e.g., Lee et al., 2016). However, with technological change, our society is now more than ever linked to and dependent on science. Our everyday decisions highly rely on scientific recommendations, for example when deciding whether to consume genetically modified food, what medical treatment to receive, or what means of transportation to choose considering its impact on the environment. Governments around the world seek the advice of scientists for all areas of life. The current COVID-19 pandemic proved the need for scientific recommendations in our daily lives even more apparent (Malecki et al., 2020; Wu et al., 2020). In this context, the success of broad science communication has gained a lot of importance. Given the exponential spread of the virus, civic compliance with scientifically recommended preventive behaviors has been an imperative for curbing the infection curve in most democratic countries. Against this background, the goal of this project was to examine the influence of the linguistic complexity of scientific statements on the effectiveness of mask-wearing on trust in the statement, in science in general, on behavioral adherence, and the dissemination of scientific findings.
Our research was built on existing findings in the field of social psychology which indicate that information which is difficult (vs. easy) to process is more likely to be distrusted (Reber & Schwarz, 1999; Unkelbach & Greifeneder, 2018). The trust advantage of simple information has also been demonstrated in several other research domains: from higher trust in players with more readable names in economic games (Zürn & Topolinski, 2017) to higher trust in privacy statements of websites with higher (vs. lower) readability (Ermakova et al., 2014). Transferred to the context of scientific information on mask-wearing, we thus hypothesized that complex (vs. simple) scientific statements decrease readers’ trust in the message, in science in general, and their behavioral adherence to the message. In addition to that, we were also interested in the underlying mechanism, namely whether difficult (vs. simple) texts could cause feelings of social exclusion, which might then result in lower trust and adherence related to the message.
To test our hypotheses, we first pretested our materials using Cardiff University’s online testing system (N = 107), to ensure that complex and simple texts indeed were perceived as such. Next we conducted a large-scale study with 630 participants from the U.S. at the beginning of January 2021 (using Amazon MTurk). In the main study, we presented participants with either a complex or easy-to-read text about the effectiveness of mask-wearing. Due to dropout, the final data set consisted of 605 participants. Participants were first presented with either a linguistically complex or simple version of a scientific statement on the effectiveness of mask-wearing to prevent the spread of the COVID-19 virus. They were then asked to complete measures of our dependent variables, namely trust in the message, trust in science, and behavioral adherence. Following the measures of the dependent variables, we assessed the mediator, namely feelings of social exclusion, and for an explorative test of potential moderating effects an additional measure of participants’ conspiracy mentality.
The results of our main study did not support our expected main effect that the linguistic complex statement version decreased participants’ trust in the message, general trust in science, and behavioral adherence compared to the simpler version. However, we identified an interesting pattern of a moderated mediation, namely that participants with a strong conspiracy mentality reacted with higher feelings of social exclusion to the complex text and as a result indicated lower levels of trust in the message and in science as well as lower behavioral adherence to the statement. Conversely, we found no effect of a complex versus simple text for people with a medium or low conspiracy mentality. This is an important finding, as it shows that trust in scientific statements can be increased by linguistic simplification for one group without limiting it for the other.
In addition to the experimental main study, we conducted a literature study to test whether the linguistic complexity of abstracts of scientific articles on the effectiveness of mask-wearing has an impact on their dissemination in the media (as measured by different Altmetric scores). We used the Web of Science Core Collection database for the collection of abstract data. Following guidelines for the conduction of systematic literature studies (Moher et al., 2009), we based our data collection on a keyword search to identify related articles that addressed the actual effectiveness of face masks, such as technical studies on different coverings’ reduction of aerosols, or studies on the impact of obligatory mask-wearing on national and regional infection rates. We first analyzed different dimensions of linguistic complexity of the abstracts using the Coh-Metrix 3.0 webtool developed by Graesser et al. (2004). We focused primarily on scores for four text dimensions: narrativity, syntactic complexity, word concreteness, and referential cohesion. On a descriptive level, scientific abstracts on the effectiveness of mask-wearing exhibit significantly lower narrativity and make use of more complex syntactic structures when compared to average texts in the English language. Word concreteness and referential cohesion, in contrast, deviate less from average texts. However, when complexity scores were related to dissemination of scientific articles through the media (e.g., tweets), other factors such as publication date or open access status were more critical predictors.
The results of our project indicate that text complexity especially matters for people with a strong conspiracy mentality. For those participants, reading a complex (vs. simple) text led to feelings of exclusion, which, in turn, affected evaluations of message trustworthiness, trust in science, as well as adherence to the message. Such effects were not observed for participants with average nor with low conspiracy mentality. Although this is an effect concerning a rather small group, it is nevertheless important, as it is advantageous if all subgroups of society are reached through targeted health communication, especially during a pandemic. Given the exponential spread of COVID-19, even a small group of people can cause (or prevent) great harm. Improving science communication is generally important because it engages the audience with scientific findings (Burns et al., 2003), which, in turn, affects the extent to which science is an important factor in shaping people’s decisions, known as scientific literacy. There is no better occasion than a global pandemic to reconsider how difficult it should be for the lay public to understand and follow crucial messages. Expressing findings and recommendations in a less complex way seems to be a promising way to reach individuals with a high conspiracy mindset—those we have been
struggling to reach.
Dissemination Strategy and Compliance with Open Science Standards
To ensure the greatest possible transparency, all measurements, experimental designs, methods, and planned sample sizes have been pre-registered at the open science framework platform (see https://osf.io/hb9aq/). All collected data was made available at the related OSF project page. We also created an accompanying ResearchGate page for the project (see https://www.researchgate.net/project/Can-science-communication-be-too-scientific-The-influence-of-text-complexity-on-trust-and-compliance-following-health-related-messages). In addition, we prepared short and easy-to-read press releases that will be posted on our universities’ websites as soon as the data of the project are published in a scientific journal. As part of the project, an article entitled “Left Out – Feelings of Social Exclusion Incite Individuals with High Conspiracy Mentality to Reject Complex Scientific Messages” was submitted to the special issue “Stay Safe and Stay at Home! Research on Language and Communication Related to Corona” in the Journal of Language and Social Psychology and is currently under revision. Furthermore, a follow-up grant proposal was submitted and accepted by the European Association of Social Psychology to further develop the obtained findings and their theoretical relevance.
- Baram-Tsabari, A., & Lewenstein, B. V. (2013). An instrument for assessing scientists’ written skills in public communication of science. Science Communication, 35(1), 56-85. https://doi.org/10.1177/1075547012440634
- Burns, T. W., O’Connor, D. J., & Stocklmayer, S. M. (2003). Science communication: A contemporary definition. Public Understanding of Science, 12(2), 183–202. https://doi.org/10.1177/0963662503012200
- Ermakova, T., Baumann, A., Fabian, B., & Krasnova, H. (2014). Privacy policies and users’ trust: does readability matter?. Twentieth Americas Conference on Information Systems. Retrieved from: https://aisel.aisnet.org/amcis2014/HumanComputerInteraction/GeneralPresentations/14/
- Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, & Computers, 36(2), 193–202. https://doi.org/10.3758/bf03195564
- Lee, E., Sutton, R. M., & Hartley, B. L. (2016). From scientific article to press release to media coverage: advocating alcohol abstinence and democratising risk in a story about alcohol and pregnancy. Health, Risk & Society, 18(5–6), 247–269. https://doi.org/10.1080/13698575.2016.1229758
- Malecki, K., Keating, J. A., & Safdar, N. (2020). Crisis communication and public perception of COVID-19 risk in the era of social media. Clinical Infectious Diseases. https://doi.org/10.1093/cid/ciaa758
- Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., & The PRISMA Group (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097
- Plavén-Sigray, P., Matheson, G. J., Schiffler, B. C., & Thompson, W. H. (2017). The readability of scientific texts is decreasing over time. Elife, 6, e27725. https://doi.org/10.7554/eLife.27725
- Reber, R., Schwarz, N., & Winkielman, P. (2004). Processing Fluency and Aesthetic Pleasure: Is Beauty in the Perceiver’s Processing Experience? Personality and Social Psychology Review, 8(4), 364–382. https://doi.org/10.1207/s15327957pspr0804_3
- Samarrai, F. (2013, March 5). New Center for Open Science Designed to Increase Research. UVA Today. https://news.virginia.edu/content/new-center-open-science-designed-increase-research-transparency-provide-free-technologies
- Unkelbach, C., & Greifeneder, R. (2018). Experiential fluency and declarative advice jointly inform judgments of truth. Journal of Experimental Social Psychology, 79, 78–86. https://doi.org/10.1016/j.jesp.2018.06.010
- Vicente-Saez, R., & Martinez-Fuentes, C. (2018). Open Science now: A systematic literature review for an integrated definition. Journal of Business Research, 88, 428–436. https://doi.org/10.1016/j.jbusres.2017.12.043
- Wu, A. W., Connors, C., & Everly, G. S. (2020). COVID-19: Peer support and crisis communication strategies to promote institutional resilience. Annals of Internal Medicine, 172(12), 822–823. https://doi.org/10.7326/M20-1236
- Zabal, A., Martin, S., Klaukien, A., Rammstedt, B., Baumert, J., & Klieme, E. (2013). Grundlegende Kompetenzen der erwachsenen Bevölkerung in Deutschland im internationalen Vergleich. In B. Rammstedt (Ed.), Grundlegende Kompetenzen Erwachsener im internationalen Vergleich : Ergebnisse von PIAAC 2012 (S. 31–76). Münster: Waxmann. https://madoc.bib.uni-mannheim.de/54587/
- Zürn, M., & Topolinski, S. (2017). When trust comes easy: Articulatory fluency increases transfers in the trust game. Journal of Economic Psychology, 61, 74-86. https://doi.org/10.1016/j.joep.2017.02.016