Pursuing replicability - independent evidence for previous claims - is important for creating generalizable knowledge(1,2). Here we attempted replications of 274 claims of positive results from 164 quantitative papers published from 2009 to 2018 in 54 journals in the social and behavioural sciences. Replications were high powered on average to detect the original effect size (median of 99.6%), used original materials when relevant and available, and were peer reviewed in advance through a standardized internal protocol. Replications showed statistically significant results in the original pattern for 151 of 274 claims (55.1% (95% confidence interval (CI) 49.2-60.9%)) and for 80.8 of 164 papers (49.3% (95% CI 43.8-54.7%)), weighed for replicating multiple claims per paper. We observed modest variation in replication rates across disciplines (42.5-63.1%), although some estimates had high uncertainty. The median Pearson's r effect size was 0.25 (95% CI 0.21-0.27) for original studies and 0.10 (95% CI 0.09-0.13) for replication studies, an 82.4% (95% CI 67.8-88.2%) reduction in shared variance. Thirteen methods for evaluating replication success provided estimates ranging from 28.6% to 74.8% (median of 49.3%). Some decline in effect size and significance is expected based on power to detect original effects and regression to the mean because we replicated only positive results. We observe that challenges for replicability extend across social-behavioural sciences, illustrating the importance of identifying conditions that promote or inhibit replicability(3,4).
Situation models—mental representations formed during comprehension—have evolved significantly beyond their origin in text-based research. Situation models are now used in accounts of comprehension across various media, including films, comics, and even real-world events. This article reviews four key developments in comprehension research over the past 25 years: grounded cognition, multitext comprehension, visual media comprehension, and everyday event comprehension. In all these lines of research, situation models continue to play an important role.
Science is integral to society because it can inform individual, government, corporate, and civil society decision-making on issues such as public health, new technologies or climate change. Yet, public distrust and populist sentiment challenge the relationship between science and society. To help researchers analyse the science-society nexus across different geographical and cultural contexts, we undertook a cross-sectional population survey resulting in a dataset of 71,922 participants in 68 countries. The data were collected between November 2022 and August 2023 as part of the global Many Labs study “Trust in Science and Science-Related Populism” (TISP). The questionnaire contained comprehensive measures for individuals’ trust in scientists, science-related populist attitudes, perceptions of the role of science in society, science media use and communication behaviour, attitudes to climate change and support for environmental policies, personality traits, political and religious views and demographic characteristics. Here, we describe the dataset, survey materials and psychometric properties of key variables. We encourage researchers to use this unique dataset for global comparative analyses on public perceptions of science and its role in society and policy-making.
This 68-country survey (n = 71,922) examines science information diets and communication behavior, identifies cross-country differences, and tests how such differences are associated with sociopolitical and economic conditions. We find that social media are the most used sources of science information in most countries, except those with democratic-corporatist media systems where news media tend to be used more widely. People in collectivist societies are less outspoken about science in daily life, whereas lower education is associated with higher outspokenness. Limited access to digital media is correlated with participation in public protests on science matters. We discuss implications for future research, policy, and practice.
Science is crucial for evidence-based decision-making. Public trust in scientists can help decision makers act on the basis of the best available evidence, especially during crises. However, in recent years the epistemic authority of science has been challenged, causing concerns about low public trust in scientists. We interrogated these concerns with a preregistered 68-country survey of 71,922 respondents and found that in most countries, most people trust scientists and agree that scientists should engage more in society and policymaking. We found variations between and within countries, which we explain with individual- and country-level variables, including political orientation. While there is no widespread lack of trust in scientists, we cannot discount the concern that lack of trust in scientists by even a small minority may affect considerations of scientific evidence in policymaking. These findings have implications for scientists and policymakers seeking to maintain and increase trust in scientists.
In this article, I outline the evolution of my research in the domain of narrative comprehension. I cover three phases: (1) the event-indexing model, which explores connecting events in narratives; (2) the immersed experiencer framework, which focuses on event representations and mental simulations; and (3) a preliminary exploration of conspiracy thinking, in which I reveal parallels with coherence generation in narrative comprehension. I emphasize the interconnectedness of these domains and call for further investigation into the intricacies of narrative comprehension.
The recent coronavirus pandemic impacted the mental health of people worldwide as it rapidly spread to most countries. Social media has provided a cost-effective method to examine these negative effects, but which aspects of the disease impacted the general population's psychology is not well understood. This study examined one potential factor that moderated people's responses to the fast-spreading, deadly coronavirus disease (COVID) during its emergence from January 2020 to March 2020. Applying sentiment analysis to 3.2 million COVID-related messages posted on Twitter from 189 countries, we examined how the physical distance to COVID impacted the attention and emotions of the general population as it spread around the globe. The spatial distance from each message's origin country to the nearest COVID-infected country was computed to use as an independent variable. Statistical analyses revealed that spatial distance significantly influenced both public attention and sentiment toward COVID, even when controlling for confounders. As the disease came closer, more tweets were posted and the average sentiment became more negative. These observations suggest that physical proximity to a threat influences how much attention people pay to the threat and how they respond to it emotionally. This is in line with previous disaster research and fits the psychological framework of construal level theory. Although these findings are limited in their generalizability, they have important implications. In practice, communicating the personal risks of a disease outbreak to distant people might increase public engagement in protective behaviors such as social distancing and hand washing, subsequently slowing disease spread.
Walter Kintsch has an unparalleled impact on the field of discourse processing. In this article, I describe the impact he has had and continues to have on my own work. I summarize my impression of Walter with two words—grace and gravitas—and provide anecdotes to show how these qualities of his personality have impacted me as a researcher. Furthermore, I discuss how his work with Teun van Dijk on situation models was instrumental for the development of the event-indexing model and the immersed-experiencer framework.
The COVID-19 pandemic has affected all domains of human life, including the economic and social fabric of societies. One of the central strategies for managing public health throughout the pandemic has been through persuasive messaging and collective behaviour change. To help scholars better understand the social and moral psychology behind public health behaviour, we present a dataset comprising of 51,404 individuals from 69 countries. This dataset was collected for the International Collaboration on Social & Moral Psychology of COVID-19 project (ICSMP COVID-19). This social science survey invited participants around the world to complete a series of moral and psychological measures and public health attitudes about COVID-19 during an early phase of the COVID-19 pandemic (between April and June 2020). The survey included seven broad categories of questions: COVID-19 beliefs and compliance behaviours; identity and social attitudes; ideology; health and well-being; moral beliefs and motivation; personality traits; and demographic variables. We report both raw and cleaned data, along with all survey materials, data visualisations, and psychometric evaluations of key variables.
Increased execution of replication studies contributes to the effort to restore credibility of empirical research. However, a second generation of problems arises: the number of potential replication targets is at a serious mismatch with available resources. Given limited resources, replication target selection should be well-justified, systematic and transparently communicated. At present the discussion on what to consider when selecting a replication target is limited to theoretical discussion, self-reported justifications and a few formalized suggestions. In this Registered Report, we proposed a study involving the scientific community to create a list of considerations for consultation when selecting a replication target in psychology. We employed a modified Delphi approach. First, we constructed a preliminary list of considerations. Second, we surveyed psychologists who previously selected a replication target with regards to their considerations. Third, we incorporated the results into the preliminary list of considerations and sent the updated list to a group of individuals knowledgeable about concerns regarding replication target selection. Over the course of several rounds, we established consensus regarding what to consider when selecting a replication target. The resulting checklist can be used for transparently communicating the rationale for selecting studies for replication.
Background Little is known about how conspiracy beliefs and health responses are interrelated over time during the course of the coronavirus disease 2019 (Covid-19) pandemic. This longitudinal study tested two contrasting, but not mutually exclusive, hypotheses through cross-lagged modeling. First, based on the consequential nature of conspiracy beliefs, we hypothesize that conspiracy beliefs predict an increase in detrimental health responses over time. Second, as people may rationalize their behavior through conspiracy beliefs, we hypothesize that detrimental health responses predict increased conspiracy beliefs over time. Methods We measured conspiracy beliefs and several health-related responses (i.e. physical distancing, support for lockdown policy, and the perception of the coronavirus as dangerous) at three phases of the pandemic in the Netherlands (N = 4913): During the first lockdown (Wave 1: April 2020), after the first lockdown (Wave 2: June 2020), and during the second lockdown (Wave 3: December 2020). Results For physical distancing and perceived danger, the overall cross-lagged effects supported both hypotheses, although the standardized effects were larger for the effects of conspiracy beliefs on these health responses than vice versa. The within-person change results only supported an effect of conspiracy beliefs on these health responses, depending on the phase of the pandemic. Furthermore, an overall cross-lagged effect of conspiracy beliefs on reduced support for lockdown policy emerged from Wave 2 to 3. Conclusions The results provide stronger support for the hypothesis that conspiracy beliefs predict health responses over time than for the hypothesis that health responses predict conspiracy beliefs over time.
We investigated how deictic shifts and memory representation influence production difficulties in reported speech. In Experiment 1, participants read short stories, including a conversation between two protagonists. Participants then recalled the last sentence from the conversation in either direct or indirect speech. When participants had verbatim memory of the to-be-reported utterance, direct speech production was faster than indirect speech if indirect speech production required a deictic shift. If participants had verbatim memory and indirect speech production did not require a deictic shift, direct and indirect speech's speech latencies were similar. In Experiment 2, we investigated the production difficulties of reported speech when verbatim memory was unavailable to participants. First, participants read short stories. Next, half of the participants completed an intervening task, whereas the other half did not. Last, participants undertook a sentence completion task by filling in a missing word in direct or indirect speech. When participants did not have verbatim memory of to-be-reported utterances, direct speech production was faster than indirect speech if a deictic shift was needed to produce direct speech. These results suggest that production difficulties in reported speech depend on how to-be-reported utterances are represented in memory and whether deictic shifts are required.
The current study investigated how psychological distance affects people's preference for direct and indirect speech in a narrative task. In three experiments, participants were instructed to first watch a video and then retell what happened in the video to an imagined/anticipated listener. We manipulated social distance (Experiment 1), temporal distance (Experiment 2), and spatial distance (Experiment 3) between participants and the listener. We compared the proportions of direct speech in the narrations from psychologically proximal versus distal conditions. Experiments 1 and 2 showed that social and temporal proximity increased the rates of direct speech. Social and temporal distance, conversely, increased the rates of indirect speech. Experiment 3 did not yield a significant difference in the use of direct and indirect speech between spatially proximal and distal conditions. Taken together, our results indicate that different psychological dimensions might have discrepant effects on people's choices between direct and indirect speech. Possible explanations for the discrepancy among different psychological distance dimensions are discussed.
Although many studies have investigated the activation of perceptual representations during language comprehension, to our knowledge only one previous study has directly tested how perceptual features are combined into representations during comprehension. In their classic study, Potter and Faulconer [(1979). Understanding noun phrases. Journal of Verbal Learning and Verbal Behavior, 18, 509-521.] investigated the perceptual representation of adjective-noun combinations. However, their non-orthogonal design did not allow the differentiation between conjunctive vs. disjunctive representations. Using randomized orthogonal designs, we observe evidence for disjunctive perceptual representations when participants represent feature combinations simultaneously (in several experiments; N = 469), and we observe evidence for conjunctive perceptual repre-sentations when participants represent feature combinations sequentially (In several experiments; N = 628). Our findings show that the generation of conjunctive representations during comprehension depends on the concatenation of linguistic cues, and thus suggest the construction of elaborate perceptual representations may critically depend on language.
The current study aims to explore the factors that could affect people’s description of a motion event endpoint. The study conducted by Liao, Dijkstra, and Zwaan (2021, Language and Cognition, 13[2], 161–190) found that two non-linguistic factors (i.e., the actor’s goal and the interlocutor’s social status) affect people’s choice between two Dutch directional prepositions (i.e., naar and richting) during event description tasks. The current study aims to extend these findings by examining the choice between a similar pair of directional prepositions in English (i.e., to and towards). Moreover, we aim to study whether grammatical aspect (i.e., the English simple present and the English progressive aspect) affects the sensitivity to the two non-linguistic factors and consequently also affects how people describe a motion event endpoint. In Experiment 1, we used the English simple present for all sentence stimuli (e.g., he walks (?) the trash bin). We found a significant effect of Interlocutor (the interlocutor’s social status) on preposition choice, but no significant effect of Intention (the actor’s goal). In Experiment 2, we replaced the English simple present with the English progressive aspect (e.g., he is walking (?) the trash bin). We found significant main effects of both Interlocutor and Intention on preposition choice. These findings extend those reported in Liao et al. (2021) Language, Cognition and Neuroscience, 35(4), 498–520 in that protagonist intention and interlocutor status were found to indeed affect motion event endpoint description. The current findings furthermore show that grammatical aspect affects people’s sensitivity to these factors, thus also affecting how a motion event endpoint is described.
Changing collective behaviour and supporting non-pharmaceutical interventions is an important component in mitigating virus transmission during a pandemic. In a large international collaboration (Study 1, N = 49,968 across 67 countries), we investigated self-reported factors associated with public health behaviours (e.g., spatial distancing and stricter hygiene) and endorsed public policy interventions (e.g., closing bars and restaurants) during the early stage of the COVID-19 pandemic (April-May 2020). Respondents who reported identifying more strongly with their nation consistently reported greater engagement in public health behaviours and support for public health policies. Results were similar for representative and non-representative national samples. Study 2 ( N = 42 countries) conceptually replicated the central finding using aggregate indices of national identity (obtained using the World Values Survey) and a measure of actual behaviour change during the pandemic (obtained from Google mobility reports). Higher levels of national identification prior to the pandemic predicted lower mobility during the early stage of the pandemic ( r = −0.40). We discuss the potential implications of links between national identity, leadership, and public health for managing COVID-19 and future pandemics.
At the beginning of 2020, COVID-19 became a global problem. Despite all the efforts to emphasize the relevance of preventive measures, not everyone adhered to them. Thus, learning more about the characteristics determining attitudinal and behavioral responses to the pandemic is crucial to improving future interventions. In this study, we applied machine learning on the multinational data collected by the International Collaboration on the Social and Moral Psychology of COVID-19 (N = 51,404) to test the predictive efficacy of constructs from social, moral, cognitive, and personality psychology, as well as socio-demographic factors, in the attitudinal and behavioral responses to the pandemic. The results point to several valuable insights. Internalized moral identity provided the most consistent predictive contribution-individuals perceiving moral traits as central to their self-concept reported higher adherence to preventive measures. Similar results were found for morality as cooperation, symbolized moral identity, self-control, open-mindedness, and collective narcissism, while the inverse relationship was evident for the endorsement of conspiracy theories. However, we also found a non-neglible variability in the explained variance and predictive contributions with respect to macro-level factors such as the pandemic stage or cultural region. Overall, the results underscore the importance of morality-related and contextual factors in understanding adherence to public health recommendations during the pandemic.
Conspiracy thinking can be viewed as a form of narrative comprehension. We routinely infer actors' goals and plans when someone tells us a story. However, comprehenders extend this practice when considering real-world events, which are not orchestrated by a narrator. Comprehenders routinely favor information that is consistent with their perspective, but conspiracy thinkers likely do this to a greater extent, due to the low levels of cognitive reflection they exhibit. Comprehenders supplement incoming information with background knowledge. Conspiracy thinkers do this as well, but their knowledge base deviates from that of the mainstream, as a result of exposure to large amounts of misinformation.
People routinely shift between direct and indirect speech in everyday communication. The factors that impact the selection between these two modes of reporting during language production are under-investigated. The present study examined how utterance-related factors (the vividness of non-verbal information and the utterance type) influence the use of direct and indirect reported speech in narratives. Participants were asked to watch and retell four movie clips. All narratives were videotaped and then transcribed verbatim for analyses. The data were analyzed using a mixed effects logistic regression model. The results showed that the utterances accompanied by vivid voice were more likely to be reported in direct speech. The vividness of facial expressions did not influence the form in which utterances were reported. In addition, we found that utterances that belonged to so-called Main Clause Phenomena were more likely to be reported in direct speech than in indirect speech. The current study helps us further understand the factors that influence structure choices during language production.