Meta-analyses play an influential role in synthesizing the existing evidence on a particular topic. Consequently, it is especially important that meta-analyses are conducted and reported to the highest standards and that the risk of bias is minimized. Preregistration can help detect and reduce bias arising from opportunistic use of “researcher degrees of freedom.” However, little is known about the prevalence and practice of the preregistration of meta-analyses in psychology. In this study, we first measured the prevalence of preregistration in all psychology meta-analyses published in 2021. Next, for 100 randomly selected preregistered meta-analyses, we evaluated the preregistration’s coverage of key meta-analytic decisions and the extent to which published meta-analyses deviated from their preregistered protocols. Of all 1,403 eligible psychology meta-analyses published in 2021, 382 (27%) were preregistered. In our random sample, we found that key PRISMA-P decision items were often omitted from preregistered protocols—out of the 23 decision items that were examined, the median number of items covered was 13 (interquartile range [IQR] = 11–14). We also found that all 100 preregistered meta-analyses contained at least one deviation from the preregistered protocol ( Mdn = 9, IQR = 6.75–11) and that most deviations were undisclosed ( Mdn = 8, IQR = 6–11). These findings suggest that the infrequent use and poor implementation of preregistration in psychology meta-analyses undermines its potential to reduce bias and increase transparency.

The School Attitude Assessment Survey-Revised (SAAS-R) is a popular scale for assessing attitudinal and motivational aspects of students’ academic achievement. However, evidence on key psychometric properties of the SAAS-R such as reliability remains limited. We conducted a reliability generalization study of the SAAS-R using meta-analytic structural equation modeling (MASEM). We included studies reporting an application of the SAAS-R and providing correlation coefficients between the SAAS-R subscales. We searched ERIC, PsycINFO, Academic Search Premier, Supplemental Index, and Web of Science from database inception to July 2023. Analyses were based on 18 independent matrices from 13 studies examining 8712 participants. Our main results, based on a one-stage, correlation-based MASEM approach and with omega total as the reliability measure, yielded an overall reliability estimate of 0.795 (95% CI 0.778–0.811). This suggests that the SAAS-R offers good score reliability for research and practice purposes. Applications using adapted versions obtained on average higher score reliabilities than the original ones. We discuss the implications of these results, which need to be interpreted with caution given the important reporting limitations of the primary studies included.

Meta-analysis is one of the most useful research approaches, the relevance of which relies on its credibility. Reproducibility of scientific results could be considered as the minimal threshold of this credibility. We assessed the reproducibility of a sample of meta-analyses published between 2000 and 2020. From a random sample of 100 articles reporting results of meta-analyses of interventions in clinical psychology, 217 meta-analyses were selected. We first tried to retrieve the original data by recovering a data file, recoding the data from document files, or requesting it from original authors. Second, through a multistage workflow, we tried to reproduce the main results of each meta-analysis. The original data were retrieved for 67% (146/217) of meta-analyses. Although this rate showed an improvement over the years, in only 5% of these cases was it possible to retrieve a data file ready for reuse. Of these 146, 52 showed a discrepancy larger than 5% in the main results in the first stage. For 10 meta-analyses, this discrepancy was solved after fixing a coding error of our data-retrieval process, and for 15 of them, it was considered approximately reproduced in a qualitative assessment. In the remaining meta-analyses (18%, 27/146), different issues were identified in an in-depth review, such as reporting inconsistencies, lack of data, or transcription errors. Nevertheless, the numerical discrepancies were mostly minor and had little or no impact on the conclusions. Overall, one of the biggest threats to the reproducibility of meta-analysis is related to data availability and current data-sharing practices in meta-analysis.

The aim of this study was to examine the reliability, validity, and factorial structure of the Spanish version of the Clance Impostor Phenomenon Scale (CIPS). A sample of 271 Spanish students was recruited to complete a translated version of the original 20-item CIPS. In our sample, the instrument showed high internal consistency reliability (ωTotal =.90) and a moderate-to-strong correlation with measures of depression (r = .633), self-esteem (r = -.754) and fear of negative evaluation (r = .666), suggesting both nomological and discriminant validity. Although the original validation of the CIPS proposed a factorial structure with three factors, subsequent validations also revealed adjustment to two- and one-factor structures. Here, we used confirmatory factor analysis (CFA) to test the three different models. The results showed that in our adaptation, a 2-factor structure might be preferred. This adaptation of the CIPS to Spanish provides clinicians with a new method to gain insight into the psychological mechanisms behind the Impostor phenomenon and suitable treatments. El objetivo del presente estudio fue el de examinar la fiabilidad, validez y estructura factorial de la adaptación española de la Clance Impostor Phenomenon Scale (CIPS). Para ello, un total de 271 estudiantes españoles completaron una versión traducida de la escala original de 20 ítems. En nuestra muestra, el instrumento mostró una alta fiabilidad, medida como consistencia interna, (ωTotal =.90) y correlaciones moderadas-altas con medidas de depresión (r =.633), autoestima (r = -.754) y miedo a las evaluaciones negativas (r = .666), lo cual sugiere tanto una validez nomológica como discriminante. Aunque en la validación original se propuso una estructura de tres factores, otros estudios han encontrado ajuste a estructuras de uno y dos factores. Aquí, utilizamos un análisis factorial confirmatorio (AFC) para probar el ajuste de estos tres modelos. Nuestros resultados muestran que, en la adaptación a español, el modelo con dos factores es el preferido. Esta adaptación al español de la CIPS provee a los profesionales clínicos una de una nueva herramienta para poder investigar los mecanismos que subyacen al síndrome del impostor, así como futuros tratamientos.

In target papers, Schönbrodt et al. (2022), and Gärtner et al. (2022) proposed to broaden the range of the considered research contributions, namely (i) bringing strong empirical evidence, (ii) building open databases, (iii) building and maintaining packages, with each dimension being scored independently in marking scheme. Using simulations, we show that the current proposal places a significant weight on software development, potentially at the expense of other academic activities – a weight that should be explicit to committees before they make use of the proposed marking scheme. Following Gärtner et al. (2022) recommendations, we promote the use of flexible weights which more closely match an institution’s specific needs by the weighting of the relevant dimensions. We propose a Shiny app that implement the marking scheme with adaptative weights to both help the hiring committee define and foresee the consequences of weights’ choices and increase the transparency and understandability of the procedure.

One of the biggest limitations of meta-analyses is that the information they provide can be affected by the biases of the included primary studies. To address this, evaluations of primary study risk of bias (RoB) can be performed and incorporated into the meta-analysis. However, research on this topic in clinical psychology is scarce. In this study, we examined this issue using a sample of clinical psychology meta-analyses that included RoB assessments. First, we evaluated meta-analysts’ assessment practices. Second, we summarized the RoB ratings of the primary studies included in the meta-analyses. Lastly, we examined the relationship between RoB ratings and effect sizes. We found some suboptimal practices in the assessment procedures, such as only half of the studies reporting that the assessment was conducted in duplicate. Regarding RoB ratings, the domains with the highest ratings were random sequence generation, blinding of outcome assessment, and incomplete outcome data, with about half of the primary studies rated as low RoB. The lowest ratings were found for allocation concealment and, especially, blinding of participants and personnel. Importantly, we found a positive association between the publication year of the primary studies and a lower RoB in most domains. Lastly, performing our own re-analysis, we found an association between RoB and effect sizes, which contrasts with the results of the analyses reported in the meta-analyses that combined those studies. We recommend caution when interpreting a lack of modulation of effect sizes in meta-analyses, as they may not have sufficient statistical power for moderator analyses.

We investigated whether chronotype and time-of-day modulate the time course of automatic and controlled semantic processing. Participants performed a category semantic priming task at either the optimal or non-optimal time of day. We varied the prime-target onset asynchrony (100-, 450-, 650-, and 850-ms SOAs) and kept the percentage of unrelated targets constant at 80%. Automatic processing was expected with the short SOA, and controlled processing with longer SOAs. Intermediate-types (Experiment 1) verified that our task was sensitive to capturing both types of processes and served as a reference to assess themin extreme chronotypes. Morning-type and evening-type participants (Experiment 2) differed in the influence of time of testing on priming effects. Morning-types applied control in all conditions, and no performance modulation by time-of-day was observed. In contrast, evening-types were most adversely affected by the time of day to shift from automatic-based to controlled-based responses. Also, they were considerably affected in successfully implementing controlled processing with long intervals, particularly at the non-optimal time of day, with inhibitory priming showing only a marginally significant effect at the longest SOA. These results suggest that extreme chronotypes may be associated with different styles of cognitive control. Morning-types would be driven by a proactive control style, whereas a reactive control style might be applied by evening-types.

Flexible updating of information in Visual Working Memory (VWM) is crucial to deal with its limited capacity. Previous research has shown that the removal of no longer relevant information takes some time to complete. Here, we sought to study the time course of such removal by tracking the accompanying drop in load through behavioral and neurophysiological measures. In the first experimental session, participants completed a visuospatial retro‐cue task in which the Cue‐Target Interval (CTI) was manipulated. The performance revealed that it takes about half a second to make full use of the retro‐cue. In a second session, we sought to study the dynamics of load‐related electroencephalographic (EEG) signals to track the removal of information. We applied Multivariate Pattern Analysis (MVPA) to EEG data from the same task. Right after encoding, results replicated previous research using MVPA to decode load. However, especially after the retro‐cue, results suggested that classifiers were mainly sensitive to a selection component, and not so much to load per se. Additionally, visual cue variations, as well as eye movements that accompany load manipulations can also contribute to decoding. These findings advise caution when using MVPA to decode VWM load, as classifiers may be sensitive to confounding operations.

Meta-analyses often present flexibility regarding their inclusion criteria, outcomes of interest, statistical analyses, and assessments of the primary studies. For this reason, it is necessary to transparently report all the information that could impact the results. In this meta-review, we aimed to assess the transparency of meta-analyses that examined the benefits of cognitive training, given the ongoing controversy that exists in this field. Ninety-seven meta-analytic reviews were included, which examined a wide range of populations with different clinical conditions and ages. Regarding the reporting, information about the search of the studies, screening procedure, or data collection was detailed by most reviews. However, authors usually failed to report other aspects such as the specific meta-analytic parameters, the formula used to compute the effect sizes, or the data from primary studies that were used to compute the effect sizes. Although some of these practices have improved over the years, others remained the same. Moreover, examining the eligibility criteria of the reviews revealed a great heterogeneity in aspects such as the training duration, age cut-offs, or study designs that were considered. Preregistered meta-analyses often specified poorly how they would deal with the multiplicity of data or assess publication bias in their protocols, and some contained non-disclosed deviations in their eligibility criteria or outcomes of interests. The findings shown here, although they do not question the benefits of cognitive training, illustrate important aspects that future reviews must consider.

The self-prioritization effect (SPE) refers to the advantage in processing stimuli associated with oneself. Here, we addressed the SPE in an attentional blink (AB) task. In Experiment 1, shapes associated to you, friend, or stranger served as T1, and letter X as T2. The AB effect was larger for you than the other label conditions, and larger for friend condition than for stranger condition. We suggest that self-associated shape increased its perceptual salience, producing greater attentional capture. In Experiment 2 participants trained with a shape-label matching task to increase familiarity with the shape-label associations before performing the AB task. The difference between friend and stranger conditions disappeared, suggesting that the difference between the two conditions observed in Experiment 1 was mainly due to differences in familiarity or frequency of use. Importantly, the advantage of you over friend and stranger conditions remained, suggesting that the SPE is a genuine effect.

Background:Decrements in performance and the propensity for increased mind-wandering (i.e., task-unrelated thoughts) across time-on-task are two pervasive phenomena observed when people perform vigilance tasks. In the present study, we asked whether processes that lead to vigilance decrement and processes that foster the propensity for mind-wandering (MW) can be dissociated or whether they share a common mechanism. In one experiment, we introduced two critical manipulations: increasing task demands and applying anodal high-definition transcranial direct current stimulation (HD-tDCS) to the left dorsolateral prefrontal cortex.Method:Seventy-eight participants were randomly assigned to one of four groups resulting from the factorial combination of task demand (low, high) and stimulation (anodal, sham). Participants completed the sustained attention to response task (SART), which included thought probes on intentional and unintentional MW. In addition, we investigated the crucial role of alpha oscillations in a novel approach. By assessing pre-post resting EEG, we explored whether participants' variability in baseline alpha power predicted performance in MW and vigilance decrement related to tDCS or task demands, respectively, and whether such variability was a stable characteristic of participants.Results:Our results showed a double dissociation, such that task demands exclusively affected vigilance decrement, while anodal tDCS exclusively affected the rate of MW. Furthermore, the slope of the vigilance decrement function and MW rate (overall, intentional and unintentional) did not correlate. Critically, resting state alpha-band activity predicted tDCS-related gains in unintentional MW alone, but not in vigilance decrement, and remained stable after participants completed the task.Conclusion:These results show that when a sustained attention task involving executive vigilance, such as the SART, is designed to elicit both vigilance decrement effects and MW, the processes leading to vigilance decrement should be differentiated from those responsible for MW, a claim that is supported by the double dissociation observed here and the lack of correlation between the measures chosen to assess both phenomena. Furthermore, the results provide the first evidence of how individual differences in alpha power at baseline may be of crucial importance in predicting the effects of tDCS on MW propensity.

Flexible updating of information in Visual Working Memory (VWM) is crucial to deal with its limited capacity. Previous research has shown that the removal of no longer relevant information takes some time to complete. Here, we sought to study the time course of such removal by tracking the accompanying drop in load through behavioral and neurophysiological measures. In a first experimental session, participants completed a visuospatial retro-cue task in which the Cue-Target Interval (CTI) was manipulated. Performance revealed that it takes about half a second to make full use of the retro-cue. In a second session, we sought to study the dynamics of load-related electroencephalographic (EEG) signals to track the removal of information. We applied Multivariate Pattern Analysis (MVPA) to EEG data from the same task. However, contrary to previous research indicating that MVPA can be used to uniquely decode VWM load, the results suggested that the classifiers were mainly sensitive to selection, visual cue variations, or eye movements that accompany load manipulations, and not so much to load per se. These findings advise caution when using MVPA to decode VWM load, as classifiers may be sensitive to confounding operations.

The Florida Obsessive Compulsive Inventory (FOCI) and its pediatric version, the Child-Florida Obsessive Compulsive Inventory (C-FOCI) are instruments for evaluating obsessive-compulsive symptomatology. A reliability generalization meta-analysis was conducted to estimate an average reliability of the scores and to identify study characteristics that explained the heterogeneity among scores. Using KR-20 and Cronbach’s alpha, a total of 23 and 20 independent samples were included in the meta-analysis for the FOCI and C-FOCI. We found an average KR-20 of .826 for the FOCI's Symptom Checklist and an alpha of .882 FOCI's Symptom Severity. An average KR-20 of .740 was found for the C-FOCI's Symptom Checklist, while an average alpha of .794 was found for the C-FOCI's Symptom Severity. Moderator analyses showed that the source of the coefficients (i.e. whether they were reported by authors or estimated) was an important variable for the FOCI Symptom Severity, and that the focus of the study (i.e. whether it was psychometric or applied) and the sample size were relevant for the C-FOCI Symptom Checklist. Conclusions: Considering that the FOCI and C-FOCI are scales characterized by their brevity and ease of use, and the reliabilities obtained here, both scales are well suited for screening purposes.

Background Circadian rhythm patterns vary across individuals, producing the existence of different chronotypes. Chronotype refers to the preferences of individuals to perform their daily life activities, being classified as intermediate-types (no specific circadian preference), morning-types (preference for early morning hours) and evening-types (preference for later hours). We investigated whether chronotype and time-of-day modulate the time course of automatic and controlled semantic processing. Participants performed a category semantic priming task at either the optimal or non-optimal time-of-day. We varied the prime-target onset asynchrony (100-, 450-, 650-, and 850-ms SOAs) and kept the percentage of unrelated targets constant at 80%. Automatic processing, producing facilitatory priming, was expected with the short SOA, and controlled processing, producing inhibitory priming, was expected with longer SOAs. Results Intermediate-types (Experiment 1) verified that our task was sensitive to capturing both types of processes. Facilitatory automatic priming was observed early in the first trials of the short 100-ms SOA, then controlled and automatic processing cancelled each other producing a lack of priming effects (2nd and 3rd subblocks of the 100-ms SOA and the 450-ms SOA). Finally, with the longer SOAs only inhibitory controlled priming was observed, which size increased with time. This time course of automatic and controlled processing in intermediate-types served as reference to assessed both types of processes in extreme chronotypes. Morning-type and evening-type participants (Experiment 2) differed in the influence of time of testing on priming effects. Morning-types applied control in all conditions, and no modulation of performance by time-of-day was observed. In contrast, evening-types were only able to suppress automatic processing when the task was performed at their optimal time of day. Also, they were considerably slower in the implementation of controlled processing as inhibitory priming occur in the longest 850-ms SOA only. Conclusions These results suggest that extreme chronotypes are associated with different styles of cognitive control. Morning-types show a proactive control style that fosters maintenance of active response strategies to resolve conflict without producing negative effects. Evening-types show a reactive control style such that conflict resolution is influenced by task demands. This dissociation suggests a better adaptation strategy of the morning-type chronotype.

Abstract Background Decrements in performance and the propensity for increased mind-wandering (i.e., task-unrelated thoughts) across time-on-task are two pervasive phenomena observed when people perform vigilance tasks. Whether vigilance decrement and mind-wandering (MW) are independent phenomena is still an open question. In the present study, we addressed such a contention by introducing two critical manipulations: increasing task demands and applying anodal high-definition transcranial direct current stimulation (HD-tDCS) over the left dorsolateral prefrontal cortex. Method Seventy-eight participants were randomly assigned to one of the four groups resulting from the factorial combination of task demand (low, high) and stimulation (anodal, sham). Participants completed the sustained attention to response task (SART), which included thought probes about intentional and unintentional MW. Moreover, we investigated the crucial role of alpha oscillations in a novel approach. By evaluating prepost resting EEG, we explored whether participant variability in the alpha power baseline predicted performance in MW and vigilance decrement related to tDCS or task demands, respectively, and whether such variability was a stable characteristic of participants. Results Our results showed that task demands exclusively affected vigilance decrement, while anodal tDCS exclusively affected the rate of MW. Critically, alpha-band activity at rest predicted tDCS-related gains in unintentional MW but not vigilance decrement and remained stable after participants completed the task. Conclusions These results suggest that vigilance decrement and MW are two independent phenomena. Additionally, the results provide the first evidence of how individual differences in alpha power at baseline may be of crucial importance in predicting the effects of tDCS on MW propensity.