The I-squared statistic has become hugely popular in meta-analysis. Motivated by the recent statement that 'Informally, it can be considered to reflect the extent to which confidence intervals displayed in a forest plot do not overlap with each other', this short note formalises this perspective. The insights provided by our analysis lend further weight to the interpretation of the I-squared statistic as a measure of inconsistency. This perspective reminds us that visual inspections of forest plots are vitally important when assessing between-study variation.
Abstract Meta-analysis is often viewed as a gold standard for synthesizing available evidence, but it faces challenges as evidence becomes available gradually and studies may be disseminated selectively and at different times. When publication timing is systematically related to study results, meta-analytic conclusions at any given time can be biased; for example, studies with statistically significant or larger effects may be published earlier than those with null or smaller effects. This phenomenon, commonly referred to as time-lag bias, is a key reason why early meta-analytic conclusions may later weaken, reverse, or become more heterogeneous as additional evidence accrues. To address the limitations of existing definitions and detection approaches, which are largely tied to monotone, effect-size-driven dissemination, we broaden the concept of time-lag bias beyond the traditional yet simple notion that “positive” studies are published earlier. Specifically, we view time-lag bias more generally as a manifestation of time-dependent evidence availability, whereby the timing of study dissemination can distort meta-analytic conclusions at different stages of evidence accrual. We illustrate this broader perspective through several real-world case studies, including classical time-lag bias, inverse time-lag bias, non-monotone temporal patterns, and anomalous evidence localized within specific time windows. These examples show that time-lag bias can arise through multiple dissemination mechanisms and may exaggerate, attenuate, or even reverse pooled estimates over time. They highlight the importance of explicitly accounting for the temporal dimension of evidence accumulation and motivate the routine use of diagnostic and sensitivity strategies to assess robustness to selective dissemination over time.
Abstract Meta-analytic structural equation modeling (MASEM) is a flexible modeling framework that combines the strengths of meta-analysis and structural equation modeling. Despite the increasing popularity of the MASEM technique, its usage and scope are limited to relatively simple meta-analytic data structures and moderator hypotheses. Within-study variability in moderator variables and dependence in primary studies’ correlation estimates tend to be handled using ad hoc methods. In this study, we demonstrate use of multilevel multivariate modeling with robust variance estimation (RVE) for handling within-study dependence when pooling studies’ correlation estimates and we propose how to incorporate categorical moderator values directly into the structural equation model for moderators with values varying within studies. We outline the proposed approach using three stages: (a) specification of the structural equation model using moderator variable categories, (b) pooling correlation matrix elements using the three-level multivariate model with RVE, and (c) testing the moderator hypotheses through either model comparison or direct estimation of structural equation model’s path differences. We illustrate the approach using three examples to demonstrate its flexibility across different MASEM models. This framework broadens the scope of moderator hypotheses that can be tested using MASEM and enhances the methodological rigor for handling dependencies in studies’ correlation estimates for MASEM.
Abstract Evidence synthesis (ES) is costly and time-intensive. Artificial intelligence (AI) tools are increasingly used to automate or semi-automate ES tasks. We conducted a living rapid review of AI tools that automate or semi-automate ES tasks. We searched for published articles and preprints through MEDLINE, Embase, and the Association for Computing Machinery Digital Library from January 1, 2021 through October 15, 2025, supplemented by hand-searching. We included existing reviews to capture evaluations before 2021. Screening was conducted in duplicate at the title/abstract level, with single full-text screening and data extraction verified by a second reviewer. We included 134 studies. Performance varied by task and evaluation approach. Search automation tools showed poor recall (median 17%), although human-in-the-loop approaches improved performance. Tools for identifying randomized controlled trials demonstrated high median recall (98%) and precision (92%). Semi-automated abstract screening tools using active learning reduced screening burden by approximately 50% while maintaining 95% recall. Large language models demonstrated variable accuracy for abstract screening (median 88% recall), full-text screening (median 96% recall), and data extraction (median 63% accuracy). Risk of bias (RoB) assessment tools showed moderate agreement with human reviewers for the Cochrane RoB tool version 1 (median 71%) but performed less well with other RoB tools. AI tools can meaningfully reduce burden in study design identification and abstract screening but perform inconsistently across other evidence synthesis tasks. Tools for search automation, data extraction, and RoB assessment require further development. No existing tool can currently replace human expertise.
Abstract Meta-analyses require an effect-size estimate and its corresponding sampling variance from primary studies. For some effect size statistics, however, estimators of sampling variance are unavailable, requiring new derivations. Traditionally, such formulas are obtained using hand-derived Taylor expansions (the delta method), but this approach can be technically demanding and inaccessible to many applied researchers. Building on the idea of single-fit parametric resampling, we introduce the SAFE bootstrap: a Single-fit, Accurate, Fast, and Easy simulation recipe that replaces complex algebra with four intuitive steps: fit, draw, transform, and summarise. SAFE is a model-based parametric bootstrap applied to summary statistics, so its performance depends on how well the assumed sampling model approximates the sampling distribution of the reported statistics. Here, we focus on two-group effect sizes that can be parameterised from standard reported summaries, using Gaussian working models for continuous outcomes and binomial or multinomial models for discrete outcomes. Within this framework, SAFE yields model-based, bias-corrected point estimates and standard errors for familiar effect sizes, and readily extends to less common statistics. We first demonstrate SAFE with a simple example, then apply it to common effect sizes, such as the standardised mean difference and the log odds ratio, as well as several less common measures. With additional coding, SAFE can also accommodate zero values and small sample sizes, albeit with important caveats. Our tutorial, accompanied by R code supplements, aims both to clarify the logic of simulation-based estimation from summary statistics and to provide a practical workflow when only summary statistics are available.
