Abstract This tutorial provides practical guidance on deciding whether to include non‐randomized studies of interventions (NRSI) in systematic reviews. Drawing on existing Cochrane and GRADE guidance, it outlines circumstances in which NRSI may provide valuable evidence alongside or in the absence of randomized controlled trials. A practical checklist supports review authors in making decisions about study eligibility for each PICO at the protocol stage, considering the availability and limitations of randomized evidence, the outcomes of interest, equity, and generalizability.
ABSTRACT Background The assessment of the trustworthiness and relevance of available evidence is a key element of evidence synthesis. For example, Cochrane systematic reviews (SRs) mandate the assessment of certainty of evidence (CoE) using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) method. However, most SRs are published outside Cochrane, and the use of CoE outside Cochrane is not universal. The use of CoE in SRs of yoga for maintaining or improving health is not known. Objective To: (a) estimate the proportion of SRs of yoga for health that assessed CoE; (b) identify which CoE systems and domains were used; and (c) examine whether CoE assessment was associated with SR publication and methodological characteristics. Methods The Cochrane Database of Systematic Reviews (CDSR), MEDLINE (Ovid), PubMed, EMBASE (Ovid), AMED (Ovid), CINAHL (EBSCO), and PsycINFO (EBSCO) were searched to April 25th 2024 for SRs of randomized controlled trials (RCTs) of yoga for health. Two reviewers independently screened studies against predefined eligibility criteria for SRs of yoga: the study (1) sought to comprehensively identify relevant RCTs on a yoga‐focused research question, (2) stated inclusion criteria for the studies, (3) searched at least one electronic database, and (4) reported the critical appraisal of the included studies. Data on SR characteristics and CoE were independently extracted from included SRs by two authors. Data analysis included frequencies and univariate and multivariable logistic regressions, conducted using R version 4.5.2. Statistical significance was set at p < .05. Results A total of 250 SRs published between 2005 and 2025 were included. More than a third of the SRs (88/250; 35.2%) were published by authors based in Asia, America and Europe. Among the 246 reviews with included RCTs, 39 (16%, 95% CI = 11.8%–20.9%) assessed CoE for at least one effect estimate. Among reviews that assessed CoE, GRADE was the most frequently used framework (35/39, 89.7%). In multivariable analysis, CoE assessment was independently associated with presence of a meta‐analysis (OR = 3.01, 95% CI = 1.08–10.21), systematic‐review journal type (OR = 56.47, 95% CI = 5.33–1151.95), and use of a Cochrane risk‐of‐bias tool (OR = 12.17, 95% CI = 2.71–124.93). Region, journal impact factor, and study types included were not independently associated with CoE assessment after adjustment. Conclusion Approximately 16% of SRs of yoga for health assessed CoE. Since this study was limited to English‐language studies identified from major databases through 2024, we cannot be certain whether this result reflects true levels of CoE assessment across all SRs of yoga for health conducted to date. Strengthening reviewer training, instituting journal requirements, and encouraging adherence to PRISMA and GRADE guidelines might increase the future percentage of yoga SRs assessing CoE.
ABSTRACT The Cochrane risk‐of‐bias tool (RoB 2) is used to assess the risk of bias in randomized trials in systematic reviews. However, studies have shown that the tool is often incorrectly used and published reviews rarely include detail regarding how it was applied or offer reflections on using it. This paper is a transparent applied case example with lessons learned and limitations of using RoB 2 in a (non‐Cochrane) systematic review evaluating the effectiveness of remote therapies for alcohol and drug use disorders. We share our reflections on aspects to plan in advance (pre‐selecting the outcome(s) and their results and piloting the tool); factors to consider when applying the tool (multiple outcomes within a study and navigating resources); overcoming common problems within specific domains (developing a review‐specific guidance document). Through our worked example, this article illustrates how review teams can engage with the RoB 2 tool within a specific review context to enhance its correct application.
ABSTRACT Introduction Global guidance provides evidence‐informed recommendations on total fat intake to prevent unhealthy weight gain in adults and children, however, evidence regarding total fat intake and other health outcomes, particularly non‐communicable diseases (NCDs), also exists. This scoping review aimed to map and identify gaps in this body of evidence. Methods Four databases were searched from 2014 to 2024 for published systematic reviews (SRs), in any population, assessing the effects or associations of reduced total fat intake (e.g., ≤ 30% total energy [TE]), compared with usual fat intake (e.g., > 30%TE), on pre‐specified health outcomes other than measures of unhealthy weight gain or acute post‐prandial outcomes. We screened independently and in duplicate. Data were extracted, checked, and narratively summarized inductively by health topic. Results Sixty‐seven SRs were included. Eight were fully aligned (all included studies addressed the scoping review question) and reported on health topics related to NCDs, for example, cancers (gastrointestinal, hematological), bone and joint health, gastrointestinal and neurological disorders, and cardiometabolic outcomes. Fifty‐nine SRs partially addressed the scoping review question, reporting additional topics, for example, cancers (breast, gynecological, urological, liver, skin), and respiratory and liver disorders. Almost one third of included SRs (n = 21) reported outcomes related to cardiometabolic risk and cardiovascular disease, obtained mostly from randomized controlled trial (RCT) evidence. Evidence for other topics was mostly observational. Overall, evidence was predominantly from high‐income countries, and in adults. Conclusions This scoping review maps available evidence on the relationships between total fat intake and health outcomes other than unhealthy weight gain and acute postprandial effects and demonstrated a heterogenous body of evidence, with several evidence gaps, predominantly relating to NCDs.
ABSTRACT Introduction The assessment of the certainty of evidence using GRADE requires evaluating inconsistency and imprecision. These domains are difficult to judge in a fully independent way. We performed a meta‐research study to propose an approach to disentangle inconsistency and imprecision based on decision thresholds. Methods We evaluated systematic reviews published in the Cochrane Database of Systematic Reviews. We included all meta‐analyses published in these reviews (i) with at least five primary studies, (ii) which did not correspond to subgroup analyses, and (iii) which had provided their results as continuous outcomes or as dichotomous outcomes with previously established decision thresholds. We recalculated each included meta‐analysis using the fixed‐ and the random‐effects models, retrieving metrics potentially expressing inconsistency or imprecision. This includes inconsistency indices, variables related to the confidence intervals (CI) of the meta‐analytical estimate and prediction intervals. Based on these variables, we performed factor analysis, hierarchical cluster analysis, and decision tree analysis. Results We assessed 8703 meta‐analyses. The factor analysis resulted in the identification of three factors: (i) one factor associated with inconsistency metrics, (ii) one factor associated with imprecision metrics, and (iii) one factor of metrics not clearly related to inconsistency or imprecision. Taken together, the analysis suggests an approach based on the number of decision thresholds crossed by the CI of the fixed‐ and random‐effects meta‐analytical estimates: the number of thresholds crossed by the CI of the fixed‐effect model informs about imprecision, while the difference in the number of thresholds crossed by the CIs of the random‐ versus the fixed‐effect models reflects inconsistency. Prediction intervals and inconsistency indices, such as the I2, can help detect cases in which inconsistency may be underestimated. Conclusion Our results may help those assessing the certainty in evidence to independently rate inconsistency and imprecision.
