AbstractIntroductionOne of the main tasks in information retrieval is the development of Boolean search strategies for systematic searches in bibliographic databases. This includes the identification of free‐text terms and controlled vocabulary. IQWiG has previously implemented its objective approach for search strategy development using a fee‐based text analysis software. However, this implementation is not fully automated, due to a lack of technical options. The aim of our project was to develop a text analysis tool for the development of Boolean search strategies using R.MethodsWe adopt an incremental approach to software development, with the first goal being to develop a minimum viable product for the previously defined use cases. To create an interactive user interface, we use the shiny framework.ResultsOur newly developed shiny app searchbuildR is a text analysis tool with a point‐and‐click user interface, that automatically extracts and ranks terms from titles, abstracts, and MeSH terms of a given test set of PubMed records. It returns searchable, interactive tables of free‐text and MeSH terms. Each free‐text term can also be viewed within its original context in the full titles and abstracts or in a user‐defined word window. In addition, 2‐word combinations are extracted and also provided as an interactive table to help the user identify free‐text term combinations, that can be searched with proximity operators in Boolean searches. The results can be exported to a CSV file. The new implementation with searchbuildR was evaluated by validating the text analysis results against the results of the previously used fee‐based software.ConclusionsQWiG has developed the shiny app searchbuildR to support the development of search strategies in systematic reviews. It is open source and can be used by researchers and other information specialists without extensive R or programming skills. The package code is openly available on GitHub at www.github.com/IQWiG/searchbuildR.

Background:Cow's milk allergy (CMA) is the most complex and common food allergy in infants. Elimination of cow's milk from the diet and replacement with a specialized formula for infants with cow's milk allergy who cannot be breastfed is an established approach to minimize the risk of severe allergic reactions while avoiding nutritional deficiencies. Given the availability of multiple options, such as extensively hydrolyzed cow's milk-based formula (eHF-CM), aminoacid formula (AAF), hydrolyzed rice formula (HRF), and soy formula (SF), there is some uncertainty regarding which formula might represent the most suitable choice with respect to health outcomes. The addition of probiotics to a specialized formula has also been proposed as a potential approach to possibly increase the benefit. We systematically reviewed specialized formulas for infants with CMA to inform the updated World Allergy Organization (WAO) DRACMA guidelines. Objective:To systematically review and synthesize the available evidence about the use of specialized formulas for the management of individuals with CMA. Methods:We searched from inception PubMed, Medline, Embase, the Cochrane Central Register of Controlled Trials (CENTRAL), and the websites of selected allergy organizations, for randomized and non-randomized trials of any language investigating specialized formulas with or without probiotics. We included all studies irrespective of the language of the original publication. The last search was conducted in January 2024. We synthesized the identified evidence quantitatively or narratively as appropriate and summarized it in the evidence profiles. We conducted this review following the PRISMA, Cochrane methods, and the GRADE approach. Results:We identified 3558 records including 14 randomized trials and 7 observational studies. Very low certainty evidence suggested that in infants with IgE-mediated CMA, eHF-CM, compared with AAF, might have higher probability of outgrowing CMA (risk ratio (RR) 2.32; risk difference (RD) 25 more per 100), while showing potentially lower probability of severe vomiting (RR 0.12, 95% CI 0.02 to 0.88; RD 23 fewer per 100, 95% CI 3 to 26) and developing food protein-induced enterocolitis syndrome (FPIES) (RR 0.15, 95% CI 0.03 to 0.82; RD 34 fewer per 100, 95% CI 7 to 39). We also found, however, that eHF-CM might be inferior to AAF in supporting a physiological growth, with respect to both weight (-5.5% from baseline, 95%CI -9.5% to -1.5%) and length (-0.7 z-score change, 95%CI -1.15 to -0.25) (very low certainty). We found similar effects for eHF-CM, compared with AAF, also in non-IgE CMA. When compared with SF, eHF-CM might favor weight gain for IgE CMA infants (0.23 z-score change, 95%CI 0.01 to 0.45), and tolerance acquisition (RR 1.86, 95%CI 1.03 to 3.37; RD 27%, 95%CI 1%-74%) for non-IgE CMA (both at very low certainty of the evidence (CoE)). The comparison of eHF-CM vs. HRF, and HRF vs. SF, showed no difference in effect (very low certainty). For IgE CMA patients, low certainty evidence suggested that adding probiotics (L. rhamnosus GG, L. casei CRL431 and B. lactis Bb-12) might increase the probability of developing CMA tolerance (RR 2.47, 95%CI 1.03 to 5.93; RD 27%, 95%CI 1%-91%), and reduce the risk of severe wheezing (RR 0.12, 95%CI 0.02 to 0.95; RD -23%, 95%CI -8% to -0.4%). However, in non-IgE CMA infants, the addition of probiotics (L. rhamnosus GG) showed no significant effect, as supported by low to very low CoE. Conclusions:Currently available studies comparing eHF-CM, AAF, HRF, and SF provide very low certainty evidence about their effects in infants with IgE-mediated and non-IgE-mediated CMA. Our review revealed several limitations in the current body of evidence, primarily arising from concerns related to the quality of studies, the limited size of the participant populations and most importantly the lack of diversity and standardization in the compared interventions. It is therefore imperative for future studies to be methodologically rigorous and investigate a broader spectrum of available interventions. We encourage clinicians and researchers to review current World Allergy Organization (WAO) Diagnosis and Rationale for Action against Cow's Milk Allergy (DRACMA) Guidelines for suggestions on how to use milk replacement formulas in clinical practice and what additional research would be the most beneficial.

Background: Systematic reviews (SRs) are usually conducted by a highly specialized group of researchers. The routine involvement of methodological experts is a core methodological recommendation. The present commentary describes the qualifications required for information specialists and statisticians involved in SRs, as well as their tasks, the methodological challenges they face, and potential future areas of involvement. Tasks and qualifications: Information specialists select the information sources, develop search strategies, conduct the searches, and report the results. Statisticians select the methods for evidence synthesis, assess the risk of bias, and interpret the results. The minimum requirements for their involvement in SRs are a suitable university degree (e.g., in statistics or librarian/information science or an equivalent degree), methodological and content expertise, and several years of experience. Key arguments: The complexity of conducting SRs has greatly increased due to a massive rise in the amount of available evidence and the number and complexity of SR methods, largely statistical and information retrieval methods. Additional challenges exist in the actual conduct of an SR, such as judging how complex the research question could become and what hurdles could arise during the course of the project. Conclusion: SRs are becoming more and more complex to conduct and information specialists and statisticians should routinely be involved right from the start of the SR. This increases the trustworthiness of SRs as the basis for reliable, unbiased and reproducible health policy, and clinical decision making.

This paper is part of a series of methodological guidance from the Cochrane Rapid Reviews Methods Group. Rapid reviews (RR) use modified systematic review methods to accelerate the review process while maintaining systematic, transparent and reproducible methods. In this paper, we address considerations for RR searches. We cover the main areas relevant to the search process: preparation and planning, information sources and search methods, search strategy development, quality assurance, reporting, and record management. Two options exist for abbreviating the search process: (1) reducing time spent on conducting searches and (2) reducing the size of the search result. Because screening search results is usually more resource-intensive than conducting the search, we suggest investing time upfront in planning and optimising the search to save time by reducing the literature screening workload. To achieve this goal, RR teams should work with an information specialist. They should select a small number of relevant information sources (eg, databases) and use search methods that are highly likely to identify relevant literature for their topic. Database search strategies should aim to optimise both precision and sensitivity, and quality assurance measures (peer review and validation of search strategies) should be applied to minimise errors.

Background Systematic literature screening is a key component in systematic reviews. However, this approach is resource intensive as generally two persons independently of each other (double screening) screen a vast number of search results. To develop approaches for increasing efficiency, we tested the use of text mining to prioritize search results as well as the involvement of only one person (single screening) in the study selection process. Method Our study is based on health technology assessments (HTAs) of drug and non-drug interventions. Using a sample size calculation, we consecutively included 11 searches resulting in 33 study selection processes. Of the three screeners for each search, two used screening tools with prioritization (Rayyan, EPPI Reviewer) and one a tool without prioritization. For each prioritization tool, we investigated the proportion of citations classified as relevant at three cut-offs or STOP criteria (after screening 25%, 50% and 75% of the citation set). For each STOP criterion, we measured sensitivity (number of correctly identified relevant studies divided by the total number of relevant studies in the study pool). In addition, we determined the number of relevant studies identified per single screening round and investigated whether missed studies were relevant to the HTA conclusion. Results Overall, EPPI Reviewer performed better than Rayyan and identified the vast majority (88%, Rayyan 66%) of relevant citations after screening half of the citation set. As long as additional information sources were screened, it was sufficient to apply a single-screening approach to identify all studies relevant to the HTA conclusion. Although many relevant publications ( n = 63) and studies ( n = 29) were incorrectly excluded, ultimately only 5 studies could not be identified at all in 2 of the 11 searches (1x 1 study, 1x 4 studies). However, their omission did not change the overall conclusion in any HTA. Conclusions EPPI Reviewer helped to identify relevant citations earlier in the screening process than Rayyan. Single screening would have been sufficient to identify all studies relevant to the HTA conclusion. However, this requires screening of further information sources. It also needs to be considered that the credibility of an HTA may be questioned if studies are missing, even if they are not relevant to the HTA conclusion.

What is new?Key findings• TMT searches reduced strategy development time compared to UP searches.• TMT searches were slightly less sensitive for simple and complex topics.What this adds to what is known?• Studied freely available TMTs to increase real world SR applicability.• TMTs useful adjuncts when timeliness is more important than comprehensiveness.What should change now?• Comparative TMT functionality studies are needed to develop “best practices.” Key findings• TMT searches reduced strategy development time compared to UP searches.• TMT searches were slightly less sensitive for simple and complex topics.What this adds to what is known?• Studied freely available TMTs to increase real world SR applicability.• TMTs useful adjuncts when timeliness is more important than comprehensiveness.What should change now?• Comparative TMT functionality studies are needed to develop “best practices.” We thank Hausner et al. for their commentary and reply to each of their points below. We are happy to agree it is entirely possible that our team would have achieved similar results if we had access to either of the fee-based software packages (i.e., SimStat/WordStat or the R Text Mining Package) used by Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (Institute for Quality and Efficiency in Health Care) information specialists in their research along with their training and support [1Hausner E. Guddat C. Hermanns T. Lampert U. Waffenschmidt S. Prospective comparison of search strategies for systematic reviews: an objective approach yielded higher sensitivity than a conceptual one.J Clin Epidemiol. 2016; 77: 118-124Abstract Full Text Full Text PDF PubMed Scopus (28) Google Scholar, 2Hausner E. Guddat C. Hermanns T. Lampert U. Waffenschmidt S. Development of search strategies for systematic reviews: validation showed the noninferiority of the objective approach.J Clin Epidemiol. 2015; 68: 191-199Abstract Full Text Full Text PDF PubMed Scopus (45) Google Scholar, 3Hausner E. Waffenschmidt S. Kaiser T. Simon M. Routine development of objectively derived search strategies.Syst Rev. 2012; 1: 19Crossref PubMed Scopus (78) Google Scholar]. Our interest lay in expanding the research base in this field by exploring the use of free text-mining tools (TMTs) available to evidence synthesis searchers and testing their performance on simple and complex topics (simple = one indication, one treatment clinical topics; complex = everything else) in a ‘real world’ environment, that is, in which searchers only have access to free tools for which no best-practice guidance currently exists [[4]Paynter R.A. Featherstone R. Stoeger E. Fiordalisi C. Voisin C. Adam G.P. A prospective comparison of evidence synthesis search strategies developed with and without text-mining tools.J Clin Epidemiol. 2021; 139: 350-360Abstract Full Text Full Text PDF PubMed Scopus (9) Google Scholar,[5]Paynter R.A. Fiordalisi C. Stoeger E. Erinoff E. Featherstone R. Voisin C. et al.A prospective comparison of evidence synthesis search strategies developed with and without text-mining tools. Agency for Healthcare Research and Quality (US), Rockville (MD)2021Crossref Google Scholar]. In our study, for both processes, the recall was generally very high (i.e., we are not saying the technology does not work) and perhaps with standardization, such as Hausner et al. describe, it could be even higher than our results. This initial foray into a different set of tools and using the creativity of information specialists to develop novel methods we hoped would lay some groundwork for future studies that may indeed establish best practices for these individual tools. Please observe our recently published short pros and cons article for more details on using TMTs [[6]Adam G.P. Paynter R. Development of literature search strategies for evidence syntheses: pros and cons of incorporating text mining tools and objective approaches.BMJ Evid Based Med. 2022; 28 (bmjebm-2021-111892)Google Scholar]. Our current research project is creating a user-friendly, comparative guide to search tools listed on the SR Toolbox website. Using standardized criteria we are evaluating availability, usability, and performance, along with providing some usage tips and tricks with the results presented in an easy-to-understand graphic. With so many tools available but often too little time to try them out, we hope our fellow information specialists will find this guide useful. We agree that the experience with TMTs and ongoing professional support are important ingredients in their successful use. We count ourselves among those evidence synthesis searchers who are not located in such an environment and thus wanted to investigate the performance of tools available to us and whether the time to learn a new tool paid off (even with no experience and no best-practices guidance available). We hope our brief comments have been useful to others regarding what works and what does not work so well. We agree that a representative seed set of citations is critical for generating a more accurate analysis. To our knowledge, there has not been any research on the effect of an unrepresentative (biased) seed set of citations on the development of searches. With the likely increase in the use of search tools, attention to this issue is critical in our opinion and the development of a risk of bias tool would be most welcomed. The requested information would indeed make an interesting additional study; however, we did not systematically collect these, so are unable to provide them. This area of research would potentially benefit from the use of a standard dataset of search strategies and relevant citations, such as that created by Scells et al. [[7]Scells H. Zuccon G. Koopman B. Deacon A. Azzopardi L. Geva S. A test collection for evaluating retrieval of studies for inclusion in systematic reviews.SIGIR. 2017; 17: 1237-1240Google Scholar], which could be used across evaluations to give more consistent results.

Background:Allergy to cow's milk is the most common food allergy in infants and it is usually outgrown by 5 years of age. In some individuals it persists beyond early childhood. Oral immunotherapy (OIT, oral desensitization, specific oral tolerance induction) has been proposed as a promising therapeutic strategy for persistent IgE-mediated cow's milk allergy. We previously published the systematic review of OIT for cow's milk allergy (CMA) in 2010 as part of the World Allergy Organization (WAO) Diagnosis and Rationale for Action against Cow's Milk Allergy (DRACMA) Guidelines. Objective:To systematically synthesize the currently available evidence about OIT for IgE-mediated CMA and to inform the updated 2022 WAO guidelines. Methods:We searched the electronic databases including PubMed, Medline, Embase, the Cochrane Central Register of Controlled Trials (CENTRAL), and the websites of selected allergy organizations. We included all studies irrespective of the language of the original publication. The last search was conducted in February 2021. We registered the protocol on Open Science Framework (10.17605/OSF.IO/AH2DT). Results:We identified 2147 unique records published between 2010 and 2021, including 13 randomized trials and 109 observational studies addressing cow's milk OIT. We found low-certainty evidence that OIT with unheated cow's milk, compared to elimination diet alone, increased the likelihood of being able to consume ≥150 ml of cow's milk in controlled settings (risk ratio (RR): 12.3, 95% CI: 5.9 to 26.0; risk difference (RD): 25 more per 100, 95% CI 11 to 56) as well as accidently ingest a small amount (≥5 ml) of cow's milk (RR: 8.7, 95% CI: 4.7 to 16.1; RD: 25 more per 100, 95% CI 12 to 50). However, 2-8 weeks after discontinuation of a successful OIT, tolerance of cow's milk persisted in only 36% (range: 20%-91%) of patients. OIT increased the frequency of anaphylaxis (rate ratio: 60.0, 95% CI 15 to 244; rate difference 5 more anaphylactic reactions per 1 person per year, 95% CI: 4 to 6; moderate evidence) and the frequency of epinephrine use (rate ratio: 35.2, 95% CI: 9 to 136.5; rate difference 268 more events per 100 person-years, 95% CI: 203 to 333; high certainty). OIT also increased the risk of gastrointestinal symptoms (RR 6.9, 95% CI 1.6-30.9; RD 28 more per 100, CI 3 to 100) and respiratory symptoms (RR 49.0, 95% CI 3.12-770.6; RD 77 more per 100, CI 62 to 92), compared with avoidance diet alone. Single-arm observational studies showed that on average 6.9% of OIT patients (95% CI: 3.8%-10%) developed eosinophilic esophagitis (very low certainty evidence). We found 1 trial and 2 small case series of OIT with baked milk. Conclusions:Moderate certainty evidence shows that OIT with unheated cow's milk in patients with IgE-mediated CMA is associated with an increased probability of being able to drink milk and, at the same time, an increased risk of serious adverse effects.

OBJECTIVES:Analytical frameworks are graphical representation of the key questions answered by a systematic review and can support the development of guideline recommendations. Our objectives were to a) conduct a systematic review to identify, describe and compare all analytical frameworks published as part of a systematic and guideline development process related to colorectal cancer (CRC), and b) to use this case study to develop guidance on how to conduct systematic reviews of analytical frameworks.METHODS:We developed a search strategy to identify eligible studies in Medline and Embase from 1996 until December 2020. We also manually searched guideline databases and websites to identify all guidelines and systematic reviews in CRC that used an analytical framework. We assessed the quality of the guidelines using the Appraisal of Guidelines for Research and Evaluation II tool. The systematic review was registered in International Prospective Register of Systematic Reviews, registration CRD42020172117.RESULTS:We screened 34,505 records and identified 1,166 guidelines and 3,127 systematic reviews on CRC of which five met our inclusion criteria. These five publications included four analytical frameworks in colorectal cancer (one update). We also describe our methodological approach to systematic reviews for analytical frameworks and underlying concepts for developing analytical framework using a bottom-up or top-down approach.CONCLUSION:Few guidelines and systematic reviews are utilizing analytical frameworks in the development of recommendations. Development of analytical frameworks should begin with a systematic search for existing analytical frameworks and follow a structured conceptual approach for their development to support guideline recommendations. Our methods may be helpful in achieving these objectives.

The history of European health technology assessment (HTA) goes back more than 30 years. Almost as old as HTA agencies themselves is the desire to achieve European collaboration. This gained further impetus with the establishment of the European Network of Health Technology Assessment (EUnetHTA) in 2006. In this context, the field of information management faced specific challenges. Although these services are an integral part of HTA and information specialists play a key role here, this field is often not adequately represented in the HTA agencies within EUnetHTA. Furthermore, the organization of HTA production, including the types of HTAs produced, as well as funding, varies considerably. In order to meet these different conditions, information specialists have created various products and defined processes. With the EUnetHTA guideline, a common methodological understanding for the production of rapid Relative Effectiveness Assessments now exists. Furthermore, the Standard Operating Procedures map the complex information retrieval processes within EUnetHTA in a hands-on manner. The newly established Information Specialist Network (ISN) will in future ensure that information specialists are involved in all EUnetHTA assessments and that the methods are applied consistently in all assessments. In addition, the steering committee of the ISN manages enquiries and can be contacted to discuss methodological issues. Major barriers such as heterogeneity in the daily work of the EUnetHTA members can only be overcome through more collaboration and training.

We are pleased to present this information retrieval issue. Our goal is to highlight the variety of tasks and organizational structures of information retrieval within heath technology assessment (HTA). This special issue was planned and organized by the HTAi Information Retrieval Group (IRG). The choice of publications in this issue reflects the versatility and high competence in information retrieval methodology that is needed as an information specialist. Furthermore, it provides insights into the daily challenges faced by information specialists.

Background: Literature searches underlie the foundations of systematic reviews and related review types. Yet, the literature searching component of systematic reviews and related review types is often poorly reported. Guidance for literature search reporting has been diverse and, in many cases, does not offer enough detail to authors who need more specific information about reporting search methods and information sources in a clear, reproducible way. This document presents the PRISMA-S (Preferred Reporting Items for Systematic reviews and Meta-Analyses literature search extension) checklist, and explanation and elaboration.Methods: The checklist was developed using a three-stage Delphi survey process, followed by a consensus conference and public review process.Results: The final checklist includes sixteen reporting items, each of which is detailed with exemplar reporting and rationale.Conclusions:The intent of PRISMA-S is to complement the PRISMA Statement and its extensions by providing a checklist that could be used by interdisciplinary authors, editors, and peer reviewers to verify that each component of a search is completely reported and, therefore, reproducible.

A retrospective analysis published by the German Institute for Quality and Efficiency in Health Care (IQWiG) in 2018 concluded that no filter for non‐randomized studies (NRS) achieved sufficient sensitivity (≥92%), a precondition for comprehensive information retrieval. New NRS filters are therefore required, taking into account the challenges related to this study type. Our evaluation focused on the development of study filters for NRS with a control group (“controlled NRS”), as this study type allows the calculation of an effect size. In addition, we assumed that due to the more explicit search syntax, controlled NRS are easier to identify than non‐controlled ones, potentially resulting in better performance measures of study filters for controlled NRS. Our aim was to develop study filters for identifying controlled NRS in PubMed and Ovid MEDLINE. We developed two new search filters that can assist clinicians and researchers in identifying controlled NRS in PubMed and Ovid MEDLINE. The reference set was based on 2110 publications in Medline extracted from 271 Cochrane reviews and on 4333 irrelevant references. The first filter maximizes sensitivity (92.42%; specificity 79.67%, precision 68.49%) and should be used when a comprehensive search is needed. The second filter maximizes specificity (92.06%; precision 82.98%, sensitivity 80.94%) and should be used when a more focused search is sufficient.

BACKGROUND:Decision-makers increasingly request rapid answers to clinical or public health questions. To save time, personnel, and financial resources, rapid reviews streamline the methodological steps of the systematic review process. We aimed to explore the validity of a rapid review approach that combines a substantially abbreviated literature search with a single-reviewer screening of abstracts and full texts using three case studies.METHODS:We used a convenience sample of three ongoing Cochrane reviews as reference standards. Two reviews addressed oncological topics and one addressed a public health topic. For each of the three topics, three reviewers screened the literature independently. Our primary outcome was the change in conclusions between the rapid reviews and the respective Cochrane reviews. In case the rapid approach missed studies, we recalculated the meta-analyses for the main outcomes and asked Cochrane review authors if the new body of evidence would change their original conclusion compared with the reference standards. Additionally, we assessed the sensitivity of the rapid review approach compared with the results of the original Cochrane reviews.RESULTS:For the two oncological topics (case studies 1 and 2), the three rapid reviews each yielded the same conclusions as the Cochrane reviews. However, the authors would have had less certainty about their conclusion in case study 2. For case study 3, the public health topic, only one of the three rapid reviews led to the same conclusion as the Cochrane review. The other two rapid reviews provided insufficient information for the authors to draw conclusions. Using the rapid review approach, the sensitivity was 100% (3 of 3) for case study 1. For case study 2, the three rapid reviews identified 40% (4 of 10), 50% (5 of 10), and 60% (6 of 10) of the included studies, respectively; for case study 3, the respective numbers were 38% (8 of 21), 43% (9 of 21), and 48% (10 of 21).CONCLUSIONS:Within the limitations of these case studies, a rapid review approach that combines abbreviated literature searches with single-reviewer screening may be feasible for focused clinical questions. For complex public health topics, sensitivity seems to be insufficient.

IntroductionIt is a challenge to stay up-to-date with the latest developments in information retrieval for health technology assessment (HTA). Summarized Research in Information Retrieval for HTA (SuRe Info) is a well-established open-access website with a selection of up-to-date key papers presented in summarized overviews. SuRe Info is maintained by the HTAi Interest Sub-Group on Information Resources; its main target group are information specialists. SuRe Info is updated twice a year by experienced information specialists. Publications on information retrieval methods are identified by running topic-specific search strategies in relevant databases. Publications fulfilling the SuRe Info inclusion criteria receive a structured abstract together with a brief critical appraisal prepared by one SuRe Info information specialist and checked by another. The key messages from the appraisals are summarized in topic-specific chapters.MethodsThe structure and work of SuRe Info is presented on the basis of the chapter “Value of using different search approaches”.ResultsThe chapter “Value of using different search approaches” was last updated in September 2018. It examines various alternative search approaches in bibliographic databases beyond the conventional Boolean search. According to this chapter, the most well-known approach, offering the most evidence, is citation searching (direct or indirect citation relationships). In contrast, little research has so far been conducted on other approaches such as full-text searches, automated retrieval methods or hand searching.ConclusionsSuRe Info is an important resource for information specialists to keep up-to-date with the literature on information retrieval. In particular when information specialists work alone and not within a larger team, it is necessary to rely on collaboration projects such as SuRe Info.

Stringent requirements exist regarding the transparency of the study selection process and the reliability of results. A 2-step selection process is generally recommended; this is conducted by 2 reviewers independently of each other (conventional double-screening). However, the approach is resource intensive, which can be a problem, as systematic reviews generally need to be completed within a defined period with a limited budget. The aim of the following methodological systematic review was to analyse the evidence available on whether single screening is equivalent to double screening in the screening process conducted in systematic reviews. We searched Medline, PubMed and the Cochrane Methodology Register (last search 10/2018). We also used supplementary search techniques and sources (“similar articles” function in PubMed, conference abstracts and reference lists). We included all evaluations comparing single with double screening. Data were summarized in a structured, narrative way. The 4 evaluations included investigated a total of 23 single screenings (12 sets for screening involving 9 reviewers). The median proportion of missed studies was 5% (range 0 to 58%). The median proportion of missed studies was 3% for the 6 experienced reviewers (range: 0 to 21%) and 13% for the 3 reviewers with less experience (range: 0 to 58%). The impact of missing studies on the findings of meta-analyses had been reported in 2 evaluations for 7 single screenings including a total of 18,148 references. In 3 of these 7 single screenings – all conducted by the same reviewer (with less experience) – the findings would have changed substantially. The remaining 4 of these 7 screenings were conducted by experienced reviewers and the missing studies had no impact or a negligible on the findings of the meta-analyses. Single screening of the titles and abstracts of studies retrieved in bibliographic searches is not equivalent to double screening, as substantially more studies are missed. However, in our opinion such an approach could still represent an appropriate methodological shortcut in rapid reviews, as long as it is conducted by an experienced reviewer. Further research on single screening is required, for instance, regarding factors influencing the number of studies missed.

In a recently published article, Quang et al. evaluate the quality of systematic reviews of health economic evaluations of interventions for hepatitis.1 As the authors of one of the systematic reviews included in the assessment,2 we would like to discuss the methods which were applied and which, in our opinion, seem to be inappropriate.