Evaluation of Grading of Recommendations Assessment, Development, and Evaluation Approach Adherence in Clinical Practice Guidelines: a Systematic Survey of Criteria and Their Applications

OBJECTIVES:Many clinical practice guidelines (CPGs) claim to follow the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach, yet their actual adherence remains uncertain. We aimed to identify and evaluate existing criteria for evaluating adherence to GRADE in CPGs and describe the extent of adherence revealed by their application. STUDY DESIGN AND SETTING:We systematically searched PubMed, Embase, Web of Science, and the Cochrane Library from January 2004 to November 2024, supplemented by manual reviews of the GRADE handbook and related websites. Eligible documents were those that either proposed or applied criteria for evaluating GRADE adherence. We summarized the criteria offered, evaluated the extent to which they meet standards for a structured instrument, and analyzed adherence findings. RESULTS:We identified 11 eligible documents: eight proposed evaluation criteria (four applying their own to evaluate CPGs) and three applied existing criteria. From these criteria, we derived 10 clearly discrete items that can be used to define good adherence to GRADE. The most common items addressed the criteria for assessing certainty of evidence (CoE) and criteria for making recommendations (seven of eight documents), while question framing and outcome selection and importance rating were less frequent (3/8). Among the eight documents proposing criteria, four failed to report their development methods and response options; furthermore, none conducted user testing or provided guidance manuals. The number of CPGs assessed in the included studies ranged from 4 to 86. Reported adherence to GRADE varied considerably across these studies; nevertheless, the median adherence rates were suboptimal for five of the 10 items. CONCLUSION:Existing criteria for assessing GRADE adherence are varied and suboptimal in usability. Evaluations based on these criteria reveal significant application gaps in key areas among CPGs that claim to use GRADE. These findings highlight the need to develop a structured, user-tested evaluation instrument and improve the implementation of GRADE in CPGs. PLAIN LANGUAGE SUMMARY:Many medical guidelines say that they use a trusted scientific method called GRADE to make their advice reliable. But we neither know if they actually follow it correctly nor were there good tools to check. We wanted to find out: 1) what checklists exist to audit GRADE use? and 2) what do those checklists tell us about how well guidelines really apply GRADE? We searched scientific databases and official GRADE sources for studies that either created or used checklists to evaluate GRADE in guidelines. We reviewed these checklists, checked their quality, and summarized what they found when applied. We found 11 relevant studies. The checklists they proposed were inconsistent and poorly designed; none were tested with real users or came with instructions. When used, they showed that many guidelines fail to properly apply key steps of the GRADE method, even though they claim to use it. For half of the important steps, less than 50% of guidelines followed them correctly. This means that current tools for checking GRADE use are unreliable, and many guidelines may not be as trustworthy as they claim. To protect patients and improve medical advice, we urgently need a better, user-friendly audit tool and more support to help guideline developers apply the GRADE method correctly.