Introduction
What is new?
Key findings- •
There was a broad-based failure of treatment recommendations for rheumatoid arthritis (RA) to address the steps that GRADE proposes to appraise evidence and develop high-quality transparent treatment recommendations.
- •
AGREE-II quality ratings were also low, with only nine guidelines rated as “Recommended for use.” Among these nine guidelines, GRADE steps were still poorly addressed.
What this adds to what was known?- •
Gaps in guideline quality have been demonstrated across many fields using AGREE-II. This study demonstrates that the steps recommended by GRADE are poorly addressed by RA guidelines and that although AGREE-II provides a broad assessment of quality, it lacks sufficient granularity to assess how well a guideline addresses GRADE's steps.
What is the implication and what should change now?- •
Neither AGREE-II nor GRADE provides a complete picture of guideline quality and both should be considered when developing or appraising the quality of guidelines.
- •
Further collaboration between AGREE and GRADE should be encouraged to develop harmonized standards for guideline development and quality appraisal.
At their best, clinical practice guidelines (CPG) provide an efficient mechanism for translating evidence and evidence-based consensus opinion to end users in an easy accessible format [1]. Unfortunately, the quality of many guidelines is poor [2]. Guideline development has been hampered by a myriad of methodological approaches, inconsistent reporting, and heterogeneous evidence-grading systems. In response, several groups have focused on obtaining international consensus on key aspects of guideline development and reporting. One such group, Grading of Recommendations Assessment, Development and Evaluation (GRADE), was formed in the year 2000. They focused their efforts on developing a broadly applicable systematic method for appraising and grading the strength of evidence and formulating and grading recommendations. The product of their efforts, the GRADE framework, was introduced in 2003–2004 [3], [4]. GRADE is more than just a grading system [5]. It outlines specific steps that should be taken before appraising the evidence, when appraising and summarizing the evidence, and when translating evidence summaries into treatment recommendations. GRADE has been adopted by the World Health Organization (WHO) and other guideline agencies and is becoming an international standard for the development of treatment recommendations.
Appraising evidence and developing treatment recommendations, however, are just part of formulating a clinical practice guideline. The Appraisal of Guidelines Research and Evaluation (AGREE) collaboration, also through international consensus, has developed a tool to allow users to assess the quality of the entire process of guideline development. The AGREE instrument has been recently updated to AGREE-II [6]. In total, AGREE-II includes 23 questions, grouped into six quality domains: Scope and Purpose; Stakeholder Involvement; Rigor of Development; Clarity of Presentation; Applicability; and Editorial Independence. Guidelines are also classified into an overall quality category and rated on a 7-point scale. AGREE-II is also intended to provide a methodological strategy for guideline development and to inform the reporting of information within guidelines [6]. It is designed to be applicable to multiple systems for developing guidelines and therefore, in comparison to GRADE, does not propose a specific series of steps that should be taken when developing treatment recommendations. AGREE-II has been widely adopted and is the standard tool to appraise guideline quality.
GRADE, in contrast to AGREE-II, is not intended as a tool to appraise guideline quality. The steps that GRADE outlines, however, should be highly relevant to an assessment of guideline quality, as they have been identified through a rigorous process and international consensus as key factors to consider when appraising evidence and developing treatment recommendations. The objective of this study was to determine how well current guidelines addressed each GRADE step and whether this could be adequately appraised by AGREE-II. Rheumatoid arthritis (RA) is an excellent candidate disease group as there are multiple and continuously evolving treatment choices with competing risks and benefits that need to be evaluated through a large volume of heterogeneous studies. GRADE outlines steps that are designed to assist in making these difficult judgments.