Adding a "GRADE" to the quality appraisal of rheumatoid arthritis guidelines identifies limitations beyond AGREE-II

Objectives: To assess how well treatment recommendations for rheumatoid arthritis (RA) address Grading of Recommendations Assessment, Development and Evaluation (GRADE) steps and determine whether these steps can be adequately appraised using Appraisal of Guidelines Research & Evaluation II (AGREE-II).

Study design and setting: We systematically reviewed English-language treatment recommendations for the pharmacologic management of RA since 2000, assessed how well GRADE steps were addressed, rated AGREE-II quality, and compared the findings.

Results: GRADE steps were poorly addressed by the 44 included guidelines. Few guidelines discussed study limitations and/or risk of bias (23%), inconsistency (50%), indirectness (39%), imprecision (23%), or potential for publication bias (0%). Observational evidence was cited in 96% but rarely evaluated systematically. Only one guideline considered evidence on patients' preferences for health outcomes, and few provided an explicit justification for the strength of evidence or recommendation. The five GRADE steps that overlapped with AGREE-II questions were addressed more frequently (by 54-100% of guidelines) than the 13 GRADE steps not directly assessed by AGREE-II (0-50%). Among the nine guidelines rated as "Recommended for use" by AGREE-II, 8 of 13 GRADE steps were not addressed consistently by any guideline.

Conclusion: GRADE's steps are poorly addressed by RA recommendations. AGREE-II provides a broad assessment of quality but lacks sufficient granularity to assess how well a guideline addresses GRADE's steps.