Quality assessment of systematic reviews and meta-analyses published in Saudi journals from 1997 to 2017

Discussion

This review revealed that SRs/MAs published in Saudi journals had a range of quality based on scoring of the articles using the AMSTAR and OQAQ scales. The tools indicated that some studies were of high quality, while others scored in the moderate and low ranges. Most of the weaknesses in these reviews were source of publication (gray literature or peer review), list of included or excluded studies, appropriate quality of the included studies used in final conclusions, the validity assessment of the included studies, comprehensive search for evidence, and selection bias assessment. Most SRs/MAs had comparable issues depending on the field of study. In quality assessments of SRs/MAs in hand surgery, the lowest scoring items were Q11 (conflict of interest included), Q1 (a priori design provided) and Q10 (likelihood of publication bias assessed).15 While in health literacy and cancer screening, the main issues were protocol registration, sources of literature searches, and a list of excluded articles.16,18 Moreover, SRs/MAs implemented a comprehensive literature search, included a statement of conflict of interest, and utilized a priori design. They also offered main conclusions and useful information related to the primary questions, the search strategy, and the characteristics of the included studies. These criteria were comparable to other studies, such as hand surgery systematic reviews, which scored the highest on Q3 (comprehensive literature search), Q7 (scientific quality assessed), and Q9 (methods used to combine the findings appropriate).15

Evidence based medicine is very popular in all specialties. Gastroenterology was the most common area investigated.8,20-23 In gastroenterology, EBM is part of the training and curriculum for many programs worldwide.8 In fact, a review published on the quality of gastroenterology articles in Saudi journals found that 80.7% were level IV based on Oxford’s level of evidence scale.23 The most common study designs were cross sectional (33.9%), followed by case reports (27.9%), and case series (18.8%).8 A total of 127 meta-analyses were included in this study.23 The overall compliance with the PRISMA statement was 20.8 ± 4.2 and the AMSTAR scale was 7.6 ± 2.4. The following items were inadequately reported, such as: protocol or registration, explaining the methods; results of additional analyses, duplicating study selections and data extraction, list of included or excluded studies. Articles published in latest years revealed a significantly better methodological quality than those issued in the preceding years.23 Based on these results, editors should encourage authors to prepare high quality of SRs/MAs and they should check the quality of each manuscript before any decision.

Moreover, a growing number of MAs/SRs have been issued in all areas. However, quality is always a concern. In an SR that investigated 127 MAs in main principal gastroenterology and hepatology journals, compliance with the AMSTAR checklist was moderate (7.6 ± 2.4) while the median for hand surgery was approximately 7. A MeaSurement Tool to Assess Systematic Reviews and OQAQ scales are a valid and a reliable tool to evaluate the quality of SRs/MAs. Following the PRISMA (preferred reporting items for SRs/MAs) statement, developed in 2009, is another way to improve the quality.27

There are several reasons that may contribute to the occurrence of low quality articles in Saudi journals. One is the low or no impact factor in Saudi journals that make the articles unappealing to well-known investigators.8 Other factors that may affect the quality of the articles are the background of the editors, process of publication, possibility of publishing supplemental materials, number of editions per year, mandatory Arabic summary, and presence of a statistician or epidemiologist on the board.

There are several limitations to this review. It would be more accurate to analyze the quality of SRs in the journals indexed by MEDLINE, which is currently one of the few evidence-based biomedical databases. Some high-quality Saudi medical journals were not processed for this article accidently (Neurosciences) or because it was indexed in 2018 (Journal of infection and public health). However, few systemic reviews were published in these journals, which the overall effect might be minimal.

In conclusion, the quality of SRs/MAs published in Saudi journals was distributed in all categories (low, medium, and high) using OQAQ and AMSTAR scales. This study shows that it is possible to improve the methodological quality of MAs in Saudi journals. Using several published and validated tools, and ensuring that authors, journal editors, and readers conduct evaluations that are more critical would aid in the improvement. Specifically, greater attention could be paid to the reporting methodology of SRs/MAs to have this category as one of the top methods to obtain reliable evidence in any medical field.