Institute for Scientific Information-indexed biomedical journals of Saudi Arabia ================================================================================ * Dileep K. Rohra * Vikram K. Rohra * Peter Cahusac ## Their performance from 2007-2014 ## Abstract **Objectives:** To compare the journal impact factor (JIF) and Eigenfactor score (ES) of Institute for Scientific Information (ISI)-indexed biomedical journals published from the Kingdom of Saudi Arabia (KSA) over the last 8 years. **Methods:** This is a retrospective study, conducted at Alfaisal University, Riyadh, KSA from January to March 2016. The Journal Citation Reports of ISI Web of Knowledge were accessed, and 6 Saudi biomedical journals were included for analysis. **Results:** All Saudi journals have improved their IF compared with their baseline. However, the performance of the Journal of Pharmaceutical Sciences and Neurosciences has been exceptionally good. The biggest improvement in percent growth in JIF was seen in the Saudi Pharmaceutical Journal (approximately 887%) followed by Neurosciences (approximately 462%). Interestingly, the ES of all biomedical journals, except Saudi Journal of Gastroenterology and Saudi Medical Journal, increased over the years. The greatest growth in ES (more than 5 fold) was noted for Neurosciences and Saudi Pharmaceutical Journal. **Conclusion:** This study shows that the overall quality of all Saudi biomedical journals has improved in the last 8 years. Research is an essential component for academics. Thousands of biomedical journals are indexed by Medline; which is considered as the largest portal of the scientific publications. The US National Library of Medicine, which operates Medline has a set of guiding criteria for the inclusion of journals into the database.1 Another related issue is that of journal impact factor (JIF). The JIF of a scientific journal is the ratio of the number of citations found in the 2 preceding years of articles published and divided by the number of citable items published in the same 2 years.2 The JIF is calculated and awarded by Thomson Reuters in their annual journal citation reports (JCR). The JCR provides quantitative tools for ranking, categorizing, and comparing journals. They have their own, but overlapping criteria with Medline for indexing journals.3 Interestingly, JIF is not available for all the journals indexed in Medline. For decades, the JIF has been used as an indicator for the relative importance of a journal and has emerged as an important parameter on which many universities, research institutes, and funding agencies assess the individual performances of scientists using the JIF of their publications.4 Even authors look at the JIF, while choosing a journal for the submission of their work.5 However, various concerns have been raised by the scientific community with regards to the flaws in calculating the JIF.6,7 These inherent flaws can even be manipulated to increase the JIF for example, by changing the number of publication output.8 Another important concern on the calculation of JIF has been the focus on the number of citations without taking into account the significance of the citations and other factors. More recently, a relatively more robust and acceptable metric, Eigenfactor score (ES) is being used to represent the significance of the journal. The Kingdom of Saudi Arabia (KSA) is an important country in the Middle East and geographically is the largest in the Gulf region, with an estimated population of 30.77 million.9 The history of science and education is not very old in this country. However, the Saudi government recognizes the significance of education, research, science, and technology in economic transformation, and encourages research and education in the country.10 This encouragement has led to significant academic and research activities, launching of various scientific journals, and growth of publications.10 In our previous study,11 we measured the research output in terms of quantity and quality in KSA over the last 5 years. However, it would be equally interesting to measure the performance of the Saudi biomedical journals. Therefore, this study was conducted with the main objective of comparing the JIF and ES of biomedical journals published from KSA over the last 8 years. The secondary objective was to compare the performance of Saudi biomedical journals with their peers in the same category. ## Methods This is a retrospective study, conducted at Alfaisal University, Riyadh, KSA from January to March, 2016. The Journal Citation Reports (JCR) of the Institute for Scientific Information (ISI) Web of Knowledge were accessed. Using the option of “Select country/region”, Saudi Arabia was selected. The search generated 10 journals published from KSA, which were indexed by ISI. This study focused on biomedical journals; therefore 4 journals specializing in chemistry, geoscience, and mathematics were excluded. The remaining 6 Saudi biomedical journals are presented in **Table 1**. View this table: [Table 1](http://smj.org.sa/content/37/11/1251/T1) Table 1 List of all Institute for Scientific Information-indexed Saudi biomedical journals analysed along with their categories. The JCR from 2007-2014 for these 6 biomedical journals were accessed to retrieve the data on JIF, Eigenfactor score and ranking of the journals in their respective category. The JIF was not available for 4 journals in 2007, 3 journals in 2008 and 2009, and 2 journals in 2010-2012. Absolute annual changes in JIF were calculated using the first year of assignment of JIF as the baseline. The number of journals published in the category “Medicine, General, and Internal” varied widely over the study years; hence, the ranking in the category. It was noticed that the analysis of the data are only on the absolute JIF of 2 Saudi medical journals (Annals of Saudi Medicine and Saudi Medical Journal), which may not reflect their performance over the years. Therefore, in order to assess the performance of these 2 Saudi journals in comparison with other journals in this category, the JIF of all other journals were retrieved and the relative performance of Saudi medical journals was compared, creating a new and unique parameter. The mean impact factor (IF) of all the journals in the category of “Medicine, General, and Internal” was calculated for each year from 2007-2014. The absolute JIF of Saudi medical journals in a particular year was subtracted from the mean IF of the category of the same year. This yielded a difference in the IF between the Saudi journal and the mean of all the journals in that category for each year. ### Statistical analysis All the extracted data from the JCR were entered into Microsoft Excel 2013 and then transferred to the Statistical Package for Social Sciences version 15 (SPSS Inc., Chicago, IL, USA) for statistical analysis and plotting. ## Results The trajectory of IF and the ranking in their respective category of all 6 biomedical journals published from KSA is presented in **Table 2**. It can be observed that the pattern of changes in the JIF and the rankings are not the same for all journals. As shown in **Figure 1A**, all Saudi journals have improved their IF compared with their baseline. However, the performance of the Journal of Pharmaceutical Sciences and Neurosciences has been exceptionally good. These 2 journals have consistently improved their IF over the last 8 years. Interestingly, peaks and troughs were noted for the Annals of Saudi Medicine. A growth in its IF was observed until the year 2012, and then declined sharply over the next 2 years. Nevertheless, this journal has maintained its IF compared with its baseline year of 2007, in spite of these troughs. A consistent but slow growth in the Saudi Medical Journal was observed over the last 8 years. Two journals, the Saudi Journal of Biological Sciences and the Saudi Journal of Gastroenterology have been assigned JIF in 2013 by ISI. The IF of the Saudi Journal of Biological Sciences has sharply increased in these recent years; whereas that of the Saudi Journal of Gastroenterology received high JIF in the year 2013, and then slightly decreased in the subsequent year. The percentage growth in the JIF of all Saudi journals compared with their baseline is presented in **Figure 1B**. As is evident from this figure, all Saudi journals except the Saudi Journal of Gastroenterology markedly improved their IF. The biggest improvement was seen in the Saudi Pharmaceutical Journal (approximately 887%) followed by the Neurosciences (approximately 462%). View this table: [Table 2](http://smj.org.sa/content/37/11/1251/T2) Table 2 Trajectory of JIF and the ranking of Saudi journals in their respective categories from the years 2007-2014. ![Figure 1](http://smj.org.sa/https://smj.org.sa/content/smj/37/11/1251/F1.medium.gif) [Figure 1](http://smj.org.sa/content/37/11/1251/F1) Figure 1 Year-wise impact factor (IF) of Saudi biomedical journals from 2007-2014 **(A)**, and percent change in the IF from the baseline year 2007 in year 2014 **(B)**. ASM - Annals of Saudi Medicine, SJBS - Saudi Journal of Biological Sciences, SJG - Saudi Journal of Gastroenterology, SMJ - Saudi Medical Journal, SPJ - Saudi Pharmaceutical Journal Next, the performance of 2 Saudi medical journals; Saudi Medical Journal and Annals of Saudi Medicine were evaluated in comparison with other journals in the same category. As shown in **Table 3**, the difference between the JIF of Saudi Medical journal and the mean of all the remaining journals belonging to the category “Medicine, General, and Internal” fluctuated continuously from 2007 to 2014. As far as Annals of Saudi Medicine is concerned, this difference kept on decreasing until 2012; thereafter, it started to increase. However, for both medical journals, this difference is still low compared with their baseline year 2007. **Figure 2** depicts the same data, which shows that there is a statistically significant decreasing trend only from 2007 to 2012 (r2>0.78, *p*<0.019). For each of the 2 journals, a quadratic relationship fits best (r2=0.863 and 0.877, *p*<0.01), which describes a decrease until 2012 and then shows a return to higher values up to 2014. View this table: [Table 3](http://smj.org.sa/content/37/11/1251/T3) Table 3 Comparative analysis of Saudi medical journals with other journals in the same category. ![Figure 2](http://smj.org.sa/https://smj.org.sa/content/smj/37/11/1251/F2.medium.gif) [Figure 2](http://smj.org.sa/content/37/11/1251/F2) Figure 2 Year-wise difference in the impact factor (IF) between the Saudi Medical Journal and the Annals of Saudi Medicine, and that of the mean IF of all the other journals in the category “Medicine; General, and Internal”. **Figure 3A** shows the trend of ES from the years 2007-2014. Interestingly, the ES of all biomedical journals, except the Saudi Journal of Gastroenterology and the Saudi Medical Journal, increased over the years. While the ES of Saudi Journal of Gastroenterology essentially remained the same over the last 2 years, it decreased markedly for the Saudi Medical Journal. When the change in the ES from the baseline year of each journal was calculated, it was consistent with the change in the JIF except for the Saudi Medical Journal (**Figure 3B**). For this journal, the ES decreased in 2014 compared with that in 2007 in spite of the increase in the JIF. The greatest growth in ES (more than 5 fold) was noted for the Neurosciences and Saudi Pharmaceutical Journal. ![Figure 3](http://smj.org.sa/https://smj.org.sa/content/smj/37/11/1251/F3.medium.gif) [Figure 3](http://smj.org.sa/content/37/11/1251/F3) Figure 3 Year-wise Eigenfactor score (ES) of Saudi biomedical journals from 2007-2014 **(A)** and percent change in the ES from the baseline year 2007 in year 2014 **(B)**. ASM - Annals of Saudi Medicine, SJBS - Saudi Journal of Biological Sciences, SJG - Saudi Journal of Gastroenterology, SMJ - Saudi Medical Journal, SPJ - Saudi Pharmaceutical Journal ## Discussion Bibliometric analysis of the publications and journals is being increasingly used to measure the quality and quantity of publications in a specific area, country, or region including the Arab world.11-17 Before the introduction of the JIF, the quality of the journal was gauged arbitrarily, based on the parameters like how long it was established, reputation of that journal, the strictness of the peer review, and the acceptance rate. With the assignment of a numerical value to represent the quality of a journal, fierce competition among the journals has been witnessed. In the exact words of Smith (2006), “*Now editors break open bottles of champagne if their impact factor rises by a tenth of a decimal point or burst into tears if it falls. They build their editorial strategies around increasing their impact factors. Authors, meanwhile, can quote the impact factors of the major journals and use them when deciding where to submit their papers*.”18 There are many concerns raised regarding the usage of the JIF.19,20 These are mainly related to the evaluation of individual publications as surrogate markers for their quality. Such usage was never intended. Garfield, the person who coined the JIF himself has warned against employing the impact factor for the evaluation of individual articles and scientists.4 It is meant to be the measure of journal performance. This is exactly how it has been employed in this study. Lately, many reports have been published, which have analysed the trends of bibliometric parameters including JIF over a certain period of time. These include time-dependent changes in trends of the journals belonging to certain fields or countries.21-23 These types of studies provide a bird’s eye view of biomedical research and document performance of scientific journals in a particular region or category.24 This study has critically analysed the performance of biomedical journals published from KSA using bibliometric parameters. It explores the bibliographical trajectory of Saudi journals indexed with ISI Web of Science. This study looked mainly at 3 parameters as surrogate of time-dependent performance of Saudi journals, namely, JIF, ranking of the journals in their respective category, and ES. The findings of this study evidently show that the Saudi biomedical journals are consistently improving their quality in terms of JIF and ES. However, the outstanding growth in the JIF and ES was noted for 2 journals; Journal of Pharmaceutical Sciences and Neurosciences. The performance of the Journal of Pharmaceutical Sciences was exceptional in the sense that It was first assigned an IF in 2010, when it ranked 250 out of 252 journals in the category of pharmacy and pharmacology. Remarkably, this journal ranked 194 out of 255 journals in the year 2014. This may be due to the fact that this journal is the official organ of the one of the most active scientific societies of KSA namely, Saudi Pharmaceutical Society, and many Saudi researchers in pharmaceutical sciences have opted to publish in this journal.17 The ranking of the 2 medical journals; Saudi Medical Journal and Annals of Saudi Medicine was impossible to evaluate since the number of journals in their category was highly variable over the study years. To control for this variability, a new parameter was created based on the difference between the IF of the journal in question and the mean of the rest of the journals in that category “Medicine, General, and Internal”, as previously mentioned. This analysis revealed that the difference between the JIF of Saudi medical journals and the mean of all the remaining journals decreased significantly until 2012, thereafter, this statistically significant difference disappeared in the last 2 years. One plausible interpretation of this finding is that although the IF of the 2 medical journals has increased compared with baseline year 2007, other journals in the same category have also increased their IF correspondingly; hence, no improvement in the ranking was observed. Despite the strength of the JIF, several limitations of this measure must be taken into account. One argument against the use of JIF as a measure of quality is the variability of JIF according to the speciality. This anomaly was taken care of in this study by comparing the JIF of each journal with itself and the contemporary journals in the same category. Effectively, each journal served as its own control over the last 8 years. Another concern raised is inflation of JIF by encouraging authors to self-cite.25 Therefore, in addition to JIF, we also analyzed the ES to circumvent this anomaly. Analysis of the performance of the Saudi journals using ES as a parameter revealed some interesting data. All journals showed remarkable growth except the Saudi Medical Journal and the Saudi Journal of Gastroenterology. We have the data of Saudi Journal of Gastroenterology for only 2 years and essentially, there was no difference of the ES over the 2 years. However, the ES of the Saudi Medical Journal consistently continued to decrease from the year 2007 until 2014 despite the increase in the JIF (compare **Figure 1A** with **3A**). This finding reinforces the notion that JIF alone is not a suitable measure to gauge the quality of a journal. It needs to be combined with more robust measure like the ES. Although the results of the present study show that there is an improvement in the JIF, ranking and ES of Saudi biomedical journals over the course of the last 8 years, this change is slow and somewhat inconsistent. The implication of this study is that the data generated will serve as a baseline for future studies evaluation of the performance of Saudi journals in the coming years, and compare them with other regional and international journals. In order to raise the standards of Saudi biomedical journals, a proactive role of the editors is of paramount importance. The journal impact factor is calculated from the equation that defines JIF as the number of citations received by a journal in a particular year for articles published during the last 2 years divided by the number of “citable” articles published by the journal during the same 2 years.26 The strategies formulated by the editors mainly take advantage of these determinants by either increasing the nominator or decreasing the denominator of the formula as reviewed by Falagas and Alexiou.27 Another study28 showed the unique and innovative measures taken by the top medical journal editors for raising the IF of their journals. This included personal approaches to productive and potentially citable research groups for submissions, improving services for authors and media publicity.28 The impact of media coverage on the number of citation and hence, the JIF was also reported by Phillips et al.29 As with other bibliometric studies, this study also has some inherent limitations. The foremost is the inclusion of those Saudi journals, which are indexed in the ISI Web of Science and whose JCR is available. There are other several journals published from KSA, which are neither indexed in Medline nor ISI. Since the main objective of the study was to document the JIF and ES as surrogate markers for quality of Saudi journals over time, this was not possible for journals having no bibliometric data. Another limitation is that the reasons underlying the observed changes in the JIF over the study period have not been analysed. It might have been due to changes in the citations or changes in the publication output, which are the main determinants of a JIF. Alternatively, there might have been the changes in the publication of review articles, which have the potential to be highly cited, and thus, increase the JIF drastically.30 In conclusion, this study shows that the overall quality of many Saudi biomedical journals has improved in the last 8 years, and the highest improvement in the IF and ES was observed for the Saudi Pharmaceutical Journal and Neurosciences. ## Footnotes * **Disclosure.** Authors have no conflict of interest, and the work was not supported or funded by any drug company. * Received June 19, 2016. * Accepted August 17, 2016. * Copyright: © Saudi Medical Journal This is an open-access article distributed under the terms of the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. ## References 1. National Institutes of Health, US National Library of Medicine Journal Selection for MEDLINE®Indexing at NLM, Available from: [https://www.nlm.nih.gov/pubs/factsheets/j\_sel\_faq.html](https://www.nlm.nih.gov/pubs/factsheets/j_sel_faq.html). Updated 2016 June 1, Accessed 2016 April 17. 2. Bornmann L, Marx W (2016) The journal Impact Factor and alternative metrics: A variety of bibliometric measures has been developed to supplant the Impact Factor to better assess the impact of individual research papers. EMBO Rep 17:1094–1097. [FREE Full Text](http://smj.org.sa/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NToiZW1ib3IiO3M6NToicmVzaWQiO3M6OToiMTcvOC8xMDk0IjtzOjQ6ImF0b20iO3M6MjA6Ii9zbWovMzcvMTEvMTI1MS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 3. Testa J The Thomson Reuters journal selection process, Available from: [http://wokinfo.com/essays/journal-selection-process/](http://wokinfo.com/essays/journal-selection-process/). Updated 2016 July 18, Accessed 2016 April 18. 4. Garfield E (2006) The history and meaning of the journal impact factor. JAMA 295:90–93. [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1001/jama.295.1.90&link_type=DOI) [PubMed](http://smj.org.sa/lookup/external-ref?access_num=16391221&link_type=MED&atom=%2Fsmj%2F37%2F11%2F1251.atom) [Web of Science](http://smj.org.sa/lookup/external-ref?access_num=000234381100025&link_type=ISI) 5. Kurdi MS, Ramaswamy AH, Lokare L, Sutagatti JG (2015) Current views and practice of faculty members and consultants regarding ‘Publications in India’: A cross-sectional study. Indian J Anaesth 59:794–800. 6. Ogden TL, Bartley DL (2008) The ups and downs of journal impact factors. Ann Occup Hyg 52:73–82. [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1093/annhyg/men002&link_type=DOI) [PubMed](http://smj.org.sa/lookup/external-ref?access_num=18316351&link_type=MED&atom=%2Fsmj%2F37%2F11%2F1251.atom) [Web of Science](http://smj.org.sa/lookup/external-ref?access_num=000253830400001&link_type=ISI) 7. Azer SA, Holen A, Wilson I, Skokauskas N (2016) Impact factor of medical education journals and recently developed indices: Can any of them support academic promotion criteria? J Postgrad Med 62:32–39. 8. Kiesslich T, Weineck SB, Koelblinger D (2016) Reasons for Journal Impact Factor Changes: Influence of changing source items. PLoS One 22(11):e0154199. 9. General Authority of Statistics, Kingdom of Saudi Arabia, Available from: [http://www.stats.gov.sa/en/english](http://www.stats.gov.sa/en/english). Accessed 2016 April 14. 10. Alshumaimri A, Aldridge T, Audretsch DB (2010) The university technology transfer revolution in Saudi Arabia. The Journal of Technology Transfer 35:585–596. 11. Abu-Dawas RB, Mallick MA, Hamadah RE, Kharraz RH, Chamseddin RA, Khan TA, et al. (2015) Comparative analysis of quantity and quality of biomedical publications in Gulf Cooperation Council countries from 2011-2013. Saudi Med J 36:1103–1109. [Abstract/FREE Full Text](http://smj.org.sa/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Mzoic21qIjtzOjU6InJlc2lkIjtzOjk6IjM2LzkvMTEwMyI7czo0OiJhdG9tIjtzOjIwOiIvc21qLzM3LzExLzEyNTEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 12. Sweileh WM, Zyoud SH, Sawalha AF, Abu-Taha A, Hussein A, Al-Jabi SW (2013) Medical and biomedical research productivity from Palestine, 2002-2011. BMC Res Notes 6:41. [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1186/1756-0500-6-41&link_type=DOI) [PubMed](http://smj.org.sa/lookup/external-ref?access_num=23375070&link_type=MED&atom=%2Fsmj%2F37%2F11%2F1251.atom) 13. Sweileh WM, Zyoud SH, Al-Jabi SW, Sawalha AF (2015) Contribution of Arab countries to breast cancer research: comparison with non-Arab Middle Eastern countries. BMC Womens Health 15:25. 14. Benamer HT, Bredan A, Bakoush O (2009) Scientific publication productivity of Libyan medical schools: a bibliometric study of papers listed in PubMed, 1988-2007. Educ Health (Abingdon) 22:310. [PubMed](http://smj.org.sa/lookup/external-ref?access_num=20029754&link_type=MED&atom=%2Fsmj%2F37%2F11%2F1251.atom) 15. Tadmouri GO, Tadmouri NB (2002) Biomedical research in the Kingdom of Saudi Arabia (1982-2000). Saudi Med J 23:20–24. [PubMed](http://smj.org.sa/lookup/external-ref?access_num=11938358&link_type=MED&atom=%2Fsmj%2F37%2F11%2F1251.atom) 16. Zyoud SH, Al-Jabi SW, Sweileh WM (2015) Scientific publications from Arab world in leading journals of Integrative and complementary medicine: a bibliometric analysis. BMC Complement Altern Med 15:308. 17. Alhaider I, Mueen Ahmed KK, Gupta BM (2015) Pharmaceutical research in the Kingdom of Saudi Arabia: A scientometric analysis during 2001-2010. Saudi Pharm J 23:215–222. 18. Smith R (2006) Commentary: the power of the unrelenting impact factor--is it a force for good or harm? Int J Epidemiol 35:1129–1130. [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1093/ije/dyl191&link_type=DOI) [PubMed](http://smj.org.sa/lookup/external-ref?access_num=16987843&link_type=MED&atom=%2Fsmj%2F37%2F11%2F1251.atom) [Web of Science](http://smj.org.sa/lookup/external-ref?access_num=000241429200005&link_type=ISI) 19. Pendlebury DA (2009) The use and misuse of journal metrics and other citation indicators. Arch Immunol Ther Exp (Warsz) 57:1–11. [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1007/s00005-009-0008-y&link_type=DOI) [PubMed](http://smj.org.sa/lookup/external-ref?access_num=19219526&link_type=MED&atom=%2Fsmj%2F37%2F11%2F1251.atom) 20. Brown T (2011) Journal quality metrics: options to consider other than impact factors. Am J Occup Ther 65:346–350. [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.5014/ajot.2011.001396&link_type=DOI) [PubMed](http://smj.org.sa/lookup/external-ref?access_num=21675341&link_type=MED&atom=%2Fsmj%2F37%2F11%2F1251.atom) 21. Jani N, Keshteli AH, Kabiri P, Esmaillzadeh A (2012) A 10-year performance trajectory of top nutrition journals’ impact factors. J Res Med Sci 17:128–132. 22. Foo JY (2009) The retrospective analysis of bibliographical trends for nine biomedical engineering journals from 1999 to 2007. Ann Biomed Eng 37:1474–1481. [PubMed](http://smj.org.sa/lookup/external-ref?access_num=19387834&link_type=MED&atom=%2Fsmj%2F37%2F11%2F1251.atom) 23. Holguin JA (2009) Occupational therapy and the journal citation reports:10-year performance trajectories. Am J Occup Ther 63:105–112. [PubMed](http://smj.org.sa/lookup/external-ref?access_num=19192733&link_type=MED&atom=%2Fsmj%2F37%2F11%2F1251.atom) 24. Zyoud SH, Al-Jabi SW, Sweileh WM, Awang R (2014) A bibliometric analysis of toxicology research productivity in Middle Eastern Arab countries during a 10-year period (2003-2012). Health Res Policy Syst 12:4. [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1186/1478-4505-12-4&link_type=DOI) [PubMed](http://smj.org.sa/lookup/external-ref?access_num=24443999&link_type=MED&atom=%2Fsmj%2F37%2F11%2F1251.atom) 25. Rossner M, Van Epps H, Hill E (2007) Show me the data. J Exp Med 204:3052–3053. [FREE Full Text](http://smj.org.sa/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiamVtIjtzOjU6InJlc2lkIjtzOjExOiIyMDQvMTMvMzA1MiI7czo0OiJhdG9tIjtzOjIwOiIvc21qLzM3LzExLzEyNTEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 26. Liu XL, Gai SS, Zhou J (2016) Journal Impact Factor: Do the Numerator and Denominator Need Correction? PLoS One, 11e0151414. 27. Falagas ME, Alexiou VG (2008) The top-ten in journal impact factor manipulation. Arch Immunol Ther Exp (Warsz) 56:223–226. [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1007/s00005-008-0024-5&link_type=DOI) [PubMed](http://smj.org.sa/lookup/external-ref?access_num=18661263&link_type=MED&atom=%2Fsmj%2F37%2F11%2F1251.atom) 28. Chew M, Villanueva EV, Van Der Weyden MB (2007) Life and times of the impact factor: retrospective analysis of trends for seven medical journals (1994-2005) and their Editors’ views. J R Soc Med 100:142–150. [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1258/jrsm.100.3.142&link_type=DOI) [PubMed](http://smj.org.sa/lookup/external-ref?access_num=17339310&link_type=MED&atom=%2Fsmj%2F37%2F11%2F1251.atom) [Web of Science](http://smj.org.sa/lookup/external-ref?access_num=000245122400013&link_type=ISI) 29. Phillips DP, Kanter EJ, Bednarczyk B, Tastad PL (1991) Importance of the lay press in the transmission of medical knowledge to the scientific community. N Engl J Med 325:1180–1183. [CrossRef](http://smj.org.sa/lookup/external-ref?access_num=10.1056/NEJM199110173251620&link_type=DOI) [PubMed](http://smj.org.sa/lookup/external-ref?access_num=1891034&link_type=MED&atom=%2Fsmj%2F37%2F11%2F1251.atom) [Web of Science](http://smj.org.sa/lookup/external-ref?access_num=A1991GK53800030&link_type=ISI) 30. Moustafa K (2015) The disaster of the impact factor. Sci Eng Ethics 21:139–142.