Abstract
This article reviews the nature and use of the journal impact factor and other common bibliometric measures for assessing research in the sciences and social sciences based on data compiled by Thomson Reuters. Journal impact factors are frequently misused to assess the influence of individual papers and authors, but such uses were never intended. Thomson Reuters also employs other measures of journal influence, which are contrasted with the impact factor. Finally, the author comments on the proper use of citation data in general, often as a supplement to peer review. This review may help government policymakers, university administrators, and individual researchers become better acquainted with the potential benefits and limitations of bibliometrics in the evaluation of research.
Similar content being viewed by others
References
Banks MA, Dellavalle R (2008) Emerging alternatives to the impact factor. OCLC Systems & Services 24(3). Available via: http://eprints.rclis.org/archive/00014614/01/OCLC2BPaper2BFinal-v2Feb2008.pdf
Bar-Ilan J (2008) Informetrics at the beginning of the 21st century: a review. J Informetrics 2: 1-52
Bensman SJ (2007) Garfield and the impact factor. Annu Rev Inf Sci Technol 41: 93-155
Bensman SJ (2007) The impact factor, total citations, and better citation mouse traps: a commentary. J Am Soc Inf Sci Technol 58: 1904-1908
Bergstrom CT (2007) Eigenfactor: measuring the value and prestige of scholarly journals. C&RL News 68:5. Available via: http://www.ala.org/ala/acrl/acrlpubs/crlnews/backissues2007/may07/eigenfactor.cfm , also see: http://www.eigenfactor.org//whyeigenfactor.htm
Bollen J, Rodriguez MA, Vande Sompel H (2006) Journal status. Scientometrics 69: 669-687
Borgman C, Furner J (2002) Scholarly communication and bibliometrics. Annu Rev Inf Sci Technol 36: 3-72
Bornmann L, Mutz R, Neuhaus C et al (2008) Citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics Sci. Environmental Politics 8: 93-102
Braun T, Glanzel W, Schubert A (2005) A Hirsch-type index for journals. Scientist 19:8
Brin S, Page L (1998) The anatomy of a large-scale hypertextual web search engine. Comput Networks ISDN Systems 30: 107-117
Butler L (2008) Using a balanced approach to bibliometrics: quantitative performance measures in the Australian Research Quality Framework. Ethics Sci Environmental Politics 8: 83-92
Egghe L, Rousseau R (2002) A general framework for relative impact factors. Can J Inf Library Sci 21: 29-38
Egghe L (2005) Continuous, weighted Lorenz theory and applications to the study of fractional relative impact factors. Inf Processing Management 41: 1330-1359
Egghe L (2006) Theory and practice of the g-index. Scientometrics 69: 131-152
Falagas ME, Kouranos VD, Arencibia-Jorge R et al (2008) Comparison of SCImago journal rank indicator with journal impact factor. FASEB J 22: 2623-2628
Garfield E (1955) Citation indexes for science. A new dimension in documentation through association of ideas. Science 122: 108-111
Garfield E (1972) Citation analysis as a tool in journal evaluation. Science 178: 471-479
Garfield E (1976) Significant journals of science. Nature 264: 609-615
Garfield E (1979) Citation Indexing: Its Theory and Application in Science, Technology, and Humanities. John Wiley & Sons, New York
Garfield E (1996) How can impact factors be improved?. BMJ 313: 411-413
Garfield E (1999) Journal impact factor: a brief review. CMAJ 161: 979-980
Garfield E (2006) The history and meaning of the journal impact factor. JAMA 295: 90-103
Garfield E, Sher IH (1963) New factors in the evaluation of scientific literature through citation indexing. Am Documentation 14: 195-201
Glanzel W (2008) Seven myths in bibliometrics: about facts and fiction in quantitative science studies. Proceedings of the WIS 2008, Berlin. Fourth International Conference on Webometrics, Informetrics and Scientometrics & Ninth COLLNET Meeting. Available via: http://www.collnet.de//Berlin-2008/GlanzelWIS2008smb.pdf
Glanzel W, Moed HF (2002) Journal impact measures in bibliometric research. Scientometrics 53: 171-193
Harnad S (2008) Validating research performance metrics against peer rankings. Ethics Sci. Environmental Politics 8: 103-107
Harzing A-WK, van der Wal R (2008) Google Scholar as a new source for citation analysis. Ethics Sci. Environmental Politics 8:61-73. Available via: http://www.harzing.com/resources.htm#/pop_hindex.htm
Hirsch JE (2005) An index to quantify an individual’s scientific research output. Proc Natl Acad Sci U S A 102: 16569-16572
Hirsch JE (2007) Does the h-index have predictive power?. Proc Natl Acad Sci U S A 104: 19193-19198
Jasco P (2005) As we may search: Comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases. Curr Sci 89: 1537-1547
Joint Committee on Quantitative Assessment of Research (2008) Citation Statistics. A report from the International Mathematical Union (IMU) in cooperation with the International Council of Industrial and Applied Mathematics (ICIAM) and the Institute of Mathematical Statistics (IMS), by Robert Adler, John Ewing (Chair), and Peter Taylor. Available via: http://www.mathunion.org/fileadmin/IMU//Report/CitationStatistics.pdf
Ketcham CM, Crawford JM (2007) The impact of review articles. Lab Invest 87: 1174-1185
Krauze T (1977) The sociology of science in Poland. In: D. A. Pendlebury: Citation indicators use and misuse Merton RK, Gaston J (eds) The Sociology of Science in Europe. Southern Illinois University Press, Carbondale, Illinois, pp. 193-223
Leydesdorff L (2008) Caveats in the use of citation indicators in research and journal evaluations. J Am Soc Inf Sci Technol 59: 278-287
Markpin T, Boonradsamee B, Ruksinsut K et al (2008) Article-count impact factor of materials science journals in SCI database. Scientometrics 75: 251-261
Meho LI, Yang K (2007) A new era in citation and bibliometric analyses: Web of Science. Scopus, and Google Scholar. J Am Soc Inf Sci Technol 58: 2105-2125
Moed HF (2002) The impact-factors debate: the ISI’s uses and limits. Nature 415: 731-732
Moed HF (2005) Citation analysis of scientific journals and journal impact measures. Curr Sci 89: 1990-1996
Moed HF (2005) Citation analysis in research evaluation. Springer, Dordrecht
Moed HF, Glanzel W, Schmoch U (eds) (2004) Handbook of quantitative science and technology research. The use of publication and patent statistics in studies of S&T systems. Kluwer Academic Publishers, Dordrecht
Moed HF, van Leeuwen TN (1995) Improving the accuracy of Institute for Scientific Information’s journal impact factors. J Am Soc Inf Sci 46: 461-467
Moed HF, van Leeuwen TN (1996) Impact factors can mislead. Nature 381: 186
Moed HF, van Leeuwen TN, Reedijk J (1996) A critical analysis of the journal impact factors of Angewandte Chemie and Journal of the American Chemical Society: inaccuracies in published impact factors based on overall citation counts only. Scientometrics 37: 105-116
Moed HF, van Leeuwen TN, Reedijk J (1999) Towards appropriate indicators of journal impact. Scientometrics 46: 575-589
Monastersky R (2005) The number that’s devouring science. The Chronicle of Higher Education 52: A12
Narin F (1976) Evaluative bibliometrics: the use of publication and citation analysis in the evaluation of scientific activity. Computer Horizons Inc., Cherry Hill, New Jersey
Nicolaisen J (2007) Citation analysis. Annu Rev Inf Sci Technol 41: 609-641
Ossowska M, Ossowski S (1936) The science of science. Organon 1: 1-12
Pinski G, Narin F (1976) Citation influence for journal aggregates of scientific publications: theory, with application to the literature of physics. Inf Processing Management 12: 297-312
Pudovkin AI, Garfield E (2004) Rank-normalized impact factor: a way to compare journal performance across subject categories. ASIST 2004: Proceedings of the 67th ASIS&T Annual Meeting 41:507-515
Rousseau R (2002) Journal evaluation: technical and practical issues. Library Trends 50: 418-439
Rousseau R (2005) Median and percentile impact factors. Scientometrics 63: 431-441
Seglen PO (1997) Why the impact factor of journals should not be used for evaluating research. BMJ 314: 498-502
Sombatsompop N, Markpin T (2005) Making an equality of ISI impact factors for different subject fields. J Am Soc Inf Sci Technol 56: 676-783
Sombatsompop N, Markpin T, Premkamolnetr N (2004) A modified method for calculating the impact factors of the journals in ISI Journal Citation Reports: polymer science category in 1997-2001. Scientometrics 60: 217-235
Sombatsompop N, Markpin T, Yochai W et al (2005) An evaluation of research performance for different subject categories using impact factor point average (IFPA) index: Thailand case study. Scientometrics 65: 293-205
Stringer MJ, Sales-Pardo M, Nunes Amaral LA (2008) Effectiveness of journal ranking schemes as a tool for locating information. PLoS One 3:e1683. Available via: http://www.plosone.org/article/info:doi2F10.13712Fjournal.pone.0001683
Thomson Reuters, http://scientific.thomson.com/products/jcr/
Thomson Reuters, http://scientific.thomsonreuters.com/products/jpi/
Thomson Reuters, http://scientific.thomson.com/products/esi/
Vaidya JS (2005) V-index: a fairer index to quantify an individual’s research output capacity. BMJ Rapid Response. Available via: http://bmj.bmjjournals.com/cgi/eletters/331//7528/1339-c#123188
van Leeuwen TN, Moed HF (2002) Development and application of journal impact measures in the Dutch science system. Scientometrics 53: 249-266
van Leeuwen TN, Moed HF (2005) Characteristics of journal impact factors: the effects of uncitedness and citation distribution on the understanding of journal impact factors. Scientometrics 63: 357-371
van Leeuwen TN, Moed HF, Reedijk J (1999) Critical comments on Institute for Scientific Information impact factors: a sample of inorganic molecular chemistry journals. J Inf Sci 25: 489-498
Vinkler P (2004) Characterization of the impact of sets of scientific papers: the Garfield (impact) factor. J Am Soc Inf Sci Technol 55: 431-435
Wouters P (1999) The Citation Culture. University of Amsterdam, Amsterdam, pp. 82-93
Zitt M, Bassecoulard E (2008) Challenges for scientometric indicators: data demining, knowledge-flow measurements and diversity issues. Ethics Sci. Environmental Politics 8: 49-50
Zitt M, Small H (2008) Modifying the journal impact factor by fractional citation weighting: the audience factor. J Am Soc Inf Sci Technol 59: 1856-1860
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Pendlebury, D.A. The use and misuse of journal metrics and other citation indicators. Arch. Immunol. Ther. Exp. 57, 1–11 (2009). https://doi.org/10.1007/s00005-009-0008-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00005-009-0008-y