Greenwood, DC (Greenwood, Darren C.) Reliability of journal impact factor rankings BMC MEDICAL RESEARCH METHODOLOGY, 7: Art. No. 48 NOV 15 2007

[Eugene Garfield]

E-mail Address: d.c.greenwood at leeds.ac.uk

URL: http://www.biomedcentral.com/1471-2288/7/48

Author(s): Greenwood, DC (Greenwood, Darren C.) 

Title: Reliability of journal impact factor rankings 

Source: BMC MEDICAL RESEARCH METHODOLOGY, 7: Art. No. 48 NOV 15 2007 

Language: English 

Document Type: Article 

Keywords Plus: GENERAL MEDICAL JOURNALS; CITATION ANALYSIS; LEAGUE TABLES; 
SCIENCE; QUALITY; ASSOCIATION; FREQUENCY; INDEXES; IDEAS; TOOL 

Abstract: Background: Journal impact factors and their ranks are used 
widely by journals, researchers, and research assessment exercises.

Methods: Based on citations to journals in research and experimental 
medicine in 2005, Bayesian Markov chain Monte Carlo methods were used to 
estimate the uncertainty associated with these journal performance 
indicators.
Results: Intervals representing plausible ranges of values for journal 
impact factor ranks indicated that most journals cannot be ranked with 
great precision. Only the top and bottom few journals could place any 
confidence in their rank position. Intervals were wider and overlapping 
for most journals.

Conclusion: Decisions placed on journal impact factors are potentially 
misleading where the uncertainty associated with the measure is ignored. 
This article proposes that caution should be exercised in the 
interpretation of journal impact factors and their ranks, and specifically 
that a measure of uncertainty should be routinely presented alongside the 
point estimate. 

Addresses: Univ Leeds, Ctr Biostat & Epidemiol, Biostat Unit, Leeds, W 
Yorkshire, England 

Reprint Address: Greenwood, DC, Univ Leeds, Ctr Biostat & Epidemiol, 
Biostat Unit, Leeds, W Yorkshire, England. 
 
Cited Reference Count: 29 

Times Cited: 0 

Publisher: BIOMED CENTRAL LTD 

Publisher Address: MIDDLESEX HOUSE, 34-42 CLEVELAND ST, LONDON W1T 4LB, 
ENGLAND 

ISSN: 1471-2288 

*DEP ED SKILLS
REF HIGH ED RES ASS : 2006 

*INT COMM MED J ED
MED EDUC 33 : 66 1999 

3.  SCI INN INV FRAM 200 : 2006 

BIRD SM
J ROYAL STAT SOC A : 168 2005 

BROOKS SP
General methods for monitoring convergence of iterative simulations 
JOURNAL OF COMPUTATIONAL AND GRAPHICAL STATISTICS 7 : 434 1998 

EBRAHIM S
Entelechy, citation indexes, and the association of ideas 
INTERNATIONAL JOURNAL OF EPIDEMIOLOGY 35 : 1117 DOI 10.1093/ije/dyl201 
2006 

GARFIELD E
CITATION ANALYSIS STUDIES 
SCIENCE 189 : 397 1975 

GARFIELD E
CITATION ANALYSIS AS A TOOL IN JOURNAL EVALUATION - JOURNALS CAN BE RANKED 
BY FREQUENCY AND IMPACT OF CITATIONS FOR SCIENCE POLICY STUDIES 
SCIENCE 178 : 471 1972
 
GARFIELD E
CITATION INDEXES FOR SCIENCE - NEW DIMENSION IN DOCUMENTATION THROUGH 
ASSOCIATION OF IDEAS 
SCIENCE 122 : 108 1955 

GILKS WR
MARKOV CHAIN M CARLO : 1996 

GOLDSTEIN H
League tables and their limitations: Statistical issues in comparisons of 
institutional performance 
JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES A-STATISTICS IN SOCIETY 
159 : 385 1996 

JANKE NC
JOURNAL EVALUATION 
SCIENCE 182 : 1196 1973 

JOSEPH KS
Quality of impact factors of general medical journals 
BRITISH MEDICAL JOURNAL 326 : 283 2003 

KLERER M
CITATION ANALYSIS 
SCIENCE 188 : 1064 1975 

MARSHALL EC
MULTILEVEL MODELLING : 127 2001 

MARSHALL EC
STAT ANAL MED DATA N : 229 1998 

MARSHALL EG
Reliability of league tables of in vitro fertilisation clinics: 
retrospective analysis of live birth rates 
BRITISH MEDICAL JOURNAL 316 : 1701 1998 

PORTA M
Quality of impact factors of general medical journals - Quality matters - 
and the choice of indicator matters too 
BRITISH MEDICAL JOURNAL 326 : 931 2003 

PORTA M
Commentary: The 'bibliographic impact factor' and the still uncharted 
sociology of epidemiology 
INTERNATIONAL JOURNAL OF EPIDEMIOLOGY 35 : 1130 DOI 10.1093/ije/dyl196 
2006 

ROSTAMIHODJEGAN A
Journal impact factors: a 'bioequivalence' issue? 
BRITISH JOURNAL OF CLINICAL PHARMACOLOGY 51 : 111 2001 

SEGLEN PO
THE SKEWNESS OF SCIENCE 
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE 43 : 628 1992 

SEGLEN PO
CITATION FREQUENCY AND JOURNAL IMPACT - VALID INDICATORS OF SCIENTIFIC 
QUALITY? 
JOURNAL OF INTERNAL MEDICINE 229 : 109 1991 

SMITH R
Commentary: The power of the unrelenting impact factor - Is it a force for 
good or harm? 
INTERNATIONAL JOURNAL OF EPIDEMIOLOGY 35 : 1129 DOI 10.1093/ije/dyl191 
2006 


SPIEGELHALTER DJ
WINBUGS USER MANUAL : 2004 

TAINER JA
SCIENCE, CITATION, AND FUNDING 
SCIENCE 251 : 1408 1991 

WADE N
CITATION ANALYSIS - NEW TOOL FOR SCIENCE ADMINISTRATORS 
SCIENCE 188 : 429 1975 

WARNER J
A critical review of the application of citation studies to the Research 
Assessment Exercises 
JOURNAL OF INFORMATION SCIENCE 26 : 453 2000 

WEALE AR
BMC MED RES METHODOL 4 : 14 2004 

WILLIAMS G
Should we ditch impact factors? Yes 
BRITISH MEDICAL JOURNAL 334 : 568 2007