Bollen, J; van de Sompel, H Usage Impact Factor: The effects of sample characteristics on usage-based impact metrics JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, 59 (1): 136-149 JAN 1 2008

[Eugene Garfield]

E-mail Address: jbollen at lanl.gov; herbertv at lanl.gov 

Author(s): Bollen, J (Bollen, Johan); van de Sompel, H (van de Sompel, 
Herbert)
 
Title: Usage Impact Factor: The effects of sample characteristics on usage-
based impact metrics 

Source: JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND 
TECHNOLOGY, 59 (1): 136-149 JAN 1 2008 

Language: English 

Document Type: Article 

Keywords Plus: CITATION ANALYSES; DIGITAL LIBRARY; JOURNAL IMPACT; 
SCIENCE; PUBLICATION; INFORMATION; STATISTICS; READERSHIP; NONSENSE; 
SYSTEM 

Abstract: There exist ample demonstrations that indicators of scholarly 
impact analogous to the citation-based ISI Impact Factor can be derived 
from usage data; however, so far, usage can practically be recorded only 
at the level of distinct information services. This leads to community-
specific assessments of scholarly impact that are difficult to generalize 
to the global scholarly community. In contrast, the ISI Impact Factor is 
based on citation data and thereby represents the global community of 
scholarly authors. The objective of this study is to examine the effects 
of community characteristics on assessments of scholarly impact from 
usage. We define a journal Usage Impact Factor that mimics the definition 
of the Thomson Scientific ISI Impact Factor. Usage Impact Factor rankings 
are calculated on the basis of a large-scale usage dataset recorded by the 
linking servers of the California State University system from 2003 to 
2005. The resulting journal rankings are then compared to the Thomson 
Scientific ISI Impact Factor that is used as a reference indicator of 
general impact. Our results indicate that the particular scientific and 
demographic characteristics of a discipline have a strong effect on 
resulting usage-based assessments of scholarly impact. In particular, we 
observed that as the number of graduate students and faculty increases in 
a particular discipline, Usage Impact Factor rankings will converge more 
strongly with the ISI Impact Factor. 

Addresses: Los Alamos Natl Lab, Digital Lib Res & Prototyping Team, Los 
Alamos, NM 87545 USA 

Reprint Address: Bollen, J, Los Alamos Natl Lab, Digital Lib Res & 
Prototyping Team, Los Alamos, NM 87545 USA. 

E-mail Address: jbollen at lanl.gov; herbertv at lanl.gov 

Cited Reference Count: 42 

Times Cited: 0 

Publisher: JOHN WILEY & SONS INC 

Publisher Address: 111 RIVER ST, HOBOKEN, NJ 07030 USA 

ISSN: 1532-2882 

NATURE 435 : 1003 2005 

*AN STAT DIV
STAT ABSTR 2004 2005 : 2004 

*MED
PLOS MED 3 : 2006 

BOLLEN J
D LIB MAGAZINE 8 : 2002 

BOLLEN J
Toward alternative metrics of journal impact: A comparison of download and 
citation data 
INFORMATION PROCESSING & MANAGEMENT 41 : 1419 DOI 
10.1016/j.ipm.2005.03.024 2005
 
BOLLEN J
JOINT C DIG LIBRARIE : 298 2006 

BOLLEN J
Detecting research trends in digital library readership 
RESEARCH AND ADVANCED TECHNOLOGY FOR DIGITAL LIBRARIES 2769 : 24 2003 

BOLLEN J
Mapping the structure of science through usage 
SCIENTOMETRICS 69 : 227 DOI 10.1007/s11192-006-0151-8 2006 

BRODY T
Earlier web usage statistics as predictors of later citation impact 
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY 
57 : 1060 DOI 10.1002/asi.20373 2006 

CHANDLER A
GRAIN 18 : 82 2006 

DARMONI SJ
Reading factor: a new bibliometric criterion for managing digital 
libraries 
JOURNAL OF THE MEDICAL LIBRARY ASSOCIATION 90 : 323 2002 

DAVIS PM
eJournal interface can influence usage statistics: Implications for 
libraries, publishers, and project COUNTER 
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY 
57 : 1243 DOI 10.1002/asi.20405 2006 

FRANKLIN B
Library usage patterns in the electronic information environment 
INFORMATION RESEARCH-AN INTERNATIONAL ELECTRONIC JOURNAL 9 : ARTN 187 2004 

GALLAGHER J
Evidence-based librarianship: Utilizing data from all available sources to 
make judicious print cancellation decisions 
LIBRARY COLLECTIONS ACQUISITIONS & TECHNICAL SERVICES 29 : 169 DOI 
10.1016/j.lcats.2005.04.004 2005 

GALVIN TJ
USE OF A UNIVERSITY-LIBRARY COLLECTION - PROGRESS REPORT ON A PITTSBURGH 
STUDY
LIBRARY JOURNAL 102 : 2317 1977 

GARFIELD E
Journal impact factor: a brief review 
CANADIAN MEDICAL ASSOCIATION JOURNAL 161 : 979 1999 

GARFIELD E
CITATION INDEXING IT : 1979 

GROOTE SL
B MED LIB ASS 89 : 372 2001 

KING DW
D LIB MAGAZINE 12 : 2006 

KURTZ MJ
Worldwide use and impact of the NASA astrophysics data system digital 
library 
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY 
56 : 36 DOI 10.1002/asi.20095 2005 

KURTZ MJ
The bibliometric properties of article readership information 
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY 
56 : 111 DOI 10.1002/asi.20096 2005 

LUTHER J
J ELECT PUBLISHING 6 : 2006 

LUWEL M
Publication delays in the science field and their relationship to the 
ageing of scientific literature 
SCIENTOMETRICS 41 : 29 1998 

MACROBERTS MH
PROBLEMS OF CITATION ANALYSIS - A CRITICAL-REVIEW 
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE 40 : 342 1989 

MCDONALD JD
J AM SOC INFORM SCI 57 : 39 2006 

MOED HF
Statistical relationships between downloads and citations at the level of 
individual documents within a single journal 
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY 
56 : 1088 DOI 10.1002/asi.20200 2005 

MONASTERSKY R
CHRON HIGHER EDUC 52 : A12 2005 

NICHOLAS D
Revisiting 'obsolescence' and journal article 'decay' through usage data: 
an analysis of digital journal use by year of publication 
INFORMATION PROCESSING & MANAGEMENT 41 : 1441 DOI 
10.1016/j.ipm.2005.03.014 2005 

OPTHOF T
Sense and nonsense about the impact factor 
CARDIOVASCULAR RESEARCH 33 : 1 1997 

PERNEGER TV
Relation between online "hit counts" and subsequent citations: prospective 
study of research papers in the BMJ 
BRITISH MEDICAL JOURNAL 329 : 546 2004 


REEDIJK J
Sense and nonsense of science citation analyses: comments on the monopoly 
position of ISI and citation inaccuracies. Risks of possible misuse and 
biased citation and impact data. 
NEW JOURNAL OF CHEMISTRY 22 : 767 1998 

RINIA EJ
Citation delay in interdisciplinary knowledge exchange 
SCIENTOMETRICS 51 : 293 2001 

SAHA S
Impact factor: a valid measure of journal quality? 
JOURNAL OF THE MEDICAL LIBRARY ASSOCIATION 91 : 42 2003 

SCALES PA
CITATION ANALYSES AS INDICATORS OF USE OF SERIALS - COMPARISON OF RANKED 
TITLE LISTS PRODUCED BY CITATION COUNTING AND FROM USE DATA
JOURNAL OF DOCUMENTATION 32 : 17 1976 

SHEPHERD PT
J INFORM PROCESSING 47 : 245 2004 

SHEPHERD PT
J SERIALS COMMUNITY 20 : 117 DOI 10.1629/20117 2007 

TSAY MY
The relationship between journal use in a medical library and citation use 
BULLETIN OF THE MEDICAL LIBRARY ASSOCIATION 86 : 31 1998 

VANDESOMPEL H
D LIB MAGAZINE 7 : 2001 

VANDESOMPEL H
D LIB MAGAZINE 1 5 : 1999 

VANDESOMPEL H
D LIB MAGAZINE 2 5 : 1999 

WEINGART P
Impact of bibliometrics upon the science system: Inadvertent consequences? 
SCIENTOMETRICS 62 : 117 2005 

WINSHIP C
ANNU REV SOCIOL 18 : 237 1992