Leydesdorff, L (Leydesdorff, Loet) Caveats for the use of citation indicators in research and journal evaluations JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, 59 (2): 278-287 JAN 15 2008

[Eugene Garfield]

E-mail Address: loet at leydesdorff.net

Author(s): Leydesdorff, L (Leydesdorff, Loet) 

Title: Caveats for the use of citation indicators in research and journal 
evaluations 

Source: JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND 
TECHNOLOGY, 59 (2): 278-287 JAN 15 2008 

Language: English 
Document Type: Article 

Keywords Plus: IMPACT FACTORS; ANGEWANDTE-CHEMIE; SCIENCE; INDEX; CHINA; 
INTERDISCIPLINARITY; SCIENTOMETRICS; BETWEENNESS; CENTRALITY; KNOWLEDGE 

Abstract: Aging of publications, percentage of self-citations, and impact 
vary from journal to journal within fields of science. The assumption that 
citation and publication practices are homogenous within specialties and 
fields of science is invalid. Furthermore, the delineation of fields and 
among specialties is fuzzy. Institutional units of analysis and persons 
may move between fields or span different specialties. The match between 
the citation index and institutional profiles varies among institutional 
units and nations. The respective matches may heavily affect the 
representation of the units. Non-institute of Scientific Information (ISI) 
journals are increasingly cornered into "transdisciplinary" Mode-2 
functions with the exception of specialist journals publishing in 
languages other than English. An "externally cited impact factor" can be 
calculated for these journals. The citation impact of non-ISI journals 
will be demonstrated using Science and Public Policy as the example. 

Addresses: Univ Amsterdam, Amsterdam Sch Commun Res ASCoR, NL-1012 CX 
Amsterdam, Netherlands 

Reprint Address: Leydesdorff, L, Univ Amsterdam, Amsterdam Sch Commun Res 
ASCoR, Kloveniersburgwal 48, NL-1012 CX Amsterdam, Netherlands. 
 
Times Cited: 0 

Publisher: JOHN WILEY & SONS INC 

Publisher Address: 111 RIVER ST, HOBOKEN, NJ 07030 USA 

ISSN: 1532-2882 

Cited Reference Count: 89

*NAT RES COUNC
RES DOCT PROGR : 2007 

ADAMS J
FUNDING RES DIVERSIT : 2003 

AHLGREN P
Requirements for a cocitation similarity measure, with special reference 
to Pearson's correlation coefficient 
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY 
54 : 550 DOI 10.1002/asi.10242 2003 

ANDERSON J
SCI PUBL POLICY 15 : 153 1988 

ASHBY R
CYBERNETICA 1 : 1 1958 

BARILAN J
ISSI NEWSLETTER 2 : 3 2006 

BENSMAN SJ
Garfield and the impact factor 
ANNUAL REVIEW OF INFORMATION SCIENCE AND TECHNOLOGY 41 : 93 2007 

BENSMAN SJ
Scientific and technical serials holdings optimization in an inefficient 
market: A LSU serials redesign project exercise 
LIBRARY RESOURCES & TECHNICAL SERVICES 42 : 147 1998 

BONACCORSI A
PRIME GEN ASSMBL MAN : 2005 

BORNMANN L
Citation environment of Angewandte Chemie 
CHIMIA 61 : 104 2007 

BOYACK KW
Mapping the backbone of science 
SCIENTOMETRICS 64 : 351 DOI 10.1007/s11192-005-0255-6 2005 

BRAAM R
P 11 INT C SCI INF C 1 : 136 2007 

BRADFORD SC
ENGINEERING-LONDON 137 : 85 1934 

BRAUN T
A Hirsch-type index for journals 
SCIENTOMETRICS 69 : 169 DOI 10.1007/s11192-006-0147-4 2006 

BRUCKNER E
EVOLUTIONARY EC CHAO : 79 1994 

BURRELL QL
Hirsch index or Hirsch rate? Some thoughts arising from Liang's data 
SCIENTOMETRICS 73 : 19 DOI 10.1007/s11192-006-1774-5 2007 

CHRISTENSEN FH
Online determination of the journal impact factor and its international 
properties 
SCIENTOMETRICS 40 : 529 1997 

COLLINS HM
THE POSSIBILITIES OF SCIENCE POLICY 
SOCIAL STUDIES OF SCIENCE 15 : 554 1985 

EGGHE L
INTRO INFORM : 1990 

EGGHE L
ISSI NEWSLETTER 2 : 8 2006 

EGGHE L
SCIENTIST 20 : 315 2006 

FRANDSEN TF
Article impact calculated over arbitrary periods 
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY 
56 : 58 DOI 10.1002/asi.20100 2005 

FREEMAN LC
SET OF MEASURES OF CENTRALITY BASED ON BETWEENNESS 
SOCIOMETRY 40 : 35 1977 

FUJIGAKI Y
Filling the gap between discussions on science and scientists' everyday 
activities: applying the autopoiesis system theory to scientific knowledge 
SOCIAL SCIENCE INFORMATION SUR LES SCIENCES SOCIALES 37 : 5 1998 

GARFIELD E
How can impact factors be improved? 
BRITISH MEDICAL JOURNAL 313 : 411 1996 

GARFIELD E
CITATION INDEXING IT : 1979 

GARFIELD E
CURR CONTENTS 3 : 5 1971 

GARFIELD E
CURRENT CONTENT 0528 : 5 1990 

GARFIELD E
The multiple meanings of impact factors 
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE 49 : 768 1998 

GARFIELD E
REV INT PSICOLOGIA C 3 : 363 2003 

GARFIELD E
SCI J CITATION REPOR : 1980 

GARFIELD E
SCIENCE 178 : 4060 1972 

GARFIELD E
Long-term vs. short-term journal impact: Does it matter? 
SCIENTIST 12 : 11 1998 

GIBBONS M
PRODUCTION KNOWLEDGE : 1994 

GLANZEL W
A new classification scheme of science fields and subfields designed for 
scientometric evaluation purposes 
SCIENTOMETRICS 56 : 357 2003 

GRANOVETTER MS
AM J SOCIOL 78 : 6 1973 

HANNEMAN RA
INTRO SOCIAL NETWORK : 2005 

HELLSTEN I
Self-citations, co-authorships and keywords: A new approach to scientists' 
field mobility? 
SCIENTOMETRICS 72 : 469 DOI 10.1007/s11192-007-1680-5 2007 

HICKS D
The difficulty of achieving full coverage of international social science 
literature and the bibliometric consequences 
SCIENTOMETRICS 44 : 193 1999 

HIRSCH JE
An index to quantify an individual's scientific research output 
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF 
AMERICA 102 : 16569 DOI 10.1073/pnas.0507655102 2005 

JIN BH
The R- and AR-indices: Complementing the h-index 
CHINESE SCIENCE BULLETIN 52 : 855 DOI 10.1007/s11434-007-0145-9 2007 

JIN BH
SCI FOCUS 1 : 8 2006 

KING DA
The scientific impact of nations 
NATURE 430 : 311 DOI 10.1038/430311a 2004 

KOSMULSKI M
ISSI NEWSLETTER 2 : 4 2006 

KREFT IGG
QUAL QUANT 22 : 127 1988 

LEYDESDORFF L
IN PRESS IS US LOSIN : 2007 

LEYDESDORFF L
Betweenness centrality as an indicator of the interdisciplinarity of 
scientific journals 
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY 
58 : 1303 DOI 10.1002/asi.20614 2007 

LEYDESDORFF L
Mapping the Chinese Science Citation Database in terms of aggregated 
journal-journal citation relations 
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY 
56 : 1469 DOI 10.1002/asi.20209 2005 

LEYDESDORFF L
J AM SOC INFORM SCI 58 : 207 2007 

LEYDESDORFF L
J AM SOC INFORM SCI 57 : 601 2006 

LEYDESDORFF L
KNOWLEDGE BASED EC M : 2006 

LEYDESDORFF L
DIMENSIONS OF CITATION ANALYSIS 
SCIENCE TECHNOLOGY & HUMAN VALUES 15 : 305 1990
 
LEYDESDORFF L
SCI TECHNOLOGY STUDI 5 : 22 1987 

LEYDESDORFF L
Mapping interdisciplinarity at the interfaces between the science citation 
index and the social science citation index 
SCIENTOMETRICS 71 : 391 DOI 10.1007/s11192-007-1694-z 2007 

LEYDESDORFF L
Are the contributions of China and Korea upsetting the world system of 
science? 
SCIENTOMETRICS 63 : 617 DOI 10.1007/s11192-005-0231-1 2005 

LEYDESDORFF L
Theories of citation? 
SCIENTOMETRICS 43 : 5 1998 

LEYDESDORFF L
Scientometrics and communication theory: Towards theoretically informed 
indicators 
SCIENTOMETRICS 38 : 155 1997 

LI ZX
How to establish a first-class international scientific journal in China? 
WORLD JOURNAL OF GASTROENTEROLOGY 12 : 6905 2006 

MARICIC S
TECHNOSCIENCE WIN : 10 1997 

MARTIN BR
ASSESSING BASIC RESEARCH - SOME PARTIAL INDICATORS OF SCIENTIFIC PROGRESS 
IN RADIO ASTRONOMY 
RESEARCH POLICY 12 : 61 1983 

MARTYN J
ASLIB OCCASIONAL PUB 1 : 1968 

MARX W
Angewandte Chemie in light of the Science Citation Index 
ANGEWANDTE CHEMIE-INTERNATIONAL EDITION 40 : 139 2001 

MEHO LI
IN PRESS J AM SOC IN : 

MOED HF
CITATION ANAL RES EV : 2005 

MOED HF
THE USE OF BIBLIOMETRIC DATA FOR THE MEASUREMENT OF UNIVERSITY-RESEARCH 
PERFORMANCE 
RESEARCH POLICY 14 : 131 1985 

MORAVCSIK MJ
APPLIED SCIENTOMETRICS - AN ASSESSMENT METHODOLOGY FOR DEVELOPING-
COUNTRIES 
SCIENTOMETRICS 7 : 165 1985 

NEDERHOF AJ
ASSESSING THE USEFULNESS OF BIBLIOMETRIC INDICATORS FOR THE HUMANITIES AND 
THE SOCIAL AND BEHAVIORAL-SCIENCES - A COMPARATIVE-STUDY 
SCIENTOMETRICS 15 : 423 1989 

NOWOTNY H
RETHINKING SCI KNOWL : 2001 

PARK HW
KOREAN J SCI CITATIO : 2007 

PARK HW
A comparison of the knowledge-based innovation systems in the economies of 
South Korea and the Netherlands using Triple Helix indicators 
SCIENTOMETRICS 65 : 3 DOI 10.1007/s11192-005-0257-4 2005 

PINSKI G
CITATION INFLUENCE FOR JOURNAL AGGREGATES OF SCIENTIFIC PUBLICATIONS - 
THEORY, WITH APPLICATION TO LITERATURE OF PHYSICS 
INFORMATION PROCESSING & MANAGEMENT 12 : 297 1976 

PRICE DJD
COMMUNICATION SCI EN : 3 1970 

PRICE DJD
NETWORKS OF SCIENTIFIC PAPERS 
SCIENCE 149 : 510 1965 

RAFOLS I
How cross-disciplinary is bionanotechnology? Explorations in the specialty 
of molecular motors 
SCIENTOMETRICS 70 : 633 DOI 10.1007/s11192-007-0305-3 2007 

ROUSSEAU R
4 INT C U EV RES EV 2004 

ROUSSEAU R
IN PRESS SIMPLE MODE : 2006 

ROUSSEAU R
Timelines in citation research 
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY 
57 : 1404 DOI 10.1002/asi.20397 2006 

SALTON G
INTRO MODERN INFORM : 1983 

SEGLEN PO
Why the impact factor of journals should not be used for evaluating 
research 
BRITISH MEDICAL JOURNAL 314 : 498 1997 

SHER IH
2 C RES PROGR EFF WA 1965 

STEGMANN J
Building a list of journals with constructed impact factors 
JOURNAL OF DOCUMENTATION 55 : 310 1999 

STUDER KE
CANC MISSION SOCIAL : 1980 

TESTA J
ISI DATABASE J SELEC : 1997 

TIJSSEN RJW
ON GENERALIZING SCIENTOMETRIC JOURNAL MAPPING BEYOND ISIS JOURNAL AND 
CITATION DATABASES 
SCIENTOMETRICS 33 : 93 1995 

VANDENBESSELAAR P
The cognitive and the social structure of STS 
SCIENTOMETRICS 51 : 441 2001 

VONTUNZELMANN N
EFFECTS SIZE RES PER : 2003 

WHITLEY RD
INTELLECTUAL SOCIAL : 1984 

ZHOU P
A comparison between the China Scientific and Technical Papers and 
Citations Database and the Science Citation Index in terms of journal 
hierarchies and interjournal citation relations 
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY 
58 : 223 DOI 10.1002/asi.20475 2007 

ZHOU P
The citation impacts and citation environments of Chinese journals in 
mathematics 
SCIENTOMETRICS 72 : 185 DOI 10.1007/s11192-007-1713-0 2007