Krestin, GP (Krestin, Gabriel P.) Evaluating the Quality of Radiology Research: What Are the Rules of the Game? RADIOLOGY, 249 (2): 418-424 NOV 2008

[Eugene Garfield]

E-mail Address: g.p.krestin at erasmusmc.nl 

Author(s): Krestin, GP (Krestin, Gabriel P.) 

Title: Evaluating the Quality of Radiology Research: What Are the Rules of 
the Game? 

Source: RADIOLOGY, 249 (2): 418-424 NOV 2008 

Excerpt: The best method of evaluating research performance and quality is 
a complex issue without simple or satisfactory answers.  However, the 
questions related to this issue will determine the path of science in the 
coming years.  For this reason, this subject deserves our continued 
scrutiny.  Metrics alone cannot always capture the essence of research, 
but they irrefutably and objectively reveal the level of competiveness of 
certain institutions and groups.  Even if scientists have difficulties 
accepting the reliability of metrics as indicators of quality-and 
therefore prefer the more subjective evaluation process performed by some 
peers-some kind of evaluation and benchmarking will ultmately be imposed 
on them by governments, policy makers, or even the general public.  I 
expect that the best systems for evaluating research quality and 
performance in biomedical imaging will be a combination of input, 
throughput and output metrics, some of which will involve the use of peer 
review as an underlying process.  I also strongly believe that we as a 
biomedical imaging community need to determine our own future by adopting 
our own system for evaluating imaging research quality - a system that we 
can accept and endorse as an entire community.

Language: English 

Document Type: Editorial Material 

Keywords Plus: BIBLIOMETRIC INDICATORS; SCIENTIFIC-RESEARCH; IMPACT 
FACTOR; INNOVATION; PERFORMANCE; DEPARTMENTS; SCIENCE 

Addresses: Erasmus MC, Univ Med Ctr Rotterdam, Dept Radiol, NL-3015 CE 
Rotterdam, Netherlands 
Reprint Address: Krestin, GP, Erasmus MC, Univ Med Ctr Rotterdam, Dept 
Radiol, S Gravendijkwal 230,Room Hs-218, NL-3015 CE Rotterdam, 
Netherlands. 

E-mail Address: g.p.krestin at erasmusmc.nl 

Cited Reference Count: 36 

Times Cited: 0 

Publisher: RADIOLOGICAL SOC NORTH AMERICA 

Publisher Address: 820 JORIE BLVD, OAK BROOK, IL 60523 USA 

ISSN: 0033-8419 

DOI: 10.1148/radiol.2491080832 

29-char Source Abbrev.: RADIOLOGY 

ISO Source Abbrev.: Radiology 

Source Item Page Count: 7 

Subject Category: Radiology, Nuclear Medicine & Medical Imaging 

ISI Document Delivery No.: 362RS 

*ORG EC COOP DEV
EV SCI RES SEL EXP : 1997 

ADAM D
The counting house 
NATURE 415 : 726 2002 

ANDREWS F
SCI PRODUCTIVITY EFF : 1979 

BANALESTANOL A
CITY U EC DISCUSSION 709 : 2007 

BAUM S
What can we learn from success? 
ACADEMIC RADIOLOGY 13 : 1449 DOI 10.1016/j.acra.2006.10.006 2006 

BRINN T
Measuring research quality: peer review 1, citation indices 0 
OMEGA-INTERNATIONAL JOURNAL OF MANAGEMENT SCIENCE 28 : 237 2000 

ERNST E
DRAWBACKS OF PEER-REVIEW 
NATURE 363 : 296 1993 

FESTEN C
REPORT RES MANAGEMEN : 2004 

FLORIDA R
IND KNOWLEDGE U IND : 589 1999 

GARFIELD E
CURR CONTENTS : 34 1976 

HIRSCH JE
An index to quantify an individual's scientific research output 
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF 
AMERICA 102 : 16569 DOI 10.1073/pnas.0507655102 2005 

HORROBIN DF
THE PHILOSOPHICAL BASIS OF PEER-REVIEW AND THE SUPPRESSION OF INNOVATION 
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION 263 : 1438 1990 

ITAGAKI MW
Impact of the national institutes of health on radiology research 
RADIOLOGY 247 : 213 DOI 10.1148/radiol.2471070745 2008 

ITAGAKI MW
Factors associated with academic radiology research productivity 
RADIOLOGY 237 : 774 DOI 10.1148/radiol.2373041508 2005 

JONISCH AI
What characterizes academic radiology departments that secure large 
amounts of external funding for research? 
ACADEMIC RADIOLOGY 13 : 1513 DOI 10.1016/j.acra.2006.09.052 2006 

KING J
A REVIEW OF BIBLIOMETRIC AND OTHER SCIENCE INDICATORS AND THEIR ROLE IN 
RESEARCH EVALUATION 
JOURNAL OF INFORMATION SCIENCE 13 : 261 1987 

LEVITT MD
Research funding: The means has become the end 
JOURNAL OF LABORATORY AND CLINICAL MEDICINE 146 : 159 DOI 
10.1016/j.lab.2005.06.001 2005 

LUUKKONEN T
BIBLIOMETRICS AND EVALUATION OF RESEARCH PERFORMANCE 
ANNALS OF MEDICINE 22 : 145 1990 

MAGUIRE MA
J AM COLL RADIOL 2 : 436 2005 

MCMILLAN GS
An analysis of the critical role of public science in innovation: the case 
of biotechnology 
RESEARCH POLICY 29 : 1 2000 

MOED HF
DEV BIBLIOMETRIC IND : 2007 

MOED HF
THE APPLICATION OF BIBLIOMETRIC INDICATORS - IMPORTANT FIELD-DEPENDENT AND 
TIME-DEPENDENT FACTORS TO BE CONSIDERED 
SCIENTOMETRICS 8 : 177 1985 

MORRIS N
The developing role of departments 
RESEARCH POLICY 31 : 817 2002 

NEDERHOF AJ
PEER-REVIEW AND BIBLIOMETRIC INDICATORS OF SCIENTIFIC PERFORMANCE - A 
COMPARISON OF CUM LAUDE DOCTORATES WITH ORDINARY DOCTORATES IN PHYSICS 
SCIENTOMETRICS 11 : 333 1987 

NELSON R
THE SIMPLE ECONOMICS OF BASIC SCIENTIFIC-RESEARCH 
JOURNAL OF POLITICAL ECONOMY 67 : 297 1959 

NOYONS EC
CT20020001 ECPPLS U : 2003 

OPTHOF T
Sense and nonsense about the impact factor 
CARDIOVASCULAR RESEARCH 33 : 1 1997 

PAOLILLO J
MULTIVARIATE APPROACH TO PERCEIVED INNOVATION IN R AND D SUBSYSTEMS 
IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT 26 : 36 1979 

RENNIE D
EDITORIAL PEER REV I : 1999 

SCHNITZLER K
HIS KURZINFORMATIO A 11 : 1995 
              
SEGLEN PO
Why the impact factor of journals should not be used for evaluating 
research 
BRITISH MEDICAL JOURNAL 314 : 498 1997 

SIZER J
STUD HIGH EDUC 13 : 101 1998 

TAYLOR J
HIGHER ED Q 55 : 42 2001 

VANRAAN AFJ
Advanced bibliometric methods as quantitative core of peer review based 
evaluation and foresight exercises 
SCIENTOMETRICS 36 : 397 1996 

WALTER G
Counting on citations: a flawed way to measure quality 
MEDICAL JOURNAL OF AUSTRALIA 178 : 280 2003 

WEALE AR
BMC MED RES METHODOL 4 : 14 2004