Caveats for the journal and field normalizations in the CWTS ("Leiden") evaluations of research performance
lwaltman at FEW.EUR.NL
Fri Mar 19 17:34:09 EDT 2010
Recently, Tobias Opthof and Loet Leydesdorff have written a critical paper
(see below) about the way in which bibliometric research performance
assessment studies are conducted by the Centre for Science and Technology
Studies (CWTS) of Leiden University. There are a number of important
inaccuracies in the paper by Opthof and Leydesdorff. CWTS also strongly
disagrees with many of their comments. In the following paper CWTS replies to
the criticism of Opthof and Leydesdorff:
Anthony F.J. van Raan, Thed N. van Leeuwen, Martijn S. Visser, Nees Jan van
Eck, and Ludo Waltman. Rivals for the crown: Reply to Opthof and
Leydesdorff. Available at http://arxiv.org/abs/1003.2113.
CWTS has also prepared a related paper on the same topic:
Ludo Waltman, Nees Jan van Eck, Thed N. van Leeuwen, Martijn S. Visser, and
Anthony F.J. van Raan. Towards a new crown indicator: Some theoretical
considerations. Available at http://arxiv.org/abs/1003.2167.
On 16/02/2010 07:46, Loet Leydesdorff wrote:
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> Caveats for the journal and field normalizations in the CWTS ("Leiden")
evaluations of research performance
> Journal of Informetrics (forthcoming).
> Abstract: The Center for Science and Technology Studies at Leiden
University advocates the use of specific normalizations for assessing research
performance with reference to a world average. The Journal Citation Score
(JCS) and Field Citation Score (FCS) are averaged for the research group or
individual researcher under study, and then these values are used as
denominators of the (mean) Citations per publication (CPP). Thus, this
normalization is based on dividing two averages. This procedure only
generates a legitimate indicator in the case of underlying normal distributions.
Given the skewed distributions under study, one should average the observed
versus expected values which are to be divided first for each publication. We
show the effects of the Leiden normalization for a recent evaluation where we
happened to have access to the underlying data.
> Tobias Opthof [1,2], Loet Leydesdorff 
>  Experimental Cardiology Group, Heart Failure Research Center, Academic
Medical Center AMC, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands.
>  Department of Medical Physiology, University Medical Center Utrecht,
Utrecht, The Netherlands.
>  Amsterdam School of Communications Research (ASCoR), University of
Amsterdam, Kloveniersburgwal 48, 1012 CX Amsterdam, The Netherlands.
> ** apologies for cross-postings
Ludo Waltman MSc
Centre for Science and Technology Studies
P.O. Box 905
2300 AX Leiden
Willem Einthoven Building, Room B5-35
Tel: +31 (0)71 527 5806
Fax: +31 (0)71 527 3911
E-mail: waltmanlr at cwts.leidenuniv.nl
More information about the SIGMETRICS