Caveats for the journal and field normalizations in the CWTS ("Leiden") evaluations of research performance

[Ludo Waltman]

Dear colleagues,

Recently, Tobias Opthof and Loet Leydesdorff have written a critical paper 
(see below) about the way in which bibliometric research performance 
assessment studies are conducted by the Centre for Science and Technology 
Studies (CWTS) of Leiden University. There are a number of important 
inaccuracies in the paper by Opthof and Leydesdorff. CWTS also strongly 
disagrees with many of their comments. In the following paper CWTS replies to 
the criticism of Opthof and Leydesdorff:

Anthony F.J. van Raan, Thed N. van Leeuwen, Martijn S. Visser, Nees Jan van 
Eck, and Ludo Waltman. Rivals for the crown: Reply to Opthof and 
Leydesdorff. Available at http://arxiv.org/abs/1003.2113.

CWTS has also prepared a related paper on the same topic:

Ludo Waltman, Nees Jan van Eck, Thed N. van Leeuwen, Martijn S. Visser, and 
Anthony F.J. van Raan. Towards a new crown indicator: Some theoretical 
considerations. Available at http://arxiv.org/abs/1003.2167.

Best regards,

Ludo Waltman


On 16/02/2010 07:46, Loet Leydesdorff wrote:
> Adminstrative info for SIGMETRICS (for example unsubscribe): 
http://web.utk.edu/~gwhitney/sigmetrics.html
> Caveats for the journal and field normalizations in the CWTS ("Leiden") 
evaluations of research performance
> Journal of Informetrics (forthcoming).
> http://arxiv.org/abs/1002.2769
>
>     Abstract: The Center for Science and Technology Studies at Leiden 
University advocates the use of specific normalizations for assessing research 
performance with reference to a world average. The Journal Citation Score 
(JCS) and Field Citation Score (FCS) are averaged for the research group or 
individual researcher under study, and then these values are used as 
denominators of the (mean) Citations per publication (CPP). Thus, this 
normalization is based on dividing two averages. This procedure only 
generates a legitimate indicator in the case of underlying normal distributions. 
Given the skewed distributions under study, one should average the observed 
versus expected values which are to be divided first for each publication. We 
show the effects of the Leiden normalization for a recent evaluation where we 
happened to have access to the underlying data.
>      
>
> Tobias Opthof [1,2], Loet Leydesdorff [3]
>  
>
> [1] Experimental Cardiology Group, Heart Failure Research Center, Academic 
Medical Center AMC, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands.
>
> [2] Department of Medical Physiology, University Medical Center Utrecht, 
Utrecht, The Netherlands.
>
> [3] Amsterdam School of Communications Research (ASCoR), University of 
Amsterdam, Kloveniersburgwal 48, 1012 CX Amsterdam, The Netherlands.
>
>  
>
> ** apologies for cross-postings
>
>


========================================================
Ludo Waltman MSc
Researcher

Centre for Science and Technology Studies
Leiden University
P.O. Box 905
2300 AX Leiden
The Netherlands

Willem Einthoven Building, Room B5-35
Tel:      +31 (0)71 527 5806
Fax:      +31 (0)71 527 3911
E-mail:   waltmanlr at cwts.leidenuniv.nl
Homepage: www.ludowaltman.nl
========================================================