Zitt, M; Ramanana-Rahary, S; Bassecoulard, E "Relativity of citation performance and excellence measures: From cross-field to cross-scale effects of field-normalization" Scientometrics 63 (2). APR 2005. p.373-401

[Eugene Garfield]

E-mail Addresses: zitt at nantes.inra.fr

TITLE:          Relativity of citation performance and excellence
                measures: From cross-field to cross-scale effects of field-
                normalisation
                (Article, English)

AUTHOR:         Zitt, M; Ramanana-Rahary, S; Bassecoulard, E

SOURCE:         SCIENTOMETRICS 63 (2). APR 2005. p.373-401 SPRINGER,
                DORDRECHT

ABSTRACT:       As citation practices strongly depend on fields, field
normalisation is recognised as necessary for fair comparison of figures in
bibliometrics and evaluation studies. However fields may be defined at
various levels, from small research areas to broad academic disciplines,
and thus normalisation values are expected to vary. The aim of this project
was to test the stability of citation ratings of articles as the level of
observation - hence the basis of normalisation - changes. A conventional
classification of science based on ISI subject categories and their
aggregates at various scales was used, namely at five levels:
all science, large academic discipline, sub-discipline, speciality and
journal. Among various normalisation methods, we selected a simple ranking
method (quantiles), based on the citation score of the article in each
particular aggregate (journal, speciality, etc.) it belonged to at each
level. The study was conducted on articles in the full SCI range, for
publication year 1998 with a four-year citation window. Stability is
measured in three ways: overall comparison of article rankings; individual
trajectory of articles; survival of the top-cited class across levels.
Overall rank correlations on the observed empirical structure are
benchmarked against two fictitious sets that keep the same embedded
structure of articles but reassign citation scores either in a totally
ordered or in a totally random distribution. These sets act respectively as
a 'worst case' and 'best case' for the stability of citation ratings.
The results show that: (a) the average citation rankings of articles
substantially change with the level of observation (b) observation at the
journal level is very particular, and the results differ greatly in all
test circumstances from all the other levels of observation (c) the lack of
cross-scale stability is confirmed when looking at the distribution of
individual trajectories of articles across the levels; (d) when considering
the top-cited fractions, a standard measure of excellence, it is found that
the contents of the 'top-cited' set is completely dependent on the level of
observation. The instability of impact measures should not be interpreted
in terms of lack of robustness but rather as the co- existence of various
perspectives each having their own form of legitimacy. A follow-up study
will focus on the micro levels of observation and will be based on a
structure built around bibliometric groupings rather than conventional
groupings based on ISI subject categories.

AUTHOR ADDRESS: M Zitt, INRA, Lereco, Nantes, France

[Addresses: Zitt M (reprint author), INRA, Lereco, Nantes, France
INRA, Lereco, Nantes, France
OST, Paris, France

E-mail Addresses: zitt at nantes.inra.fr

Publisher: SPRINGER, VAN GODEWIJCKSTRAAT 30, 3311 GZ DORDRECHT, NETHERLANDS

IDS Number: 924RX

ISSN: 0138-9130

Cited References:
BARRE R, 2002, SCI TECHNOLOGY INDIC.
BASSECOULARD E, 1999, SCIENTOMETRICS, V44, P323.
CZAPSKI G, 1997, SCIENTOMETRICS, V40, P437.
EGGHE L, 2002, CAN J INFORM LIB SCI, V27, P29.
FISHER D, 2001, MINERVA, V39, P299.
GARFIELD E, 1986, CURR CONTENTS, P182.
GLANZEL W, 2002, SCIENTOMETRICS, V53, P171.
HAITUN SD, 1982, SCIENTOMETRICS, V4, P89.
KATZ JS, 1999, RES POLICY, V28, P501.
KOSTOFF RN, 2002, SCIENTOMETRICS, V53, P49.
MARSHAKOVASHAIKEVI, 1996, SCIENTOMETRICS, V35, P283.
MCROBERTS MH, 1989, J AM SOC INFORM SCI, V40, P342.
MOED HF, 2002, NATURE, V415, P731.
MURUGESAN P, 1978, J AM SOC INFORM SCI, V29, P141.
NARIN F, 1978, EVALUATIVE BIBLIOMET.
PINSKI G, 1976, INFORMATION PROCESSI, V12, P297.
RAMIREZ AM, 2000, SCIENTOMETRICS, V47, P3.
SCHUBERT A, 1986, SCIENTOMETRICS, V9, P281.
SCHUBERT A, 1988, HDB QUANTITATIVE STU, P137.
SCHUBERT A, 1996, SCIENTOMETRICS, V36, P311.
SEGLEN PO, 1997, ALLERGY, V52, P1050.
SEN BK, 1992, J DOC, V48, P318.
SIEGEL S, 1956, NONPARAMETRIC STAT B.
SMALL H, 1985, SCIENTOMETRICS, V7, P391.
SOLARI A, 2000, SCIENTOMETRICS, V47, P605.
TUSSEN RJW, 2002, SCIENTOMETRICS, V54, P381.
VANRAAN AFJ, 2000, SCIENTOMETRICS, V47, P347.
VANRAAN AFJ, 2001, SCIENTOMETRICS, V51, P347.
VINKLER P, 2002, SCIENTOMETRICS, V53, P267.
VINKLER P, 2004, J AM SOC INF SCI TEC, V55, P431.
WASMER E, 2001, SEARCH SCI EXCELLENC.
ZITT M, 1994, SCIENTOMETRICS, V30, P333.
ZITT M, 2003, SCIENTOMETRICS, V56, P259.