Early citation advantage?

[Loet Leydesdorff]

> The trouble with scientometrics (and sometimes also
> psychometrics) is that we tend to construct or derive the
> metric but never bother going through the step of validating
> it! We just baptize it as having face validity. With pure
> quantity measures such as publication counts, this might
> possibly be legitimate (if we really which to treat
> publications as products, and ends in themselves.) But
> starting already with citations, validating needs to be done.
> Metrics cannot be declared valid by a-priori fiat.

Dear Stevan,

You use the word "metrics" sometimes for descriptive statistics (e.g.,
citation countes) and in other emails for multiple regression analysis which
would have predictive power on the basis of an explanatory model. The
descriptive statistics do not have predictive power unless one adds a
theoretical assumption.

For example, one can make the assumption that the Markov property holds for
the UK system: If we would know nothing more about the UK publication
system, the best prediction of the next stage of the system would be the
state of this system today.(1) Given that assumption, it makes sense to
extrapolate citation counts of today to tomorrow. If one would have reasons
to assume, however, that policy interventions (e.g., the RAE) matter, the
Markov assumption might become less valid because the system would be
disturbed. :-)

Let me also note that quite a few of us have spent many years on validating
the descriptive indicators. As has been noted in other mails, there are
serious problems with their validity. Most of these studies ("citation
context analysis") were done in the 1980s. Nowadays--as then--there is a lot
of pressure to provide policy-makers and managers with relatively simple
numbers for making allocation decisions. However, there may be other
considerations (e.g., portfolio analysis) which may be as worthwile for
policy decisions as previous citation counts. Indeed, one should go in the
direction of a multiple regression analysis, but this can only be done
step-by-step because each equation requires a theory in the background.

With best wishes,


Loet

(1) Loet Leydesdorff, On the "Scientometric Decline" of British Science.
Scientometrics 20 (1991) 363-367
________________________________

Loet Leydesdorff
Amsterdam School of Communications Research (ASCoR)
Kloveniersburgwal 48, 1012 CX Amsterdam
Tel.: +31-20- 525 6598; fax: +31-20- 525 3681
loet at leydesdorff.net ; http://www.leydesdorff.net/