Potential Metric Abuses (and their Potential Metric Antidotes)

[Stevan Harnad]

On Mon, 27 Jan 2003, [identity removed] wrote:

> In an editorial (citing *all* the articles in the journal to help increase
> the impact factor), the Editor of the Journal of Allergy and Immunology
> apparently insists that all accepted articles cite as many as possible (at
> least 20, I understand) articles from the journal itself in order to
> increase the impact factor. This strategy has also been adopted by American
> Journal of Respiratory and Critical Care Medicine and for revised
> manuscripts authors are urged to cite as many relevant articles from the
> journal in the last two years as possible.
>
> Just illustrating the level of 'objectivity' in some of these metrics.

Deliberate auto-citation (whether by an author citing himself, or
journal-articles citing their own journal) is the easiest trick to pull
-- agreed -- but also, in a digital database (like the open-access OAI
harvester across the entire institutionally self-archived peer-reviewed
journal literature), the easiest one to detect, check against baseline
data (auto/allo-citation ratios in other journals) and accordingly factor
out (or even penalize), if it proves to have been inflated!

(If you're worried about early detection, that's what time-series
analysis is for.)

Never underestimate the potential power of digitometric analysis on a
digital database! After all, we're only (!) talking about 20,000
journals, or 2,000,000 annual articles here!

Stevan Harnad