Future UK RAEs to be Metrics-Based

[Stevan Harnad]

On Mon, 18 Sep 2006, Richard Poynder wrote:

> >CO:
> >However, thank heavens for (4) because, as you rightly point out, a 
> >metrics based system will sweep all this nonsense aside.
> 
> Does not a metrics based system pose some problems for the 
> humanities? I haven't followed the discussion closely, but I get the 
> feeling that this is primarily intended as a fix for the sciences 
> isn't it? How is it envisaged working in the humanities?

I hope Charles Oppenheim will respond directly about the studies he has done on
metrics in the RAE humanities disciplines. Meanwhile:

(1) Wherever anyone has checked the correlation between journal citation
counts and RAE outcome, the correlation has always been significant
and sizeable, hence predictive.  Humanities disciplines have not been
exceptions.

(2) In book-based fields, what has likewise not been looked at is
supplementing the journal-article citation metric with a book-citation
metric. (That's still metrics!)

(3) And then there are all the other candidate metrics, most still
untested: downloads, co-citations, hubs/authorities, recursive CiteRank,
download/citation growth parameters (latency, slope, peak, longevity),
semantic metrics, etc.

An a-priori declaration, free of any supporting evidence  -- by any
discipline today -- that its work is an exception, not assessable by
metrics, makes about as much sense as an a-priori declaration, without
any supporting evidence, that a discipline's work is not assessable by
any form of comparative performance evaluation at all. (Who's to say whether
subjective evaluations have any validity either?)

Stevan Harnad