Future UK RAEs to be Metrics-Based

[Stevan Harnad]

On Thu, 23 Mar 2006, Ian Sommerville wrote:

> This is the wording from the budget document
>
> "The Government
> wants this to continue, but thinks the close correlation between
> Research Council income
> and QR income may provide an opportunity for allocating QR using a
> radically simpler
> system. "
>
> The point is made that, at an institutional level, there is a 0.98
> correlation between research income and QR. No mention of citation
> impact. An alternative metric may be proposed for the humanities.

You are quite right that the default metric many have in mind is research
income, but be patient! Now that the door has been opened to objective
metrics, this will spawn more and more candidates for enriching the metric
equation. If RAE top-slicing seeks to be an independent funding source,
it will want to have some predictive metrics that are independent of
prior funding; if it just wants to redundantly follow research funding,
it need merely scale up research grants to absorb what would have been
the RAE top-slice!

The important thing is to scrap the useless, time-wasting RAE
preparation/evaluation ritual we were all faithfully performing, when the
outcome was already so predictable from other, cheaper sources. Objective
metrics are the natural, sensible way to conduct such an exercise,
and once we are doing metrics, many powerful new predictive measures will
present themselves, over and above grant income and citations. The ranking
will not come from one variable, but from a multiple regression equation,
with many weighted predictor metrics.

Stevan Harnad
Professor of Cognitive Science
Department of Electronics and Computer Science
University of Southampton
Highfield, Southampton
SO17 1BJ UNITED KINGDOM
phone: +44 23-80 592-388
fax:   +44 23-80 592-865
harnad at ecs.soton.ac.uk
http://www.ecs.soton.ac.uk/~harnad/

> Ian
>
>
>
>
> On 23 Mar 2006, at 14:47, Stevan Harnad wrote:
>
> >     ** Apologies for Cross-Posting **
> >
> > Don't say we didn't tell you so! The wasteful, time-consuming RAE will
> > be replaced by metrics, chief among them citation impact, which
> > already
> > correlated with and predicted the RAE outcome anyway, without being
> > explicitly counted. Now it can be explicitly counted (along with other
> > powerful new metrics) and all the rest of the ritualistic time-wasting
> > can be abandoned, without ceremony. This is a great boost for
> > institutional self-archiving in OA Institutional Repositories, not
> > only
> > as the obvious, optimal means of submission, but as the means of
> > maximising research impact:  http://irra.eprints.org/software/bronze/
> > (I hope RCUK is listening!):
> >
> >     "Research exercise to be scrapped"
> >      Donald MacLeod, Guardian Wednesday March 22, 2006
> >      http://education.guardian.co.uk/RAE/story/0,,1737082,00.html
> >
> > Cf: Harnad, S. (2001) Why I think that research access, impact and
> > assessment
> >     are linked. Times Higher Education Supplement 1487: p. 16.
> >     http://www.thes.co.uk/search/story.aspx?story_id=74889
> >     http://cogprints.org/1683/
> >
> >     Harnad, S. (2003) Why I believe that all UK research output
> >     should be online.
> >     Times Higher Education Supplement. Friday, June 6 2003.
> >     http://www.thes.co.uk/search/story.aspx?story_id=92599
> >     http://eprints.ecs.soton.ac.uk/7728/
> >
> >     Harnad, S., Carr, L., Brody, T. & Oppenheim, C. (2003) Mandated
> >     online RAE CVs Linked to University Eprint Archives: Improving
> >     the UK Research Assessment Exercise whilst making it cheaper and
> >     easier. Ariadne.
> >     http://www.ariadne.ac.uk/issue35/harnad/
>
>