Future UK RAEs to be Metrics-Based

[Stevan Harnad]

On Tue, 28 Mar 2006, Loet Leydesdorff wrote:

> I agree with this need for calibration, but it seems unlikely that this will
> be achieved within one RAE round. Thereafter, you have to wait four years
> for a second measurement point.

No, there are retro data available for the past RAEs as well. And the
calibration is to improve the predictive power of the RAE. To merely
duplicate its prior predictive power, the strongly correlated metrics are
already enough.

> Furthermore, there is the problem of how
> to combine the multivariate regression with the time series perspective. The
> former will provide you with comparative statics, but not with insights in
> the dynamics of the system (autocorrelation problems!). It seems that you
> are formulating a research program rather than a policy measure.

There's definitely an emerging research programme there (Open Access Web
Scientometrics) but there's no need for the results of the research to
be in in order to conclude that the wasteful, inefficient non-metric
components of the RAE should be replaced by the cheap, fast, efficient
metrics that are already highly correlated with the outcome. The
research programme is for *improving* on the prior outcomes.

Bref: Scrapping the non-metric white-elephant in favour of existing
metrics is the policy part; improving on existing metrics is the
research part.

> Perhaps, I lost you. I downloaded the report of the UK government which you
> cited, but it is more on technology and innovation than on science. After a
> lot of the usual rhetoric, they advise to develop "a metrics" without
> specifying anything.

The government's is just a policy document. Have a look at

    http://openaccess.eprints.org/index.php?/archives/75-guid.html

for a summary and at the references cited in

     Harnad, S., Carr, L., Brody, T. & Oppenheim, C. (2003) Mandated
     online RAE CVs Linked to University Eprint Archives: Improving the
     UK Research Assessment Exercise whilst making it cheaper and easier.
     Ariadne 35.  http://www.ariadne.ac.uk/issue35/harnad/

for some of the correlational evidence.

Stevan Harnad
American Scientist Open Access Forum
http://amsci-forum.amsci.org/archives/American-Scientist-Open-Access-Forum.html

Chaire de recherche du Canada           Professor of Cognitive Science
Ctr. de neuroscience de la cognition    Dpt. Electronics & Computer Science
Université du Québec à Montréal         University of Southampton
Montréal, Québec                        Highfield, Southampton
Canada  H3C 3P8                         SO17 1BJ United Kingdom
http://www.crsc.uqam.ca/                http://www.ecs.soton.ac.uk/~harnad/