Future UK RAEs to be Metrics-Based

[Stephen J Bensman]

Speaking as a capitalist pig, the entire RAE system is just another example
of socialists hoisting themselves on their own petards.  Point 1 below
contains the essence of the problem.  The US has done pioneering work on
the evaluation of research-doctorate programs but was never silly enough to
allocate research resources on the basis of it.  Luckily because these
evaluations were usually screwed up in some way.  Allocation of research
resources was always done on a project-by-project basis by the NSF, NIH,
and others, with experts in the fields evaluating individual research
proposals.  The Europeans have a tendency to overplan everything with
disastrous consequences--the disaster in Eastern Europe just being the
latest example of it.

SB




Jonathan Levitt <jonathan at LEVITT.NET>@LISTSERV.UTK.EDU> on 03/30/2006
10:45:44 AM

Please respond to ASIS&T Special Interest Group on Metrics
       <SIGMETRICS at LISTSERV.UTK.EDU>

Sent by:    ASIS&T Special Interest Group on Metrics
       <SIGMETRICS at LISTSERV.UTK.EDU>


To:    SIGMETRICS at LISTSERV.UTK.EDU
cc:     (bcc: Stephen J Bensman/notsjb/LSU)

Subject:    Re: [SIGMETRICS] Future UK RAEs to be Metrics-Based


Hi,



As a student in London, I have found the debate on the UK RAE very
interesting.



My suggestion is that it is important to consider, not only which metric to
apply, but also other issues such as:

1) Whom to rate:  Currently departments are rated in the RAE.  Is it a good
idea for the RAE to allocate funding to departments rather than to the
researcher or the research team?  Currently, no funding would be allocated
to an exceptional researcher or research team at a lowly rated department.

2) How often to rate: Currently the rating takes place every six years.  Is
it a good idea to make funding decisions so infrequently?   I understand
that other funding bodies make these decisions on an annual basis.

3) How accessible to make the results: The results of the RAE exercises are
published on the Internet.  Is it a good idea to make these ratings so
widely accessible?  Before I applied to be a research student in London I
downloaded the RAE ratings of the departments in my field in London and
based my decision on where to study on the RAE ratings.



Best regards,

Jonathan.



----- Original Message -----
From: "Stevan Harnad" <harnad at ECS.SOTON.AC.UK>
To: <SIGMETRICS at listserv.utk.edu>
Sent: Thursday, March 23, 2006 3:47 PM
Subject: [SIGMETRICS] Future UK RAEs to be Metrics-Based


> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
>    ** Apologies for Cross-Posting **
>
> Don't say we didn't tell you so! The wasteful, time-consuming RAE will
> be replaced by metrics, chief among them citation impact, which already
> correlated with and predicted the RAE outcome anyway, without being
> explicitly counted. Now it can be explicitly counted (along with other
> powerful new metrics) and all the rest of the ritualistic time-wasting
> can be abandoned, without ceremony. This is a great boost for
> institutional self-archiving in OA Institutional Repositories, not only
> as the obvious, optimal means of submission, but as the means of
> maximising research impact:  http://irra.eprints.org/software/bronze/
> (I hope RCUK is listening!):
>
>    "Research exercise to be scrapped"
>     Donald MacLeod, Guardian Wednesday March 22, 2006
>     http://education.guardian.co.uk/RAE/story/0,,1737082,00.html
>
> Cf: Harnad, S. (2001) Why I think that research access, impact and
> assessment
>    are linked. Times Higher Education Supplement 1487: p. 16.
>    http://www.thes.co.uk/search/story.aspx?story_id=74889
>    http://cogprints.org/1683/
>
>    Harnad, S. (2003) Why I believe that all UK research output
>    should be online.
>    Times Higher Education Supplement. Friday, June 6 2003.
>    http://www.thes.co.uk/search/story.aspx?story_id=92599
>    http://eprints.ecs.soton.ac.uk/7728/
>
>    Harnad, S., Carr, L., Brody, T. & Oppenheim, C. (2003) Mandated
>    online RAE CVs Linked to University Eprint Archives: Improving
>    the UK Research Assessment Exercise whilst making it cheaper and
>    easier. Ariadne.
>    http://www.ariadne.ac.uk/issue35/harnad/