Future UK RAEs to be Metrics-Based

[Jonathan Levitt]

Hi,



As a student in London, I have found the debate on the UK RAE very
interesting.



My suggestion is that it is important to consider, not only which metric to
apply, but also other issues such as:

1) Whom to rate:  Currently departments are rated in the RAE.  Is it a good
idea for the RAE to allocate funding to departments rather than to the
researcher or the research team?  Currently, no funding would be allocated
to an exceptional researcher or research team at a lowly rated department.

2) How often to rate: Currently the rating takes place every six years.  Is
it a good idea to make funding decisions so infrequently?   I understand
that other funding bodies make these decisions on an annual basis.

3) How accessible to make the results: The results of the RAE exercises are
published on the Internet.  Is it a good idea to make these ratings so
widely accessible?  Before I applied to be a research student in London I
downloaded the RAE ratings of the departments in my field in London and
based my decision on where to study on the RAE ratings.



Best regards,

Jonathan.

----- Original Message -----
From: "David Goodman" <dgoodman at Princeton.EDU>
To: <SIGMETRICS at LISTSERV.UTK.EDU>
Sent: Wednesday, March 29, 2006 5:55 AM
Subject: Re: [SIGMETRICS] Future UK RAEs to be Metrics-Based


> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
> Writing from a perspective where the RAE does not have
> any personal practical significance, I comment that such
> measures have a remarkable tendency to
> correlate with the preconceived rankings.
>
> For an area  somewhat relevant to this list,
> journal selection and deselection, there are
> numerous factors besides IF. Librarians select among
> them to pretend to the faculty that objective decisions are
> being made. I have long experience in this art, and
> will be glad to demonstrate.
>
> Consider the importance of RAE to a department and an
> university. How could anyone in the UK could be
> both sufficently competent and  unbiased
> to decide on the rankings of UK departments? Perhaps they
> should judge Japanese departments and vice versa.
>
> Dr. David Goodman
> Associate Professor
> Palmer School of Library and Information Science
> Long Island University
> and formerly
> Princeton University Library
>
> dgoodman at liu.edu
> dgoodman at princeton.edu
>
> ----- Original Message -----
> From: Ian Rowlands <ir at SOI.CITY.AC.UK>
> Date: Tuesday, March 28, 2006 3:22 pm
> Subject: Re: [SIGMETRICS] Future UK RAEs to be Metrics-Based
> To: SIGMETRICS at listserv.utk.edu
>
>> Adminstrative info for SIGMETRICS (for example unsubscribe):
>> http://web.utk.edu/~gwhitney/sigmetrics.html
>>
>> Hi Peter
>>
>> No, I disagree.  This is where the discussion starts.  In the world of
>> evidence-based policy, metricians hold the key.  Metrics form
>> targets, targets
>> shape behaviour, behaviour determines outcomes.  The metrics -
>> outcomesrelationship needs to be explored.  Whether we like it or
>> not, this is our
>> responsibility, we are joined at the hip with the social
>> responsibilities that
>> come with judging others.
>>
>> Ian
>>
>> Quoting Peter Ingwersen <PI at DB.DK>:
>>
>> > Adminstrative info for SIGMETRICS (for example unsubscribe):
>> > http://web.utk.edu/~gwhitney/sigmetrics.html
>> >
>> > Dear Loet and Stevan et al. - I think we should stop this
>> discussion now,
>> > prior to too many mails arriving into our boxes for no use.
>> Please convey
>> > your discussion between you two. Loet is right in that the entire
>> issue is
>> > political AND that the metrics we possess are not strong enough to
>> > substitute human assessemnts, e.g., of how the research is actually
>> > performed in the labs (good Lab. practice, etc.). Stevan might
>> like the
>> > metrics, including fancy inlink analyses, not mentioned thus far,
>> but:> actually, the correlations referred to (published) cover only
>> the top-ranked
>> > (and low-ranked) institutions in the RAE rankings - not really
>> > distinguishing between the large portion of mid-positioned research
>> > institutions in the UK. Hence, all this talk of substitution by
>> metrics is
>> > beneficial/fair to some - not to the entire body of research. My
>> best> regards - Peter Ingwersen
>> >
>> > -----Original Message-----
>> > From: Loet Leydesdorff
>> > To: SIGMETRICS at LISTSERV.UTK.EDU
>> > Sent: 28-03-06 20:56
>> > Subject: Re: [SIGMETRICS] Future UK RAEs to be Metrics-Based
>> >
>> > Adminstrative info for SIGMETRICS (for example unsubscribe):
>> > http://web.utk.edu/~gwhitney/sigmetrics.html
>> >
>> > > Bref: Scrapping the non-metric white-elephant in favour of
>> > > existing metrics is the policy part; improving on existing
>> > > metrics is the research part.
>> >
>> > Thus, you wish to make the RAE completely technocratic, while we
>> know> that
>> > we don't have reliable models for giving this strong type of policy
>> > advice.
>> > If I translate this for the Netherlands--having a dual system
>> relying> more
>> > on peer review than the UK--I can see the advantages, but also the
>> > disadvantages. For one, it might give my unit more money!
>> >
>> > With best wishes,
>> >
>> >
>> > Loet
>> > ________________________________
>> > Loet Leydesdorff
>> > Amsterdam School of Communications Research (ASCoR),
>> > Kloveniersburgwal 48, 1012 CX Amsterdam.
>> > Tel.: +31-20- 525 6598; fax: +31-20- 525 3681;
>> > loet at leydesdorff.net ; http://www.leydesdorff.net/