Future UK RAEs to be Metrics-Based

Thu Apr 6 01:13:37 EDT 2006

If I correctly understand, you wish to say that any ranking of authors or
institutions is ultimately dependent on how the sets are defined. A
definition of sets on the basis of institutions would thus make the RAE
operation circular.

With best wishes,  Loet

________________________________
Loet Leydesdorff
Amsterdam School of Communications Research (ASCoR),
Kloveniersburgwal 48, 1012 CX Amsterdam.
Tel.: +31-20- 525 6598; fax: +31-20- 525 3681;
loet at leydesdorff.net ; http://www.leydesdorff.net/

> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics
> [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Stephen J Bensman
> Sent: Wednesday, April 05, 2006 11:16 PM
> To: SIGMETRICS at LISTSERV.UTK.EDU
> Subject: Re: [SIGMETRICS] Future UK RAEs to be Metrics-Based
>
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
> Loet,
> The principle in real estate is "location, location,
> location."  The principle in program evaluation is "set
> definition, set definition, set definition."  I pointed out
> in another posting that a major discovery of the 1993 NRC
> rating was that all previous ratings in the biosciences were
> incorrect due to an incorrect method of classification
> resulting in non-comparable sets.  I am somewhat proud that I
> was able to show to the NRC people how a change in
> classification method had an enormous impact on the ratings
> of LSU, turning us from a nonentity into something quite
> respectable and more in line with Louisiana's pioneering role
> in medicine through mainly the Ochsner Clinic and the first
> attempt at a charity hospital system.
>
> SB
>
>
>
>
> Loet Leydesdorff <loet at LEYDESDORFF.NET>@LISTSERV.UTK.EDU> on
> 03/31/2006
> 11:57:46 AM
>
> Please respond to ASIS&T Special Interest Group on Metrics
>        <SIGMETRICS at LISTSERV.UTK.EDU>
>
> Sent by:    ASIS&T Special Interest Group on Metrics
>        <SIGMETRICS at LISTSERV.UTK.EDU>
>
>
> To:    SIGMETRICS at LISTSERV.UTK.EDU
> cc:     (bcc: Stephen J Bensman/notsjb/LSU)
>
> Subject:    Re: [SIGMETRICS] Future UK RAEs to be Metrics-Based
>
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
> Dear Stephen,
>
> Although I am politically at the other end of the spectrum, I
> fully agree with your critique of the RAE. But the critique
> would equally hold for a "metric" that would rate departments
> against each other as proposed by some of our colleagues. The
> problem is to take departments as units of analysis.
>
> With best wishes,
>
>
> Loet
>
> > -----Original Message-----
> > From: ASIS&T Special Interest Group on Metrics
> > [mailto:SIGMETRICS at listserv.utk.edu] On Behalf Of Stephen J Bensman
> > Sent: Thursday, March 30, 2006 10:32 PM
> > To: SIGMETRICS at listserv.utk.edu
> > Subject: Re: [SIGMETRICS] Future UK RAEs to be Metrics-Based
> >
> > Adminstrative info for SIGMETRICS (for example unsubscribe):
> > http://web.utk.edu/~gwhitney/sigmetrics.html
> >
> > Gee, I consider myself anything but a cultural elitist.
> > After all, I work at LSU.  The basic problem of the RAE is
> that it is
> > biased against an institution like LSU.  At least under the
> American
> > system, good researchers at a place like LSU have an even chance to
> > obtain research funding, and many take advantage of this
> system.  That
> > way a good researcher maintains his independence and advance his
> > career.  This way LSU plays a major role as a launch pad for up and
> > coming scientists.  The British RAE always reminded me of
> the Tsarist
> > system of krugovaia poruka, where all the peasants of a
> commune were
> > held liable for communal taxes.  This was the taxation system of
> > serfdom, causing peasants to be chained to the commune, stifling
> > individual initiative, thereby causing agricultural stagnation, and
> > ultimately a violent revolution.
> >  If this makes me a cultural elitist, then so be it.
> >
> > SB
> >
> >
> >
> >
> > Phil Davis <pmd8 at CORNELL.EDU>@LISTSERV.UTK.EDU> on 03/30/2006
> > 02:09:28 PM
> >
> > Please respond to ASIS&T Special Interest Group on Metrics
> >        <SIGMETRICS at LISTSERV.UTK.EDU>
> >
> > Sent by:    ASIS&T Special Interest Group on Metrics
> >        <SIGMETRICS at LISTSERV.UTK.EDU>
> >
> >
> > To:    SIGMETRICS at LISTSERV.UTK.EDU
> > cc:     (bcc: Stephen J Bensman/notsjb/LSU)
> >
> > Subject:    Re: [SIGMETRICS] Future UK RAEs to be Metrics-Based
> >
> > Adminstrative info for SIGMETRICS (for example unsubscribe):
> > http://web.utk.edu/~gwhitney/sigmetrics.html
> >
> > Stephen, I wouldn't call you a "capitalist pig" but a
> willfully blind,
> > cultural elitist.  In countries where education is wholly
> (or mostly)
> > funded by the government -- not just the UK and Europe, but
> Canada and
> > others -- the government is concerned about making sure
> that everyone
> > gets some modicum of funding.  That does not mean a completely
> > equitable rationing system, but it ensures a base-level of
> funding.
> > In the United States, this base-level funding often comes
> from one's
> > own department or college.  Granted, the
> capitalist-approach you speak
> > of does reward the best and greatest, and this Winner-takes-all
> > approach does result in pioneering research, yet it only
> rewards the
> > few.
> >
> > --Phil Davis
> >
> >
> >
> > Stephen Bensman wrote:
> >
> > >Speaking as a capitalist pig, the entire RAE system is just another
> > example
> > >of socialists hoisting themselves on their own petards.
> > Point 1 below
> > >contains the essence of the problem.  The US has done
> > pioneering work
> > >on the evaluation of research-doctorate programs but was
> never silly
> > >enough
> > to
> > >allocate research resources on the basis of it.  Luckily
> > because these
> > >evaluations were usually screwed up in some way.  Allocation of
> > >research resources was always done on a project-by-project
> > basis by the
> > >NSF, NIH, and others, with experts in the fields evaluating
> > individual
> > >research proposals.  The Europeans have a tendency to overplan
> > >everything with disastrous consequences--the disaster in
> > Eastern Europe
> > >just being the latest example of it.
> > >
> > >SB
> >
>