"Bibliometric Distortion": The Babblarazzi Are At It Again...

[Jonathan Adams]

Exactly so.  So long as the indicators remain separate from policy
effects (such as funding) they remain sound.
The problem arises where indicators become a target for the purpose of
conducting policy, as Goddard (1975) suggested in regard to UK
economics:
http://en.wikipedia.org/wiki/Goodhart%27s_law
And Campbell (1976) in relation to social science
http://en.wikipedia.org/wiki/Campbell%27s_Law

Jonathan Adams
 
Director, Evidence Ltd
+ 44 113 384 5680
 

-----Original Message-----
From: ASIS&T Special Interest Group on Metrics
[mailto:SIGMETRICS at listserv.utk.edu] On Behalf Of Loet Leydesdorff
Sent: 16 November 2007 12:41
To: SIGMETRICS at listserv.utk.edu
Subject: Re: [SIGMETRICS] "Bibliometric Distortion": The Babblarazzi Are
At It Again...


If I correctly remember, the Leiden normalization implies that one
compares
the citation scores with the expected citation scores given the
publication
profile of a group. You are right that a game follows naturally: if one
publishes in journals below one's level, one can expect to obtain a
higher
than expected citation score. Since all distributions are skewed, this
effect would be reinforced. 

Hitherto, this has not been a major problem because the scores where not
directly related as input to funding. 

With best wishes, 


Loet 

> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics 
> [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Jonathan Adams
> Sent: Friday, November 16, 2007 1:26 PM
> To: SIGMETRICS at LISTSERV.UTK.EDU
> Subject: Re: [SIGMETRICS] "Bibliometric Distortion": The 
> Babblarazzi Are At It Again...
> 
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
> 
> I'm not planning this.  I understand the recommendation is from our
> colleagues at Leiden, in a report to HEFCE that will be made public
> later this month.
> So far as outputs in Thomson-indexed journals go, I think 
> it's feasible
> (and we have been analysing some scenarios using earlier data
> reconciliation and analyses we did for HEFCE) but I wouldn't recommend
> it because of the games playing that I suspect would ensue.
> 
> Jonathan Adams
>  
> Director, Evidence Ltd
> + 44 113 384 5680
>  
> 
> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics
> [mailto:SIGMETRICS at listserv.utk.edu] On Behalf Of Loet Leydesdorff
> Sent: 16 November 2007 12:05
> To: SIGMETRICS at listserv.utk.edu
> Subject: Re: [SIGMETRICS] "Bibliometric Distortion": The 
> Babblarazzi Are
> At It Again...
> 
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
> 
> > The proposal
> > is that post-2008 the metrics assessment would be of all output,
> > creating a profile and then deriving a metric derived from that.
> 
> Dear Jonathan, 
> 
> How are you planning to do this? Interesting. 
> 
> Best wishes, 
> 
> 
> Loet
>