accuracy of Thomson data

[Jean-Claude Guédon]

All this discussion leads to an interesting question:

Routinely - in a recent (and very useful) paper ("Caveats for the Use of
Citation Indicators...", Loet Leydesdorff uses the expression
"codification process" - impact factors are given with three decimals.
When I used to study chemistry, eons ago, my teachers used to tell us
that measurements provided as is with no estimate measurement errors
were simply meaningless. Do citation measurements escape this
fundamental rule? I think not, especially when I see three or four
significant figures being produced without anyone batting a single
eyelash...

When I see journal editors battling to get past their nearest competitor
at the level of the third or fourth significant figures when we all know
that these are the artifacts of calculations, not reflections of
reality, I cannot help smile; or sigh, for such a treatment of figures
intensifies the role of the impact factor as an arbiter of journal
competition, which is very good for Thompson Scientific's business, but
not much else. Perhaps some people at Scopus, etc. could produce a good
statistical study of this phenomenon... :-) 

Imagine documenting the fact that impact factors are known ±30% ... 

Has anyone ever tried to put a general error figure on impact (and
impact factors) measurements?

Best,

Jean-Claude Guédon

Le jeudi 20 décembre 2007 à 09:40 -0600, Thomas Krichel a écrit :

> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
> 
>   On the ISI accuracy debate, here is an interesting 
>   contribution by Mark Doyle on the PAM mailing list.
> 
>   Cheers,
> 
>   Thomas Krichel                    http://openlib.org/home/krichel
>                                 RePEc:per:1965-06-05:thomas_krichel
>                                                skype: thomaskrichel
> 
> 
> 
> 
> 
> Hi,
> 
> On Dec 20, 2007, at 8:35 AM, A. Ben Wagner wrote:
> 
> > Note also the errors are often not Thomson's fault,
> > but the result of notoriously poor citation checking by journal
> > authors and
> > editors.  And it seems to be getting worse in the Internet age when
> > any one
> > including a 5th grader can put a bibliography out on the web.
> 
> This simply is not true. Most, if not all, major publishers (and
> many, many more) now check references against CrossRef and get DOIs.
> In doing the PROLA archive, we tagged references for material from
> 1893 to 1995 and there was about a 10% error rate in citations over
> this time period. Nowadays, the error rate in our citations is well
> under 1% because of CrossRef. For PRL and other APS journals (and
> even Nature), we have found error rates (these are undercounts) in
> ISI's data ranging from about 17% to 25% when comparing Web of
> Science and JCR data. It is inexcusable really. ISI could be doing
> MUCH better for WoS and JCR by using XML references and metadata
> tagged with DOIs from publishers to feed their process, but they have
> shown little interest in doing this (at least for our material).
> Scopus, for instance, gets data about our citations from us this way.
> 
> Best,
> Mark
> 
> Mark Doyle
> Assistant Director, Journal Information Systems
> The American Physical Society
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20071220/0afb2c7e/attachment.html>