accuracy of Thomson data

[Stephen J Bensman]

Loet,
In re to your first paragraph, if you read the paper that is posted on
Dr. Garfield's Web site, you will see that the probability was very
stable across time both in terms of the Impact Factor and Total Cites
not at the aggregate level but within one discipline--chemistry.  This
stability is essentially a reflection of the stability of the underlying
social stratification system of chemistry.  The only consistent change
in probability was the continuing increase in dominance of the journals
at the top of both measures, indicating the success-breeds-success
mechanism of the Matthew Effect--technically known as "contagion" in
statistics.  This is the basic reason for the stability of both the
social stratification system and the journal system based upon it.  

In re your method, I must admit that generally it shoots right by me.
When I was young, a mathematics teacher once told me that minds can be
divided into two types--those that think linearly and are good in
algebra, and those that think spatially and are good in geometry.  The
teacher said that people very good in algebra often have hard time in
geometry.  I was extremely good in algebra but had a hard time in
geometry due to difficulty in grasping spatial relationships.  It seems
that computer programmers have to be able to think spatially as well,
and I have a hard time understanding this.

However, your most important recent discovery--in my opinion--was in
your recent "Caveats" paper, where you prove that JCRs do not aggregate
Total Cites across title changes, splits, etc.  In other words, much of
the backfile is left out of the count.  From my historical perspective,
this is a most serious flaw.  You also have to understand that I am a
catalog librarian and define serial bibliographic entities in terms of
volume count--if the volume sequence is consistent, it is still the same
journal despite the title change.  I investigated the problem briefly
and found that I could not aggregate more than three years.  I would
like to know how you did that.  For me it means that for Total Cites to
be good, Total Cites across 3 years have to be highly correlated with
Total Cites across the entire backfile.  An initial view was that such a
short backfile was a sufficient sample.  If this is so, then the Total
Cites measure is good. 

Stephen J. Bensman
LSU Libraries
Louisiana State University
Baton Rouge, LA   70803
USA
notsjb at lsu.edu

-----Original Message-----
From: ASIS&T Special Interest Group on Metrics
[mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Loet Leydesdorff
Sent: Friday, December 21, 2007 1:05 AM
To: SIGMETRICS at LISTSERV.UTK.EDU
Subject: Re: [SIGMETRICS] accuracy of Thomson data


Dear Stephen and colleagues, 

These (Spearman) correlations teach us, in my opinion, that the measures
covary across disciplines at the aggregate level. Since impact factors
are
higher in the biomedical sciences, library usages and expert ratings
would
be expected to be higher in these sciences as well?

I would expect the c/p ratio, the impact factor, and the immediacy
factor to
correlate highly as one group, but total publication, total citations,
library usage, expert ratings, etc., as a second group (size related). 

As you know, my preference goes in the direction of mapping local
citation
impact environments.  These can be visualized using, for example, the
files
at http://www.leydesdorff.net/jcr06 . (See: Visualization of the
Citation
Impact Environments of Scientific Journals: An online mapping exercise,
Journal of the American Society for Information Science and Technology
58(1), 25-38, 2007.) The normalization in terms of the vector space
(cosine)
takes care of the size effects in the relations (links), and size can be
considered as an attribute of the nodes. 

I have no better operationalization at the moment, but I am open for
suggestions.

Best wishes, 


Loet

________________________________

Loet Leydesdorff 
Amsterdam School of Communications Research (ASCoR), 
Kloveniersburgwal 48, 1012 CX Amsterdam. 
Tel.: +31-20- 525 6598; fax: +31-20- 525 3681 
loet at leydesdorff.net ; http://www.leydesdorff.net/ 

 

> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics 
> [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Stephen J. Bensman
> Sent: Thursday, December 20, 2007 10:37 PM
> To: SIGMETRICS at LISTSERV.UTK.EDU
> Subject: Re: [SIGMETRICS] accuracy of Thomson data
> 
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
> 
> I try to do some of this in the paper posted on Dr. 
> Garfield's Web site at:
> 
>  
> 
> http://garfield.library.upenn.edu/bensman/bensmanegif22007.pdf
> 
>  
> 
> You might want to look at the second half of the paper, where 
> I discuss 
> the Impact Factor in terms of Poisson lambdas, sampling 
> variance, random 
> error, etc.  The amazing thing to me, at least, is that 
> despite all the 
> random error and sampling variance, there is a remarkable 
> stability of 
> probability across time with Spearman rhos of 0.9 and above with high 
> respectable correlations with Total Cites, library use, and expert 
> ratings.  Most impact factors move up and down within 
> extremely narrow 
> limits across time.  I found a similar phenomenon in a paper 
> just accepted 
> by JASIST called "Distributional Differences of the Impact 
> Factor in the 
> Sciences vs. the Social Sciences: An Analysis of the Probabilistic 
> Structure of the 2005 Journal Citation Reports."   I no 
> longer own the 
> copyright and so cannot post it, but I suppose that I can let 
> you read it 
> on a private basis, if you're willing to suffer the pain of 
> reading it.  
> There is much more to the Impact Factor than meets the eye, 
> and it is an 
> extremely good measure for many purposes, if of extremely 
> doubtful use for 
> ranking purposes in the vast bulk of the cases.   
> 
>  
> 
> Stephen J. Bensman, Ph.D.
> LSU Libraries
> Louisiana State University
> Baton Rouge, LA   70803
> USA
> notsjb at lsu.edu
>