accuracy of Thomson data

Fri Dec 21 11:31:00 EST 2007

Dear Stephen, 

As you know, I was interested in the so-called "externally-cited impact
factor" of a journal which is not included in the set and then I realized
that one can search on non-includied journals. In the Web-version, it is
easy. You can just type "J PHYS CHEM-US" in the cited reference search or if
you wish "WALL STREET J" and you get the citations. 

It seems virtually impossible for me for the ISI (or Scopus) to attribute
the citations to "J Phys Chem-US" to either "J Phys Chem A" or "J Phys Chem
B". The same holds true for the BBA-volumes, etc. I don't blame them, but
one should be aware of the problem. This underestimates the citations, but
as you know we found also 21.5% overestimation in the case of "Angewandte
Chemie" because of people citing both editions. However, the group of
journals for which this latter effect may happen, can be delimited.

You don't mind if I am not going to explain the vector space model. I think
the reference is Salton & McGill (1983) and if I remember correctly, the
explanation is rather algebraic.

With best wishes, 

Loet

________________________________

Loet Leydesdorff 
Amsterdam School of Communications Research (ASCoR), 
Kloveniersburgwal 48, 1012 CX Amsterdam. 
Tel.: +31-20- 525 6598; fax: +31-20- 525 3681 
loet at leydesdorff.net ; http://www.leydesdorff.net/ 

> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics 
> [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Stephen J Bensman
> Sent: Friday, December 21, 2007 4:01 PM
> To: SIGMETRICS at LISTSERV.UTK.EDU
> Subject: Re: [SIGMETRICS] accuracy of Thomson data
> 
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
> 
> Loet,
> In re to your first paragraph, if you read the paper that is posted on
> Dr. Garfield's Web site, you will see that the probability was very
> stable across time both in terms of the Impact Factor and Total Cites
> not at the aggregate level but within one discipline--chemistry.  This
> stability is essentially a reflection of the stability of the 
> underlying
> social stratification system of chemistry.  The only consistent change
> in probability was the continuing increase in dominance of 
> the journals
> at the top of both measures, indicating the success-breeds-success
> mechanism of the Matthew Effect--technically known as "contagion" in
> statistics.  This is the basic reason for the stability of both the
> social stratification system and the journal system based upon it.  
> 
> In re your method, I must admit that generally it shoots right by me.
> When I was young, a mathematics teacher once told me that minds can be
> divided into two types--those that think linearly and are good in
> algebra, and those that think spatially and are good in geometry.  The
> teacher said that people very good in algebra often have hard time in
> geometry.  I was extremely good in algebra but had a hard time in
> geometry due to difficulty in grasping spatial relationships. 
>  It seems
> that computer programmers have to be able to think spatially as well,
> and I have a hard time understanding this.
> 
> However, your most important recent discovery--in my opinion--was in
> your recent "Caveats" paper, where you prove that JCRs do not 
> aggregate
> Total Cites across title changes, splits, etc.  In other 
> words, much of
> the backfile is left out of the count.  From my historical 
> perspective,
> this is a most serious flaw.  You also have to understand that I am a
> catalog librarian and define serial bibliographic entities in terms of
> volume count--if the volume sequence is consistent, it is 
> still the same
> journal despite the title change.  I investigated the problem briefly
> and found that I could not aggregate more than three years.  I would
> like to know how you did that.  For me it means that for 
> Total Cites to
> be good, Total Cites across 3 years have to be highly correlated with
> Total Cites across the entire backfile.  An initial view was 
> that such a
> short backfile was a sufficient sample.  If this is so, then the Total
> Cites measure is good. 
> 
> Stephen J. Bensman
> LSU Libraries
> Louisiana State University
> Baton Rouge, LA   70803
> USA
> notsjb at lsu.edu
> 
> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics
> [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Loet Leydesdorff
> Sent: Friday, December 21, 2007 1:05 AM
> To: SIGMETRICS at LISTSERV.UTK.EDU
> Subject: Re: [SIGMETRICS] accuracy of Thomson data
> 
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
> 
> Dear Stephen and colleagues, 
> 
> These (Spearman) correlations teach us, in my opinion, that 
> the measures
> covary across disciplines at the aggregate level. Since impact factors
> are
> higher in the biomedical sciences, library usages and expert ratings
> would
> be expected to be higher in these sciences as well?
> 
> I would expect the c/p ratio, the impact factor, and the immediacy
> factor to
> correlate highly as one group, but total publication, total citations,
> library usage, expert ratings, etc., as a second group (size 
> related). 
> 
> As you know, my preference goes in the direction of mapping local
> citation
> impact environments.  These can be visualized using, for example, the
> files
> at http://www.leydesdorff.net/jcr06 . (See: Visualization of the
> Citation
> Impact Environments of Scientific Journals: An online mapping 
> exercise,
> Journal of the American Society for Information Science and Technology
> 58(1), 25-38, 2007.) The normalization in terms of the vector space
> (cosine)
> takes care of the size effects in the relations (links), and 
> size can be
> considered as an attribute of the nodes. 
> 
> I have no better operationalization at the moment, but I am open for
> suggestions.
> 
> Best wishes, 
> 
> 
> Loet
> 
> ________________________________
> 
> Loet Leydesdorff 
> Amsterdam School of Communications Research (ASCoR), 
> Kloveniersburgwal 48, 1012 CX Amsterdam. 
> Tel.: +31-20- 525 6598; fax: +31-20- 525 3681 
> loet at leydesdorff.net ; http://www.leydesdorff.net/ 
> 
>  
> 
> > -----Original Message-----
> > From: ASIS&T Special Interest Group on Metrics 
> > [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Stephen J. Bensman
> > Sent: Thursday, December 20, 2007 10:37 PM
> > To: SIGMETRICS at LISTSERV.UTK.EDU
> > Subject: Re: [SIGMETRICS] accuracy of Thomson data
> > 
> > Adminstrative info for SIGMETRICS (for example unsubscribe):
> > http://web.utk.edu/~gwhitney/sigmetrics.html
> > 
> > I try to do some of this in the paper posted on Dr. 
> > Garfield's Web site at:
> > 
> >  
> > 
> > http://garfield.library.upenn.edu/bensman/bensmanegif22007.pdf
> > 
> >  
> > 
> > You might want to look at the second half of the paper, where 
> > I discuss 
> > the Impact Factor in terms of Poisson lambdas, sampling 
> > variance, random 
> > error, etc.  The amazing thing to me, at least, is that 
> > despite all the 
> > random error and sampling variance, there is a remarkable 
> > stability of 
> > probability across time with Spearman rhos of 0.9 and above 
> with high 
> > respectable correlations with Total Cites, library use, and expert 
> > ratings.  Most impact factors move up and down within 
> > extremely narrow 
> > limits across time.  I found a similar phenomenon in a paper 
> > just accepted 
> > by JASIST called "Distributional Differences of the Impact 
> > Factor in the 
> > Sciences vs. the Social Sciences: An Analysis of the Probabilistic 
> > Structure of the 2005 Journal Citation Reports."   I no 
> > longer own the 
> > copyright and so cannot post it, but I suppose that I can let 
> > you read it 
> > on a private basis, if you're willing to suffer the pain of 
> > reading it.  
> > There is much more to the Impact Factor than meets the eye, 
> > and it is an 
> > extremely good measure for many purposes, if of extremely 
> > doubtful use for 
> > ranking purposes in the vast bulk of the cases.   
> > 
> >  
> > 
> > Stephen J. Bensman, Ph.D.
> > LSU Libraries
> > Louisiana State University
> > Baton Rouge, LA   70803
> > USA
> > notsjb at lsu.edu
> > 
> 