JCR Total Cites Flaw Example

Sat Feb 23 10:03:37 EST 2008

Loet,
The problem with the WoS is that it designed to retrieve by the year being cited and not by the year during which the citations are being made.  To make sure that the title segments are equivalent, you have to be able to do the latter--define by the year during which the references are being made.  The ISI instructor thought that it could be done but that it would take several steps in which to do this.  For me this is crucial, because I think that total cites are better surrogates for what I needed--faculty ratings and library use.  It was your paper that put me on to this, and I realized that in working with total cites, I was working with a random congomlerate not representative of what I need.  This first thing you have to do is classify the data by biblliographic entry and subject, and it is not possible to do the former with the JCR.  To emphasize the importance of this, I am now going to cite Karl Pearson:

The importance of classification was emphasized by Karl Pearson (1911) in his magisterial work, The Grammar of Science, which he periodically revised and published in three editions from 1892 to 1911.  In this book Pearson listed as a key feature of the scientific method the "[c]areful and accurate classification of facts and observation of their correlation and sequence" (p. 37).  He drove this point home in the following passage:

                ...The classification of facts and the formation of absolute judgments upon 

               the basis of this classification--judgments independent of the idiosyncrasies 

                of the individual mind--essentially sum up the aim and method of modern science. 

               The scientific man has above all things to strive at self-elimination in his

               judgments, to provide an argument which is as true for each individual mind 

               as for his own.  The classification of facts, the recognition of their sequence and                                                                                                                      relative significance is the function of science, and the habit of forming 

               a judgment upon these facts unbiassed by personal feeling is characteristic 

               of what may be termed the scientific frame of mind.  (p. 6) 

In Pearson's opinion, "...whenever we come across a conclusion in a scientific work which does not flow from the classification of facts..., then we are dealing with bad science" (p. 10). 

SB

________________________________

From: ASIS&T Special Interest Group on Metrics on behalf of Loet Leydesdorff
Sent: Sat 2/23/2008 2:45 AM
To: SIGMETRICS at listserv.utk.edu
Subject: Re: [SIGMETRICS] JCR Total Cites Flaw Example

Dear Stephen, 

I now realized that the total cites on the WoS are dependent on which year one has as a starting year. This is not the case for the JCR data. OK: let's wait for the reactions of the colleagues from Thomson/ISI. 

Best wishes, 

Loet

________________________________

Loet Leydesdorff 
Amsterdam School of Communications Research (ASCoR), 
Kloveniersburgwal 48, 1012 CX Amsterdam. 
Tel.: +31-20- 525 6598; fax: +31-20- 525 3681 
loet at leydesdorff.net <mailto:loet at leydesdorff.net> ; http://www.leydesdorff.net/ <http://www.leydesdorff.net/>  

________________________________

	From: ASIS&T Special Interest Group on Metrics [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Stephen J Bensman
	Sent: Friday, February 22, 2008 10:21 PM
	To: SIGMETRICS at LISTSERV.UTK.EDU
	Subject: Re: [SIGMETRICS] JCR Total Cites Flaw Example

	Adminstrative info for SIGMETRICS (for example unsubscribe): http://web.utk.edu/~gwhitney/sigmetrics.html 

	I just did the cited reference search and got 2070.  The problem I have with doing this is whether this is equivalent to the cites during a given JCR year.  I have asked the ISI people on how to do this, and I will forward you their answer when I get it.  When I discussed it with the ISI instructor, it seemed more complex than that.  The problem is not getting the cites but restricting it to the cites during a given JCR year at least on the Web for the sake of equivalency.  There also seemed problems of different forms of entry requiring Boolean operators for capturing.  I hope to hear from her over the next week or so.  I am sure that it can be done, but I like my facts from horses' mouths.  

	Stephen J. Bensman

	LSU Libraries

	Louisiana State University

	Baton Rouge, LA   70803

	USA

	notsjb at lsu.edu

________________________________

	From: ASIS&T Special Interest Group on Metrics [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Loet Leydesdorff
	Sent: Friday, February 22, 2008 2:12 PM
	To: SIGMETRICS at LISTSERV.UTK.EDU
	Subject: Re: [SIGMETRICS] JCR Total Cites Flaw Example

	Dear Stephen, 

	This is not so difficult: You go to the WoS, Cited Reference Search, and type in "ANIM LEARN BEHAV" as the journal abbreviation. This leads today to a total cites of 1,958. 

	You can do the same with the CD-Rom version of the JCR. The JCR 2006, for example, gives you 1,803. On the webversion of the JCR, one can search with ISSN. 

	Best wishes, 

	Loet

________________________________

	Loet Leydesdorff 
	Amsterdam School of Communications Research (ASCoR), 
	Kloveniersburgwal 48, 1012 CX Amsterdam. 
	Tel.: +31-20- 525 6598; fax: +31-20- 525 3681 
	loet at leydesdorff.net <mailto:loet at leydesdorff.net> ; http://www.leydesdorff.net/ <http://www.leydesdorff.net/>  

________________________________

		From: ASIS&T Special Interest Group on Metrics [mailto:SIGMETRICS at listserv.utk.edu] On Behalf Of Stephen J Bensman
		Sent: Friday, February 22, 2008 8:11 PM
		To: SIGMETRICS at listserv.utk.edu
		Subject: [SIGMETRICS] JCR Total Cites Flaw Example

		Adminstrative info for SIGMETRICS (for example unsubscribe): http://web.utk.edu/~gwhitney/sigmetrics.html 

		Vis-à-vis what I wrote on the flaws of the JCR definition of bibliographic entity, I am posting the following concrete example

		In 2003 the journal Animal Learning & Behavior, which is published by the Psychonomic Society, changed its title.  This title was published since 1973, and 30 volumes had been published at the time of the change.  Its new title was Learning & Behavior, whose volume numbering began thus: Vol. 31, no. (Feb. 2003)-.  As can be seen, the volume numbering is consecutive for the two titles, and it remained a journal of the Psychonomic Society.  From the viewpoint of logic and AACR2 cataloging rules, these two titles form a single bibliographic entity, which we shall call a "journal."

		However, since these titles are not alphabetically consecutive, the JCR treats them as two different bibliographic entities, which we shall call "title segments."  The JCR citation counts relate only to these title segments.  In the 2004 JCR Learning & Behavior had 49 total cites and an impact factor of 1.030.  Since Animal Learning & Behavior fell within the 2-year time limit of the impact factor, it was still covered.  This title segment had 1,195 total cites and an impact factor of 2.059.  Going forward now to the 2006 JCR, we find that Learning & Behavior has 241 total cites and an impact factor of 1.926.  Since Animal Learning & Behavior is now outside the two-year impact factor limit, it has dropped from JCR coverage, and it is impossible to retrieve citation data on this title segment through the JCR.

		It is quite obvious from the above that 2006 JCR is giving quite a false picture of the importance of Learning & Behavior as measured by total cites, which I have found to be a better surrogate for expert ratings and library use than the impact factor.  The historical perspective has been lost due to a logical flaw in the definition of a bibliographic entity.  In any type of statistical research involving total cites, Learning & Behavior will appear as a screaming outlier that distorts all results.

		Since the fault cannot be corrected through the JCR, it becomes a question of how do you use Web of Science to recover the lost data.  If there is a solution to this problem, it should also be applicable to journals not covered in the JCRs. 

		Stephen J. Bensman

		LSU Libraries

		Louisiana State University

		Baton Rouge, LA   70803

		USA

		notsjb at lsu.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20080223/b1eec441/attachment.html>