JCR Total Cites Flaw Example

Sat Feb 23 14:33:58 EST 2008

Loet, 
Thanks for the information on the difference between the CD ROM version and Web version of the JCRs.  When I return to work Monday, I will check whether LSU has the CD-ROM version.  However, in 2004 we switched from the CD-ROM to the Web version, I thought this a big improvement as I had desktop delivery.  LSU will not fork out for both versions, if it requires another subscription.  However, it is surprising that Thomson would do such a thing--making a backward technology have more information than an advanced technology.  LSU is trying to create a virtual library because the faculty do not want to come to a library in another building.  They want desktop delivery, and so do I.  Paper is out of the question, as a key physics journal grows at 15-20 linear feet per year, and we are out of physical capacity. If the CD-ROM version can do this, then it should be simple for them to make it a feature of the Web version.

This is also a crucial issue for you, because the citing/cited, half-life etc. packages are all based on total cites.  It means that if you cannot define your bibliographic entities properly, all your pretty pictures may not be true reflections of reality.

SB

________________________________

From: ASIS&T Special Interest Group on Metrics on behalf of Loet Leydesdorff
Sent: Sat 2/23/2008 10:14 AM
To: SIGMETRICS at LISTSERV.UTK.EDU
Subject: Re: [SIGMETRICS] JCR Total Cites Flaw Example

Dear Stephen, 

Using the CD-Rom version of the JCR, one can retrieve 1803 total cites for "Anim Learn Behav" in 2006. However, in the webversion this number is no longer available (because of the two-year time windown for the impact factor). 

The WoS version of the database itself (SCI) I get 799 hits for "Anim Learn Behav" as cited reference work, but one would have to add the numbers in the column "citing articles" to find the number of "total cites." That is a lot of work.

Best wishes, 

Loet

________________________________

Loet Leydesdorff 
Amsterdam School of Communications Research (ASCoR), 
Kloveniersburgwal 48, 1012 CX Amsterdam. 
Tel.: +31-20- 525 6598; fax: +31-20- 525 3681 
loet at leydesdorff.net <mailto:loet at leydesdorff.net> ; http://www.leydesdorff.net/ <http://www.leydesdorff.net/>  

________________________________

	From: ASIS&T Special Interest Group on Metrics [mailto:SIGMETRICS at listserv.utk.edu] On Behalf Of Stephen J Bensman
	Sent: Saturday, February 23, 2008 4:04 PM
	To: SIGMETRICS at listserv.utk.edu
	Subject: Re: [SIGMETRICS] JCR Total Cites Flaw Example

	Adminstrative info for SIGMETRICS (for example unsubscribe): http://web.utk.edu/~gwhitney/sigmetrics.html 
	Loet,
	The problem with the WoS is that it designed to retrieve by the year being cited and not by the year during which the citations are being made.  To make sure that the title segments are equivalent, you have to be able to do the latter--define by the year during which the references are being made.  The ISI instructor thought that it could be done but that it would take several steps in which to do this.  For me this is crucial, because I think that total cites are better surrogates for what I needed--faculty ratings and library use.  It was your paper that put me on to this, and I realized that in working with total cites, I was working with a random congomlerate not representative of what I need.  This first thing you have to do is classify the data by biblliographic entry and subject, and it is not possible to do the former with the JCR.  To emphasize the importance of this, I am now going to cite Karl Pearson:

	The importance of classification was emphasized by Karl Pearson (1911) in his magisterial work, The Grammar of Science, which he periodically revised and published in three editions from 1892 to 1911.  In this book Pearson listed as a key feature of the scientific method the "[c]areful and accurate classification of facts and observation of their correlation and sequence" (p. 37).  He drove this point home in the following passage:

	                ...The classification of facts and the formation of absolute judgments upon 

	               the basis of this classification--judgments independent of the idiosyncrasies 

	                of the individual mind--essentially sum up the aim and method of modern science. 

	               The scientific man has above all things to strive at self-elimination in his

	               judgments, to provide an argument which is as true for each individual mind 

	               as for his own.  The classification of facts, the recognition of their sequence and                                                                                                                      relative significance is the function of science, and the habit of forming 

	               a judgment upon these facts unbiassed by personal feeling is characteristic 

	               of what may be termed the scientific frame of mind.  (p. 6) 

	In Pearson's opinion, "...whenever we come across a conclusion in a scientific work which does not flow from the classification of facts..., then we are dealing with bad science" (p. 10). 

	SB

________________________________

	From: ASIS&T Special Interest Group on Metrics on behalf of Loet Leydesdorff
	Sent: Sat 2/23/2008 2:45 AM
	To: SIGMETRICS at listserv.utk.edu
	Subject: Re: [SIGMETRICS] JCR Total Cites Flaw Example

	Adminstrative info for SIGMETRICS (for example unsubscribe): http://web.utk.edu/~gwhitney/sigmetrics.html 
	Dear Stephen, 

	I now realized that the total cites on the WoS are dependent on which year one has as a starting year. This is not the case for the JCR data. OK: let's wait for the reactions of the colleagues from Thomson/ISI. 

	Best wishes, 

	Loet

________________________________

	Loet Leydesdorff 
	Amsterdam School of Communications Research (ASCoR), 
	Kloveniersburgwal 48, 1012 CX Amsterdam. 
	Tel.: +31-20- 525 6598; fax: +31-20- 525 3681 
	loet at leydesdorff.net <mailto:loet at leydesdorff.net> ; http://www.leydesdorff.net/ <http://www.leydesdorff.net/>  

________________________________

		From: ASIS&T Special Interest Group on Metrics [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Stephen J Bensman
		Sent: Friday, February 22, 2008 10:21 PM
		To: SIGMETRICS at LISTSERV.UTK.EDU
		Subject: Re: [SIGMETRICS] JCR Total Cites Flaw Example

		Adminstrative info for SIGMETRICS (for example unsubscribe): http://web.utk.edu/~gwhitney/sigmetrics.html 

		I just did the cited reference search and got 2070.  The problem I have with doing this is whether this is equivalent to the cites during a given JCR year.  I have asked the ISI people on how to do this, and I will forward you their answer when I get it.  When I discussed it with the ISI instructor, it seemed more complex than that.  The problem is not getting the cites but restricting it to the cites during a given JCR year at least on the Web for the sake of equivalency.  There also seemed problems of different forms of entry requiring Boolean operators for capturing.  I hope to hear from her over the next week or so.  I am sure that it can be done, but I like my facts from horses' mouths.  

		Stephen J. Bensman

		LSU Libraries

		Louisiana State University

		Baton Rouge, LA   70803

		USA

		notsjb at lsu.edu

________________________________

		From: ASIS&T Special Interest Group on Metrics [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Loet Leydesdorff
		Sent: Friday, February 22, 2008 2:12 PM
		To: SIGMETRICS at LISTSERV.UTK.EDU
		Subject: Re: [SIGMETRICS] JCR Total Cites Flaw Example

		Dear Stephen, 

		This is not so difficult: You go to the WoS, Cited Reference Search, and type in "ANIM LEARN BEHAV" as the journal abbreviation. This leads today to a total cites of 1,958. 

		You can do the same with the CD-Rom version of the JCR. The JCR 2006, for example, gives you 1,803. On the webversion of the JCR, one can search with ISSN. 

		Best wishes, 

		Loet

________________________________

		Loet Leydesdorff 
		Amsterdam School of Communications Research (ASCoR), 
		Kloveniersburgwal 48, 1012 CX Amsterdam. 
		Tel.: +31-20- 525 6598; fax: +31-20- 525 3681 
		loet at leydesdorff.net <mailto:loet at leydesdorff.net> ; http://www.leydesdorff.net/ <http://www.leydesdorff.net/>  

________________________________

			From: ASIS&T Special Interest Group on Metrics [mailto:SIGMETRICS at listserv.utk.edu] On Behalf Of Stephen J Bensman
			Sent: Friday, February 22, 2008 8:11 PM
			To: SIGMETRICS at listserv.utk.edu
			Subject: [SIGMETRICS] JCR Total Cites Flaw Example

			Adminstrative info for SIGMETRICS (for example unsubscribe): http://web.utk.edu/~gwhitney/sigmetrics.html 

			Vis-à-vis what I wrote on the flaws of the JCR definition of bibliographic entity, I am posting the following concrete example

			In 2003 the journal Animal Learning & Behavior, which is published by the Psychonomic Society, changed its title.  This title was published since 1973, and 30 volumes had been published at the time of the change.  Its new title was Learning & Behavior, whose volume numbering began thus: Vol. 31, no. (Feb. 2003)-.  As can be seen, the volume numbering is consecutive for the two titles, and it remained a journal of the Psychonomic Society.  From the viewpoint of logic and AACR2 cataloging rules, these two titles form a single bibliographic entity, which we shall call a "journal."

			However, since these titles are not alphabetically consecutive, the JCR treats them as two different bibliographic entities, which we shall call "title segments."  The JCR citation counts relate only to these title segments.  In the 2004 JCR Learning & Behavior had 49 total cites and an impact factor of 1.030.  Since Animal Learning & Behavior fell within the 2-year time limit of the impact factor, it was still covered.  This title segment had 1,195 total cites and an impact factor of 2.059.  Going forward now to the 2006 JCR, we find that Learning & Behavior has 241 total cites and an impact factor of 1.926.  Since Animal Learning & Behavior is now outside the two-year impact factor limit, it has dropped from JCR coverage, and it is impossible to retrieve citation data on this title segment through the JCR.

			It is quite obvious from the above that 2006 JCR is giving quite a false picture of the importance of Learning & Behavior as measured by total cites, which I have found to be a better surrogate for expert ratings and library use than the impact factor.  The historical perspective has been lost due to a logical flaw in the definition of a bibliographic entity.  In any type of statistical research involving total cites, Learning & Behavior will appear as a screaming outlier that distorts all results.

			Since the fault cannot be corrected through the JCR, it becomes a question of how do you use Web of Science to recover the lost data.  If there is a solution to this problem, it should also be applicable to journals not covered in the JCRs. 

			Stephen J. Bensman

			LSU Libraries

			Louisiana State University

			Baton Rouge, LA   70803

			USA

			notsjb at lsu.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20080223/db689ef9/attachment.html>