Comments on Prathap, Arun's law of impact factor depreciation

Fri Jan 12 13:53:21 EST 2001

This opening sentence is absolutely untrue. The percentage of citedness for
the highest impact journals is in excess of 90%. The ISI database that
reports on these data is called Journal Performance Indicators.
http://www.isinet.com/isi/products/rsg/products/jpi/index

I suspect that you are being influenced by a report in Science about ten
years ago in which a reporter got hold of some printouts he did not
understand and misquoted the data. There was an excellent rebuttal by David
Pendlebury of ISI which follows.

http://www.the-scientist.com/yr1998/july/pendlebury.html

}
Science, 251:1410-1411, 1991

----------------------------------------------------------------------------
----

David A. Pendlebury

Letters to the Editor

Science, Citation, and Funding

Hamilton's two articles about the percentage of journal literature that
remains uncited within 5 years of publication require comment and further
explanation. The figures reported by Hamilton -- 47.4% uncited for the
sciences, 74.7% for the social sciences, and 98.0% for the arts and
humanities -- are indeed correct. However, as Maxine Singer was quoted as
saying in Hamilton's first article, it is necessary to know what's in the
numbers before interpreting them.

These statistics represent every type of article that appears in journals
indexed by the Institute for Scientific Information (ISI) in its Science
Citation Index, Social Sciences Citation Index, and Arts & Humanities
Citation Index. The journals' ISI indexes contain not only articles,
reviews, and notes, but also meeting abstracts, editorials, obituaries,
letters like this one, and other marginalia, which one might expect to be
largely un-cited. In 1984, the year of the data quoted by Hamilton, about
27% of the items indexed in the Science Citation Index were such marginalia.
The comparable figures for the social sciences and arts and humanities were
48% and 69%, respectively.

If one analyzes the data more narrowly and examines the extent of uncited
articles alone (this information was not yet available when Hamilton wrote
his articles), the figures shrink, some more than others: 22.4% of 1984
science articles remained uncited by the end of 1988, as did 48.0% of social
sciences articles and 93.1% of articles in arts and humanities journals. It
ought to be pointed out that the book represents a considerably more
important vehicle of communication in the social sciences and humanities
than in the sciences. The figures given above reflect only the journal
literature of the social sciences and arts and humanities.

The figures originally quoted by Hamilton seem to have been interpreted by
many readers as some sort of measure of the health of U.S. science. The
numbers, however, reflect a lack of citation of papers by authors the world
over-not only those by U.S. researchers. This point was raised in Hamilton's
first article.

If one restricts the analysis even further and examines the extent of
uncited articles by U.S. authors alone, the numbers are even less
"worrisome." Only 14.7% of 1984 science articles by U.S. authors were left
un-cited by the end of 1988. We estimate the share of uncited 1984 articles
by non-U.S. scientists to be about 28%. (Comparable figures for social
sciences and arts and humanities articles by U.S. authors are not yet
available.)

A certain level of "uncitedness" in the journal literature is probably more
an expression of the process of knowledge creation and dissemination than
any sort of measure of performance. A trend toward more or less
"uncitedness," however, might be meaningful. For the 1980s, we see no such
trend in the scientific literature: the numbers are essentially flat, both
for the United States alone and for the world. In the social sciences,
however, we do detect a decrease in uncited papers -- from 49.7% for 1981
articles to 45.3% for 1985 articles. In the arts and humanities, the figure
of 93% uuncited is fairly steady from 1981 through 1985.

This, we hope, serves to illustrate the great range of statistics one can
derive depending upon what "cut" is made from the ISI databases. For
example, articles published in the highest impact journals like Science are
almost never left uncited.

We will be generating, over the coming months, article-only statistics, both
U.S. and worldwide, for subdisciplines in the sciences, social sciences, and
humanities, corresponding to the overall database statistics referred to by
Hamilton in his second article. We have not yet produced a report on these
statistics, but in light of the great interest in the numbers, we will now
do so.

We hope this information clarifies the record and will end further
misunderstanding or politicalization of these statistics.

David A. Pendlebury
Research Department
Institute for Scientific Information
3501 Market Street
Philadelphia, PA 19104

david.pendlebury at isinet.com

Articles by David Hamilton:
David Hamilton, "Publishing by -- and for? -- the Numbers"
Science, 250:1331-2, 1990

David Hamilton, "Research Papers: Who's Uncited Now?"
Science, 251:25, 1991

When responding, please attach my original message
----------------------------------------------------------------------------
----
Eugene Garfield, Ph.D.  E-mail: garfield at codex.cis.upenn.edu
 Telephone:  (215)243-2205 // Fax: (215)387-1266
 Web site: www.eugenegarfield.org
Past President, American Society for Information Science & Technology
(ASIS&T) -    www.asis.org
Chairman Emeritus, ISI, 3501 Market St , Philadelphia, PA 19104-3389
   www.isinet.com
Pres.,Ed.-in-Chief,  The Scientist, 3600 Market St , Philadelphia, PA
19104-2645
   www.the-scientist.com

-----Original Message-----
From: Eric Ackermann [mailto:eackerma at VT.EDU]
Sent: Friday, January 12, 2001 10:56 AM
To: SIGMETRICS at LISTSERV.UTK.EDU
Subject: [SIGMETRICS] Comments on Prathap, Arun's law of impact factor
depreciation

In relation to who gets cited in general, it might be useful to remember
that, in general, 50 percent of all papers published in journals covered by
the ISI citation databases are never cited at all, regardless of the impact
factor of the journal in which it was published.

 Perhaps before we rush to
the conclusion that sociocultural discrimination is the cause of the
pattern detected by Prathap, we should consider that the result may in fact
due to the use of the impact factor as a predictive surrogate. Such a use
of the impact factor is a risky proposition at best using higher level of
aggregations (such as academic departments or entire countries), must less,
as Aparna Basu points out, "when very small specific samples [such as from
specific individuals] are used."

What do others on the list think about Prathap's idea?

Eric Ackermann
School of Information Sciences
University of Tennessee-Knoxville

>----------------------------------------------------------------------
>
>Date:    Wed, 10 Jan 2001 17:59:28 +0530
>From:    Aparna Basu <aparna at NISTADS.RES.IN>
>Subject: Re: ART: Prathap,             Arun's law of impact factor
depreciation
>
>I would like to make a comment on the paper by Prathap, 'Arun's law..', and
>would like to know what others on the list feel about it.
>
>The possibility of evaluation using scientometric methods provokes people
>into conducting the kind of exercise that Prathap has undertaken without
>taking into account that scientometrics has a kind of statistical validity
>which is eroded when very small specific samples are used. In that case,
>refining techniques when the methodology itself is in question seems to be
>missing the point.
>
>In this connection I would also invite comments on using scientometric
>evaluation for individual scientists/papers.
>
>Aparna Basu
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20010112/d598946e/attachment.html>