Pasted below is an article that appeared in today's Wall Street Journal on
the impact factor.
For those of you with strong stomachs and a taste for this type of stuff, I
have completed the first two parts of a rather brutal three-part
statistical analysis of the impact factor, which I am willing to share with
you.
SB
(Embedded image moved to file: pic04144.gif)Search Results for Selected
Items
June 5, 2006 DOW JON...
Loading results
(Embedded image moved to file: pic23196.gif)loading
(E (Embedded image moved to file: pic20222.gif) (Embedded image moved (
mb The Wall Street Journal to file: E
ed pic07129.gif) m
de (Embedded image moved b
d to file: e
im pic02161.jpg) d
ag d
e e
mo d
ve i
d m
to a
fi g
le e
: m
pi o
c0 v
55 e
35 d
.g t
if o
) f
i
l
e
:
p
i
c
2
0
4
5
0
.
g
i
f
)
June 5, 2006
(Embedded
image moved
to file:
pic11173.gif)
DOW JONES REPRINTS
(Embedded image moved to file: pic10466.gif)
(Embedded image moved to file: pic12044.gif)
(Embedded image moved to file: pic17253.gif)This copy is for your personal, non-commercial use only. To order
presentation-ready copies for distribution to your colleagues, clients or customers, use the Order Reprints
tool at the bottom of any article or visit:
www.djreprints.com.
• See a sample reprint in PDF format.
• Order a reprint of this article now.
Science Journals Artfully Try
To Boost Their Rankings
By SHARON BEGLEY
June 5, 2006; Page B1
John B. West has had his share of requests, suggestions and demands from the scientific journals where he
submits his research papers, but this one stopped him cold.
Dr. West, the Distinguished Professor of Medicine and Physiology at the University of California, San Diego,
School of Medicine, is one of the world's leading authorities on respiratory physiology and was a member of
Sir Edmund Hillary's 1960 expedition to the Himalayas. After he submitted a paper on the design of the human
lung to the American Journal of Respiratory and Critical Care Medicine, an editor emailed him that the paper
was basically fine. There was just one thing: Dr. West should cite more studies that had appeared in the
respiratory journal.
If that seems like a surprising request, in the world of scientific publishing it no longer is. Scientists and
editors say scientific journals increasingly are manipulating rankings -- called "impact factors" -- that are
based on how often papers they publish are cited by other researchers.
"I was appalled," says Dr. West of the request. "This was a clear abuse of the system because they were trying
to rig their impact factor."
Just as television shows have Nielsen ratings and colleges have the U.S. News rankings, science journals have
impact factors. Now there is mounting concern that attempts to manipulate impact factors are harming
scientific research.
Conceived 40 years ago, impact factors are essentially a grading system of how important the papers a journal
publishes are. "Importance" is measured by how many other papers cite it, indicating that the discoveries,
methodologies or insights it describes are advancing science.
Impact factors are calculated annually for some 5,900 science journals by Thomson Scientific, part of the
Thomson Corp., of Stamford, Conn. Numbers less than 2 are considered low. Top journals, such as the Journal of
the American Medical Association, score in the double digits. Researchers and editors say manipulating the
score is more common among smaller, newer journals, which struggle for visibility against more established
rivals.
Thomson Scientific is set to release the latest impact factors this month. Thomson has long advocated that
journal editors respect the integrity of the rankings. "The energy that's put into efforts to game the system
would be better spent publishing excellent papers," says Jim Testa, director of editorial development at the
company.
Impact factors matter to publishers' bottom lines because librarians rely on them to make purchasing
decisions. Annual subscriptions to some journals can cost upwards of $10,000.
The result, says Martin Frank, executive director of the American Physiological Society, which publishes 14
journals, is that "we have become whores to the impact factor." He adds that his society doesn't engage in
these practices.
Journals can manipulate impact factors with legitimate editorial decisions. One strategy is to publish many
review articles, says Vicki Cohn, managing editor of Mary Ann Liebert Inc., a closely held New Rochelle, N.Y.,
company that publishes 59 journals. Reviews don't report new results but instead summarize recent findings in
a field. Since it is easier for scientists to cite one review than the dozens of studies that it summarizes,
reviews get a lot of citations, raising a journal's impact score.
"Journal editors know how to increase their impact factor legitimately," says Ms. Cohn. "But there is growing
suspicion that journals are using nefarious means to pump it up."
One questionable tactic is to ask authors to cite papers the journal already has published, as happened to
UCSD's Dr. West, who says that he has great respect for the journal and its editors despite this episode. He
declined the request, and the journal published his paper anyway, in March.
Richard Albert, the deputy editor of the American Journal of Respiratory and Critical Care Medicine, says that
the request goes out to every scientist who submits a paper. "It's boilerplate, a form letter," he says. The
letter has been in use for many years, according to Dr. Albert, who says he has always opposed the inclusion
of the passage but was overruled by the journal's former editor.
Journals also can resort to "best-of" features, such as running annual summaries of their most notable papers.
When Artificial Organs did this in 2005, all 145 citations were to other Artificial Organs papers. Editor Paul
Malchesky says the feature was conceived "as a service to the readership. It was not my intention to affect
our impact factor. In terms of how we run our operation, I don't base that on impact factor."
Self-citation can go too far. In 2005, Thomson Scientific dropped the World Journal of Gastroenterology from
its rankings because 85% of the citations it published were to its own papers and because few other journals
cited it. Editors of the journal, which is based in Beijing, did not answer emails requesting comment.
Journals can limit citations to papers published by competitors, keeping the rivals' impact factors down. An
analysis of citations in the Journal of Telemedicine and Telecare shows very few citations of papers in a
competitor, Telemedicine and e-Health, "while we cited them liberally," says editor Rashid Bashshur, director
of telemedicine at the University of Michigan, Ann Arbor.
Richard Wootton, editor of JTT, says that he believes it's true that his journal cites its competitor less
frequently than Dr. Bashshur's journal cites JTT, "but it doesn't seem to me that there is a sinister
explanation." Dr. Wootton adds that "when we edit a paper...we sometimes ask authors to ensure that the
relevant literature is cited." But "I can state unequivocally that we do not attempt to manipulate the JTT's
impact factor. For a start, I wouldn't know how to."
Scientists and publishers worry that the cult of the impact factor is skewing the direction of research. One
concern, says Mary Ann Liebert, president and chief executive of her publishing company, is that scientists
may jump on research bandwagons, because journals prefer popular, mainstream topics, and eschew less-popular
approaches for fear that only a lesser-tier journal will take their papers. When scientists are discouraged
from pursuing unpopular ideas, finding the correct explanation of a phenomenon or a disease takes longer.
"If you look at journals that have a high impact factor, they tend to be trendy," says immunologist David
Woodland of the nonprofit Trudeau Institute, of Saranac Lake, N.Y., and the incoming editor of Viral
Immunology. He recalls one journal that accepted immunology papers only if they focused on the development of
thymus cells, a once-hot topic. "It's hard to get into them if you're ahead of the curve."
As examples of that, Ms. Liebert cites early research on AIDS, gene therapy and psychopharmacology, all of
which had trouble finding homes in established journals. "How much that relates to impact factor is hard to
know," she says. "But editors and publishers both know that papers related to cutting-edge and perhaps obscure
research are not going to be highly cited."
Another concern is that impact factors, since they measure only how many times other scientists cite a paper,
say nothing about whether journals publish studies that lead to something useful. As a result, there is
pressure to publish studies that appeal to an academic audience oriented toward basic research.
Journals' "questionable" steps to raise their impact factors "affect the public," Ms. Liebert says.
"Ultimately, funding is allocated to scientists and topics perceived to be of the greatest importance. If
impact factor is being manipulated, then scientists and studies that seem important will be funded perhaps at
the expense of those that seem less important."
Write to Sharon Begley at sharon.begley at wsj.com1
URL for this article:
http://online.wsj.com/article/SB114946859930671119.html
Hyperlinks in this Article:
(1) mailto:sharon.begley at wsj.com
Copyright 2006 Dow Jones & Company, Inc. All Rights Reserved
This copy is for your personal, non-commercial use only. Distribution and
use of this material are governed by our Subscriber Agreement and by
copyright law. For non-personal use or to order multiple copies, please
contact Dow Jones Reprints at 1-800-843-0008 or visit www.djreprints.com.
(Embedded image moved to file: pic20649.gif)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic04144.gif
Type: image/gif
Size: 1510 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20060605/cb10a0af/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic23196.gif
Type: image/gif
Size: 565 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20060605/cb10a0af/attachment-0001.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic20222.gif
Type: image/gif
Size: 4923 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20060605/cb10a0af/attachment-0002.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic07129.gif
Type: image/gif
Size: 547 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20060605/cb10a0af/attachment-0003.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic02161.jpg
Type: image/jpeg
Size: 6340 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20060605/cb10a0af/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic05535.gif
Type: image/gif
Size: 45 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20060605/cb10a0af/attachment-0004.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic20450.gif
Type: image/gif
Size: 45 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20060605/cb10a0af/attachment-0005.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic11173.gif
Type: image/gif
Size: 45 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20060605/cb10a0af/attachment-0006.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic10466.gif
Type: image/gif
Size: 45 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20060605/cb10a0af/attachment-0007.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic12044.gif
Type: image/gif
Size: 45 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20060605/cb10a0af/attachment-0008.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic17253.gif
Type: image/gif
Size: 258 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20060605/cb10a0af/attachment-0009.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic20649.gif
Type: image/gif
Size: 43 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20060605/cb10a0af/attachment-0010.gif>