Journal Prestige
Stephen J Bensman
notsjb at LSU.EDU
Fri Dec 30 14:07:41 EST 2005
An interesting take on scientific journal prestige. -- SB
(E (Embedded image moved to file: pic15455.gif)The Wall (
mb Street Journal E
ed m
de b
d e
im d
ag d
e e
mo d
ve i
d m
to a
fi g
le e
: m
pi o
c2 v
88 e
11 d
.g t
if o
) f
i
l
e
:
p
i
c
2
2
3
7
1
.
g
i
f
)
December 30, 2005
(Embedded
image moved
to file:
pic22323.gif)
COMMENTARY
Mere Magazines
By THOMAS P. STOSSEL
December 30, 2005; Page A16
Recently I was working in a Zambian orphanage when a young woman with worsening shortness of breath and chest
pain asked me for help. Armed only with a stethoscope, I could do nothing other than diagnose a probable
lethal tuberculous infection of the heart. Without devices and drugs developed by companies, doctors are not
very useful.
It was therefore discouraging to return to my Boston-based medical center and witness leading medical journals
sanctimoniously demonizing not only the technologies developed by drug companies but also the companies
themselves. The Journal of the American Medical Association has declared industry-sponsored research
categorically untrustworthy, and, to publish it, demands that an academic researcher be an author and take
responsibility for its integrity, and also that an independent academic statistician analyze its data. This
and other journals rail obsessively against "financial conflicts of interest" of academic researchers working
with companies and conduct inquisitions to identify every possible financial motive that might corrupt
researchers' objectivity.
The ongoing Merck situation is a case in point. The New England Journal of Medicine wants the company to
correct a five-year-old paper that, they allege, inappropriately excluded three late-breaking adverse events
associated with the painkiller Vioxx. The company has correctly responded that published research projects
always have defined beginnings and endings, and that it reported all adverse events to the FDA. With the drug
off the market and Merck mired in litigation, what problem this correction would solve is unclear.
Nevertheless, a Dec. 11 New York Times editorial excoriated Merck for "manipulating a journal article" and
informed doctors "that they will need to take the findings of industry-backed studies with skeptical caution."
The message in all this is clear: Medical academics are saints -- devoted selflessly to patient care -- and
corporate people are sinners, morally blinded by greed. But having worked in academic medicine for over 35
years and consulted for companies, this Manichean duality is inconsistent with my experience and a woeful
distortion of reality. In a Sept. 8 article in the New England Journal of Medicine, I reported that no
systematic evidence exists that corporate sponsorship of academic research contributes to misconduct, bias,
public mistrust or poor research quality.
On the other hand, many academic colleagues working in my field of basic biological research (I study how your
body cells crawl around, which has no obvious commercial value) would run over their grandmothers to claim
priority for a discovery, impose their pet theory on the field, obtain a research grant, win an award or
garner a promotion. It's the same in other scientific fields, and no wonder, because for relatively modest
remuneration we compete for scarce resources and labor in obscurity to achieve small advances few understand
or appreciate. We exercise our ambitions by publishing research papers in high-profile journals.
* * *
The research journal revolutionized scientific communication in the 17th century. But until the scientific
enterprise grew larger than the first journals could accommodate, no peer review restricted publication. Once
restrictions arose, human competitiveness established a journal prestige pecking order that grew in importance
as research became more prevalent and complex. The more obscure one's research, the greater the premium on
publishing it in a prestigious journal, where those who administer limited rewards might see it, and where the
news media are more likely to hype it.
But unbeknownst to the media, the journals at the top got there because of herd behavior by reqearchers, not
because they are better than lower-tier journals at vetting research quality. Here's why: Researchers submit
their best work to the top journals, which can therefore afford to maintain their prestige by rejecting, not
publishing, many high quality papers. That's brand creation -- not science. Most of their editorial effort
goes into deciding which submitted papers are sufficiently newsworthy. Anonymous peer review by jealous
competitors has its merits, but it has a tendency to select for fashionable if relatively unoriginal and
inoffensive papers. Top medical journals compete for papers describing large clinical trials reporting small
effects of treatments for diseases affecting many people, although these reports often do not substantively
advance scientific knowledge, and many subsequently are invalidated.
And no description of medical research in a medical journal comes close to the detail level or intense
scrutiny imposed by the FDA on companies' documentation of drug or device development before approval. Space
constraints for readability and cost-savings preclude journals from publishing detailed information on the
order of what companies file with the FDA, and unpaid journal peer reviewers, not to mention practicing
doctors, would never read it anyway. The recent Korean cloning fiasco, in which the leading science journals
published blatantly fraudulent papers, wasn't the first such incident to afflict prestige journals, and it
could never happen under conditions of FDA review. Indeed, doctors should take all studies published in
"prominent medical journals" with "skeptical caution."
The lower stringency of journals compared to the FDA is a good thing, because academic biomedical research
would come to a screeching halt if subjected to anything even approximating FDA examination. Scientific
knowledge advances reasonably efficiently, and new technologies emerge, despite the looseness of journals. And
researchers' craving for prestige goads them to greater efforts.
If reporters understood that journals are magazines, not Holy Scripture, we might not be witnessing ever more
onerous regulations inhibiting interactions between academic and industry science. Prestigious biomedical
journals are good for our health -- provided they stick to their core business of facilitating imperfect
communication between researchers. Leave drug and device monitoring to the FDA -- and theology to theologians.
Mr. Stossel is American Cancer Society Professor at Harvard Medical School and co-director of the division of
hematology at Brigham and Women's Hospital.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic15455.gif
Type: image/gif
Size: 4923 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051230/e23bb89f/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic28811.gif
Type: image/gif
Size: 45 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051230/e23bb89f/attachment-0001.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic22371.gif
Type: image/gif
Size: 45 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051230/e23bb89f/attachment-0002.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic22323.gif
Type: image/gif
Size: 45 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051230/e23bb89f/attachment-0003.gif>
More information about the SIGMETRICS
mailing list