Mistakes in Scientific Studies

Wed Aug 10 13:25:05 EDT 2011

With markets plunging and sovereign debt ratings being reduced, we can
perhaps take heart that, according to The Wall Street Journal, there is
at least one other thing surging besides the price of gold.

Stephen J Bensman, Ph.D.

LSU Libraries

Lousiana State University

Baton Rouge, LA 70803

USA

 The Wall Street Journal <http://online.wsj.com/home-page>  Health

Mistakes in Scientific Studies Surge .

By GAUTAM NAIK
<http://online.wsj.com/search/term.html?KEYWORDS=GAUTAM+NAIK&bylinesearc
h=true>  

It was the kind of study that made doctors around the world sit up and
take notice: Two popular high-blood-pressure drugs were found to be much
better in combination than either alone.

"There was a 'wow' reaction," recalls Franz Messerli, a New York doctor
who, like many others, changed his prescription habits after the 2003
report.

Unfortunately, it wasn't true. Six and a half years later, the
prestigious medical journal the Lancet retracted the paper, citing
"serious concerns" about the findings.

The damage was done. Doctors by then had given the drug combination to
well over 100,000 patients. Instead of protecting them from kidney
problems, as the study said the drug combo could do, it left them more
vulnerable to potentially life-threatening side effects, later studies
showed. Today, "tens of thousands" of patients are still on the dual
therapy, according to research firm SDI.

View Full Image

When a study is retracted, "it can be hard to make its effects go away,"
says Sheldon Tobe, a kidney-disease specialist at the University of
Toronto.

And that's more important today than ever because retractions of
scientific studies are surging.

Since 2001, while the number of papers published in research journals
has risen 44%, the number retracted has leapt more than 15-fold, data
compiled for The Wall Street Journal by Thomson Reuters reveal.

Just 22 retraction notices appeared in 2001, but 139 in 2006 and 339
last year. Through seven months of this year, there have been 210,
according to Thomson Reuters Web of Science, an index of 11,600
peer-reviewed journals world-wide. 

In a sign of the times, a blog called "Retraction Watch" has popped up
to monitor the flow.

Science is based on trust, and most researchers accept findings
published in peer-reviewed journals. The studies spur others to embark
on related avenues of research, so if one paper is later found to be
tainted, an entire edifice of work comes into doubt. Millions of
dollars' worth of private and government funding may go to waste, and,
in the case of medical science, patients can be put at risk.

At the Mayo Clinic, a decade of cancer research, partly taxpayer-funded,
went down the drain when the prestigious Minnesota institution concluded
that intriguing data about harnessing the immune system to fight cancer
had been fabricated. Seventeen scholarly papers published in nine
research journals had to be retracted. A researcher, who protests his
innocence, was fired.

View Full Image

The Lancet 

Lancet editor Richard Horton said journals don't have adequate systems
to investigate misconduct.

View Full Image

University Hospital Basel 

Swiss doctor Regina Kunz raised questions.

In another major flameout, 18 research journals have said they are
planning to retract a total of 89 published studies by a German
anesthesiologist, many concerning a drug used for maintaining blood
pressure during surgery. Authorities in Britain now are reviewing their
usage guidelines as a result, and a prosecutor in Germany is conducting
a criminal probe, which he says includes the possibility that data were
fabricated. The anesthesiologist couldn't be reached for comment.

Why the backpedaling on more and more scientific research? Some
scientific journals argue that the increase could indicate the journals
have become better at detecting errors. They point to how software has
made it easier to uncover plagiarism.

Others claim to find the cause in a more competitive landscape, both for
the growing numbers of working scientific researchers who want to
publish to advance their careers, and for research journals themselves. 

"The stakes are so high," said the Lancet's editor, Richard Horton. "A
single paper in Lancet and you get your chair and you get your money.
It's your passport to success."

Retractions related to fraud showed a more than sevenfold increase
between 2004 and 2009, exceeding the twofold rise in retractions related
to mere error, according to an analysis published in the Journal of
Medical Ethics. The analyst, Grant Steen, reached that conclusion after
studying 742 medicine and biology papers that were withdrawn from 2000
to 2010. He said 73.5% were retracted simply for error but 26.6% were
retracted for fraud. 

Another researcher, John Budd of the University of Missouri-Columbia,
looked at roughly the same set of journals, though over a longer period,
and also found the prevalence of scientific misconduct to be on the
increase. 

The Lancet, founded in England in 1823, is among the world's most
influential science publications. Just as research published in the
Lancet can have a significant impact, so can the withdrawal of a Lancet
paper, especially after it has been in the public domain and influencing
scientific thought for years. 

In a notorious case, the Lancet last year retracted a study claiming a
link between autism and the measles-mumps-rubella vaccine-12 years after
it was published. In the intervening years, measles cases spiked in the
U.K. as some parents refused to let their children be vaccinated. The
lead investigator on the now-retracted study, Andrew Wakefield, was
stripped of his license to practice medicine in Britain after
authorities concluded that he had engaged in "serious professional
misconduct."

In an emailed response, Dr. Wakefield says he continues to defend his
research, both "the findings and the conclusions."

The Lancet issued just a single retraction notice from 2001 through
2005, but five in the next five years, according to Thomson Reuters.
They included the one finding great promise in the combined use of two
common blood-pressure drugs. The case illustrates both why it can take
so long to pull a flawed study and the fallout that can result.

One of the blood-pressure drugs, called an ACE inhibitor, relaxes blood
vessels by preventing the body from creating a particular hormone. The
other drug, called an ARB, acts on the same hormonal system but in a
different way.

High blood pressure can lead to kidney damage, and each of the two
common drugs reduces a sign of impending kidney disease-the loss of
protein through the urinary system. 

A Japanese researcher, Naoyuki Nakao, wondered if using both drugs at
once would be even better at reducing this sign of kidney trouble. Sure
enough, he reported that the combo was dramatically better than either
drug alone. 

The Lancet published his study, dubbed "Cooperate," in January 2003. It
jumped to the No. 2 spot among the most-cited papers published by the
Lancet that month and created a buzz at medical conferences.

Doctors increasingly prescribed the dual therapy. By 2008, about 140,000
patients in the U.S. were on it, according to SDI, the research firm.

But the report struck some kidney-disease specialists in Switzerland as
too good to be true. The report said patients given the drug combo saw a
76% decrease in protein loss, compared with 42% with one drug by itself
and 44% with the other one alone. To see such a dramatic difference was
unusual, says one of the Swiss doctors, Regina Kunz, who also was
dubious of a particular statistical result in the study. 

"It was too perfect an effect. You wouldn't expect it with such a small
sample size," says Dr. Kunz, director of the Academy of Insurance
Medicine in Basel. "I think the peer reviewers should have caught it."

She and three colleagues wrote to the Lancet in 2006 urging it to look
into the matter. The Lancet's editor, Dr. Horton, says the journal
passed their concerns on to Dr. Nakao in Japan, who responded with some
"recalculations." 

The Lancet then passed all this material on an independent reviewer, who
concluded in December 2006 that "it was impossible to tell whether data
in the [original paper] were the result of fraud or incompetence,"
according to Dr. Horton.

The Lancet tried to get Dr. Nakao to respond, "but he seemed to be
prevaricating," according to Dr. Horton. Dr. Nakao, now at Isekai
Hospital in Osaka, declined to be interviewed.

In May 2008, the Lancet published a "letter of concern" by the Swiss
doctors who had first written to the journal in 2006. The letter
wondered whether certain inconsistencies were "only a case of extremely
sloppy reporting or a hint towards more severe problems with the data."

The Lancet now took the matter to another outside group, a U.K.
nonprofit called the Committee on Publication Ethics. According to Dr.
Horton, it decided the work was incompetent rather than fraudulent.

The Lancet then wrote to a Japanese hospital where Dr. Nakao worked when
he published his study. This hospital said it would do an investigation,
but it would take six months. 

Pressure mounted. Dr. Messerli in New York, a cardiologist, wrote to the
Lancet in mid-2009 arguing that it had a "moral obligation" to withdraw
the paper. The Lancet said it would await the results of the hospital
investigation.

The hospital investigating committee examined medical records at another
Japanese hospital where Dr. Nakao said he and his colleagues had done
the research on 336 patients. But committee members "were not able to
identify even a single patient who matched the contents of the paper,"
said Yutuka Sanada, the president of the hospital that investigated,
called Showa University Fujigaoka Hospital, in Yokohama.

"Dr. Nakao was not able to explain" this, he added, but "insisted that
his paper was not a fabricated one." 

The investigation took until the summer of 2009, about a year after the
Lancet first contacted the university hospital.

"We should have raised the alarm with the university earlier," Dr.
Horton now says.

In October 2009, nearly seven years after the Lancet published the
blood-pressure study and three years after questions were raised about
it, the Lancet printed a retraction notice.

It said the Japanese hospital investigation had concluded that the
researchers hadn't obtained proper patient consent; that they hadn't
obtained approval for the study from the ethics committee of the
hospital where they said the research was done; and that the involvement
of a statistician in the clinical trial couldn't be verified. 

The Lancet also pointed to a finding by the investigating committee that
the trial wasn't "double-blind," a standard precaution in which neither
researchers nor patients know who is getting what drug or placebo.
Instead, the committee found that Dr. Nakao knew who was getting the
drug combination and who wasn't-a situation many investigators consider
tantamount to fraud.

By the time the Lancet retracted the study, concerns were growing about
potential harm to patients who got the combination therapy, except in
certain rare cases where patients benefited. Data from clinical trials
involving high blood pressure involving diabetes, coronary heart disease
and advanced age persuasively showed that any small benefits of the
combination therapy were easily outweighed by the side effects.

"Even patients with uncomplicated essential hypertension were not
entirely able to escape this fashionable trend" in treatment, Dr.
Messerli in New York wrote in the European Heart Journal.

As often happens, the original paper had inspired clinical research by
others to test the dual therapy-studies that enrolled up to 36,000
patients, according to Dr. Steen, the analyst who did a study of
retractions. "If there's a bad trial out there, there will be more
flawed secondary trials, which put more patients at risk," he said. 

Dr. Kunz in Switzerland said the Lancet and its peer reviewers ought to
have been more skeptical about the overly positive results and should
have caught the statistical anomaly she noticed. "Journals all want to
have spectacular results," she said. "Increasingly, they're willing to
publish more risky papers."

The Lancet's Dr. Horton dismisses that notion. He says journals hit by
fraud and error are becoming more conservative about publishing
provocative research. But he also says journals and research
institutions don't have adequate systems in place to properly
investigate misconduct. 

The apparent rise in scientific fraud, said Dr. Horton "is a scar on the
moral body of science."

-Miho Inada contributed to this article.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20110810/13f1b667/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 13013 bytes
Desc: image001.jpg
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20110810/13f1b667/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.gif
Type: image/gif
Size: 625 bytes
Desc: image002.gif
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20110810/13f1b667/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 100683 bytes
Desc: image003.jpg
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20110810/13f1b667/attachment-0001.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.jpg
Type: image/jpeg
Size: 15311 bytes
Desc: image004.jpg
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20110810/13f1b667/attachment-0002.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.jpg
Type: image/jpeg
Size: 78999 bytes
Desc: image005.jpg
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20110810/13f1b667/attachment-0003.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.jpg
Type: image/jpeg
Size: 15711 bytes
Desc: image006.jpg
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20110810/13f1b667/attachment-0004.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image007.jpg
Type: image/jpeg
Size: 63677 bytes
Desc: image007.jpg
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20110810/13f1b667/attachment-0005.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image008.jpg
Type: image/jpeg
Size: 33976 bytes
Desc: image008.jpg
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20110810/13f1b667/attachment-0006.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image009.gif
Type: image/gif
Size: 43 bytes
Desc: image009.gif
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20110810/13f1b667/attachment-0001.gif>