[Asis-l] REPORT: Little Evidence for Effectiveness of Scientific Reer Review

[Gerry Mckiernan]

              Little Evidence for Effectiveness of Scientific Peer Review

  A Most Revealing (and Perhaps Disturbing) Report on  Effectiveness of Scientific Peer Review

[Thanks, Ben Toth, NHS Information Authority (UK) for informing me of this major report!!!]

   The report focuses on biomedical journals. I'd be interested in Any and All similar studies for *other* disciplines.

/Gerry

Gerry McKiernan 
Effective Librarian 
Iowa State University
Ames IA 50011

gerrymck at iastate.edu 

***********************************************************

   Little Evidence for Effectiveness of Scientific Peer Review
by  Caroline White  / BMJ 2003;326:241 ( 1 February )
[http://bmj.com/cgi/content/full/326/7383/241/a ]

DateLine: London

    Despite its widespread use and costs, little hard evidence exists that peer review improves the quality of published biomedical research, concludes a systematic review from the international Cochrane Collaboration. 

[ http://bmj.com/cgi/reprint/326/7383/241/a.pdf ]

Yet the system, which has been used for at least 200 years, has only recently come under scrutiny, with its assumptions about fairness and objectivity rarely tested, say the review authors. With few exceptions, journal editors-and clinicians-around the world continue to see it as the hallmark of serious scientific endeavour. Published last week, the review is the third in a series from the Cochrane Collaboration Methods Group. 
...

Only the latter escapes a drubbing, with the reviewers concluding that technical editing does improve the readability, accuracy, and overall quality of published research.The Cochrane reviewers based their findings on 21 studies of the peer review process from an original trawl of only
135. 

[snip]

On the basis of the current evidence, "the practice of peer review is based on faith in its effects, rather than on facts," state the authors, who call forlarge, government funded research programmes to test the
effectiveness of the system and investigate possible alternatives.
"As the information revolution gathers pace, an empirically proven method of quality assurance is of paramount importance," they contend.
Professor Tom Jefferson, who led the Cochrane review, suggested
that further research might prove that peer review, or an evolved form of it, worked. At the very least, it needed to be more open and accountable.
But he said that there had never even been any consensus
on its aims and that it would be more appropriate to refer to it as
"competitive review." 

[snip]

REPORT FULL-TEXT AVAILABLE 

Editorial peer-review for improving the quality of reports of biomedical studies 

[ http://www.update-software.com/Cochrane/MR000016.pdf ]

Jefferson TO, Alderson P, Davidoff F, Wager E

This is a reprint of a Cochrane methodology review, prepared and maintained by the Cochrane Collaboration and published in
The Cochrane Library 2003, Issue 1