Peer Review Scandals

[David Wojick]

This is a common confusion. A typical peer review takes a few hours because 
it just involves reading the paper. The primary objective is to say whether 
the results are important enough to publish in the reviewing journal. 
Replication means repeating the research, which may take days, weeks, 
months or more, depending on the project. Reading and research are very 
different things, hence so are review and replication..

As for your second claim, failure to replicate does not show that the 
original research is unsound. This is another common confusion. There may 
be a lot of procedural subtlety in the original research, which is not 
conveyed in the journal article, which is very brief. As a result the 
replication attempt may fail simply because something was done differently. 
This has been discussed at length at The Scholarly Kitchen. My wife 
recently pointed out an amusing example from baking, which is applied 
chemistry. Forty people each made an angel food cake from the same recipe 
and all the resulting cakes had in common was that each had a hole in the 
middle. Journal articles seldom provide even a recipe, so failure to 
replicate is not telling.

David Wojick
http://insidepublicaccess.com/
http://scholarlykitchen.sspnet.org/author/dwojick/

At 02:31 PM 7/18/2014, you wrote:
>Adminstrative info for SIGMETRICS (for example unsubscribe):
>http://web.utk.edu/~gwhitney/sigmetrics.html
>
>David Wojick claimed:
>|--------------------------------------------------------------------------|
>|"[. . .]                                                                  |
>|                                                                          |
>|Of course peer review has nothing to do with replication."                |
>|--------------------------------------------------------------------------|
>
>It is dubious to claim that being approved by reviewers should not
>involve replication.
>
>|---------------------------------------------------------------------------|
>|"My guess is there are between 5 and 10 million peer reviews a year, but it|
>|only takes 4 or 5 anecdotes, some way off base, to generate broad claims   |
>|of wholesale corruption, that is hurting science. This is what social      |
>|movements feed on, and there is plenty to go around.                       |
>|                                                                           |
>|[. . .]"                                                                   |
>|---------------------------------------------------------------------------|
>
>Lack of replication harms science.
>
>Regards,
>C. Gloster