Peer Review Scandals

[David Wojick]

Dowman, I do not consider a single failure of replication to be compelling 
evidence against the original research, far from it. Especially not if the 
procedure is complex, subtle or delicate. I was involved in the cold fusion 
case and there were hundreds of attempts at replication by different 
research groups. When they all failed the judgement became clear, but only 
then.

My best regards,

David

At 03:08 PM 7/18/2014, you wrote:
>Adminstrative info for SIGMETRICS (for example unsubscribe): 
>http://web.utk.edu/~gwhitney/sigmetrics.html
>David,
>
>Your first point is spot-on, but I must take issue with the second. 
>Repeatability of experimental work is a cornerstone of science. While some 
>observations are inherently unrepeatable, for example the discovery of a 
>rare fossil, others are not. For example, the cold fusion debacle of the 
>late 80's and early 90's revolved around the inability of numerous other 
>groups to repeat the experiment. While unrepeatability may be due to 
>factors other than a mistake in the original research, we'd be foolish not 
>to consider failed efforts to repeat an experiment as evidence of error in 
>the original. Perhaps not decisive evidence, but compelling evidence 
>nonetheless.
>
>Regards,
>
>Dowman
>
>
>On 18 July 2014 11:55, David Wojick 
><<mailto:dwojick at craigellachie.us>dwojick at craigellachie.us> wrote:
>Adminstrative info for SIGMETRICS (for example unsubscribe):
><http://web.utk.edu/~gwhitney/sigmetrics.html>http://web.utk.edu/~gwhitney/sigmetrics.html
>
>This is a common confusion. A typical peer review takes a few hours 
>because it just involves reading the paper. The primary objective is to 
>say whether the results are important enough to publish in the reviewing 
>journal. Replication means repeating the research, which may take days, 
>weeks, months or more, depending on the project. Reading and research are 
>very different things, hence so are review and replication..
>
>As for your second claim, failure to replicate does not show that the 
>original research is unsound. This is another common confusion. There may 
>be a lot of procedural subtlety in the original research, which is not 
>conveyed in the journal article, which is very brief. As a result the 
>replication attempt may fail simply because something was done 
>differently. This has been discussed at length at The Scholarly Kitchen. 
>My wife recently pointed out an amusing example from baking, which is 
>applied chemistry. Forty people each made an angel food cake from the same 
>recipe and all the resulting cakes had in common was that each had a hole 
>in the middle. Journal articles seldom provide even a recipe, so failure 
>to replicate is not telling.
>
>David Wojick
><http://insidepublicaccess.com/>http://insidepublicaccess.com/
>http://scholarlykitchen.sspnet.org/author/dwojick/
>
>
>At 02:31 PM 7/18/2014, you wrote:
>Adminstrative info for SIGMETRICS (for example unsubscribe):
><http://web.utk.edu/~gwhitney/sigmetrics.html>http://web.utk.edu/~gwhitney/sigmetrics.html
>
>David Wojick claimed:
>|--------------------------------------------------------------------------|
>|"[. . .] 
>Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â 
>Â  Â  Â  Â  Â  Â  Â  Â |
>| 
>Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â 
>Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â |
>|Of course peer review has nothing to do with replication." 
>Â  Â  Â  Â  Â  Â  Â  Â |
>|--------------------------------------------------------------------------|
>
>It is dubious to claim that being approved by reviewers should not
>involve replication.
>
>|---------------------------------------------------------------------------|
>|"My guess is there are between 5 and 10 million peer reviews a year, but it|
>|only takes 4 or 5 anecdotes, some way off base, to generate broad claims   |
>|of wholesale corruption, that is hurting science. This is what social 
>Â  Â  Â |
>|movements feed on, and there is plenty to go around. 
>Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  |
>| 
>Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â 
>Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  |
>|[. . .]" 
>Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â 
>Â  Â  Â  Â  Â  Â  Â  Â  |
>|---------------------------------------------------------------------------|
>
>Lack of replication harms science.
>
>Regards,
>C. Gloster
>
>
>
>
>--
>______________________________
>Dowman P Varn, PhD
>Complexity Sciences Center & Department of Physics 
>Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â  Â
>University of California, One Shields Ave
>Davis, CA 95616
>
>Cell:Â 646.228.7256 Â
>Email: <mailto:dpv at complexmatter.org>dpv at complexmatter.org
>Web site: <http://wissenplatz.org>wissenplatz.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20140718/09d57d30/attachment.html>