Peer Review Scandals

[Dowman P Varn]

David,

Your first point is spot-on, but I must take issue with the second.
Repeatability of experimental work is a cornerstone of science. While some
observations are inherently unrepeatable, for example the discovery of a
rare fossil, others are not. For example, the cold fusion debacle of the
late 80's and early 90's revolved around the inability of numerous other
groups to repeat the experiment. While unrepeatability may be due to
factors other than a mistake in the original research, we'd be foolish not
to consider failed efforts to repeat an experiment as evidence of error in
the original. Perhaps not decisive evidence, but compelling evidence
nonetheless.

Regards,

Dowman


On 18 July 2014 11:55, David Wojick <dwojick at craigellachie.us> wrote:

> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
> This is a common confusion. A typical peer review takes a few hours
> because it just involves reading the paper. The primary objective is to say
> whether the results are important enough to publish in the reviewing
> journal. Replication means repeating the research, which may take days,
> weeks, months or more, depending on the project. Reading and research are
> very different things, hence so are review and replication..
>
> As for your second claim, failure to replicate does not show that the
> original research is unsound. This is another common confusion. There may
> be a lot of procedural subtlety in the original research, which is not
> conveyed in the journal article, which is very brief. As a result the
> replication attempt may fail simply because something was done differently.
> This has been discussed at length at The Scholarly Kitchen. My wife
> recently pointed out an amusing example from baking, which is applied
> chemistry. Forty people each made an angel food cake from the same recipe
> and all the resulting cakes had in common was that each had a hole in the
> middle. Journal articles seldom provide even a recipe, so failure to
> replicate is not telling.
>
> David Wojick
> http://insidepublicaccess.com/
> http://scholarlykitchen.sspnet.org/author/dwojick/
>
>
> At 02:31 PM 7/18/2014, you wrote:
>
>> Adminstrative info for SIGMETRICS (for example unsubscribe):
>> http://web.utk.edu/~gwhitney/sigmetrics.html
>>
>> David Wojick claimed:
>> |-----------------------------------------------------------
>> ---------------|
>> |"[. . .]
>>  |
>> |
>>  |
>> |Of course peer review has nothing to do with replication."
>>  |
>> |-----------------------------------------------------------
>> ---------------|
>>
>> It is dubious to claim that being approved by reviewers should not
>> involve replication.
>>
>> |-----------------------------------------------------------
>> ----------------|
>> |"My guess is there are between 5 and 10 million peer reviews a year, but
>> it|
>> |only takes 4 or 5 anecdotes, some way off base, to generate broad claims
>>   |
>> |of wholesale corruption, that is hurting science. This is what social
>>    |
>> |movements feed on, and there is plenty to go around.
>>   |
>> |
>>   |
>> |[. . .]"
>>   |
>> |-----------------------------------------------------------
>> ----------------|
>>
>> Lack of replication harms science.
>>
>> Regards,
>> C. Gloster
>>
>


-- 
______________________________
Dowman P Varn, PhD
Complexity Sciences Center & Department of Physics

University of California, One Shields Ave
Davis, CA 95616

Cell: 646.228.7256
Email: dpv at complexmatter.org
Web site: wissenplatz.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20140718/9d3d26ee/attachment.html>