Peer Review Scandals

[David Wojick]

Sorry Stephen, but I do not believe that a lot of data is being cooked. 
Some certainly, but that is to be expected when millions of people are 
involved. I read this as a false scare being promulgated by people with an 
agenda, namely the "open" movement.

David

At 03:07 PM 7/18/2014, you wrote:
>Adminstrative info for SIGMETRICS (for example unsubscribe):
>http://web.utk.edu/~gwhitney/sigmetrics.html
>
>Needless to say, lack of ability to replicate is one of the main reasons 
>for financial losses in the pharmaceutical industry, sending many 
>companies down wrong paths.  The problem is so bad that it is my 
>understanding that the NIH is requiring that data be made openly 
>accessible if the research is the result of an NIH grant so that it can be 
>replicated.  This is one of the main reasons why universities are 
>establishing open access institutional repositories.  LSU is considering 
>the Dataverse Network, on which ASIST just presented a Webinar.  If 
>interested, see the URL below:
>
>   http://en.wikipedia.org/wiki/Dataverse
>
>There is more than cakes being cooked in various kitchens.  There is a lot 
>of data also being cooked.
>
>Stephen J. Bensman
>LSU Libraries
>USA
>
>-----Original Message-----
>From: ASIS&T Special Interest Group on Metrics 
>[mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of David Wojick
>Sent: Friday, July 18, 2014 1:56 PM
>To: SIGMETRICS at LISTSERV.UTK.EDU
>Subject: Re: [SIGMETRICS] Peer Review Scandals
>
>Adminstrative info for SIGMETRICS (for example unsubscribe):
>http://web.utk.edu/~gwhitney/sigmetrics.html
>
>This is a common confusion. A typical peer review takes a few hours 
>because it just involves reading the paper. The primary objective is to 
>say whether the results are important enough to publish in the reviewing 
>journal.
>Replication means repeating the research, which may take days, weeks, 
>months or more, depending on the project. Reading and research are very 
>different things, hence so are review and replication..
>
>As for your second claim, failure to replicate does not show that the 
>original research is unsound. This is another common confusion. There may 
>be a lot of procedural subtlety in the original research, which is not 
>conveyed in the journal article, which is very brief. As a result the 
>replication attempt may fail simply because something was done differently.
>This has been discussed at length at The Scholarly Kitchen. My wife 
>recently pointed out an amusing example from baking, which is applied 
>chemistry. Forty people each made an angel food cake from the same recipe 
>and all the resulting cakes had in common was that each had a hole in the 
>middle. Journal articles seldom provide even a recipe, so failure to 
>replicate is not telling.
>
>David Wojick
>http://insidepublicaccess.com/
>http://scholarlykitchen.sspnet.org/author/dwojick/
>
>At 02:31 PM 7/18/2014, you wrote:
> >Adminstrative info for SIGMETRICS (for example unsubscribe):
> >http://web.utk.edu/~gwhitney/sigmetrics.html
> >
> >David Wojick claimed:
> >|--------------------------------------------------------------------------|
> >|"[. . .]                                                                  |
> >|                                                                          |
> >|Of course peer review has nothing to do with replication."                |
> >|--------------------------------------------------------------------------|
> >
> >It is dubious to claim that being approved by reviewers should not
> >involve replication.
> >
> >|------------------------------------------------------------------------ 
> ---|
> >|"My guess is there are between 5 and 10 million peer reviews a year, 
> but it|
> >|only takes 4 or 5 anecdotes, some way off base, to generate broad 
> claims   |
> >|of wholesale corruption, that is hurting science. This is what 
> social      |
> >|movements feed on, and there is plenty to go 
> around.                       |
> >| 
>     |
> >|[. . 
> .]"                                                                   |
> >|------------------------------------------------------------------------ 
> ---|
> >
> >Lack of replication harms science.
> >
> >Regards,
> >C. Gloster