Peer Review Scandals

[Stephen J Bensman]

As I have made abundantly clear, I do not think that open access is a "movement" but a technical and financial necessity.  There is a need for replication, or otherwise NIH would not make data access a requirement..  What would you say to combining the two functions--an article could not be published unless its research can be replicated?  In other words, with the data easily accessible, editors could send the article for replication before publishing.  No publication without replication.  Almost sounds like a revolutionary chant you can shout in the street while rioting.

Steve B.

-----Original Message-----
From: ASIS&T Special Interest Group on Metrics [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of David Wojick
Sent: Friday, July 18, 2014 2:25 PM
To: SIGMETRICS at LISTSERV.UTK.EDU
Subject: Re: [SIGMETRICS] Peer Review Scandals


Sorry Stephen, but I do not believe that a lot of data is being cooked. 
Some certainly, but that is to be expected when millions of people are involved. I read this as a false scare being promulgated by people with an agenda, namely the "open" movement.

David

At 03:07 PM 7/18/2014, you wrote:
>Adminstrative info for SIGMETRICS (for example unsubscribe):
>http://web.utk.edu/~gwhitney/sigmetrics.html
>
>Needless to say, lack of ability to replicate is one of the main 
>reasons for financial losses in the pharmaceutical industry, sending 
>many companies down wrong paths.  The problem is so bad that it is my 
>understanding that the NIH is requiring that data be made openly 
>accessible if the research is the result of an NIH grant so that it can 
>be replicated.  This is one of the main reasons why universities are 
>establishing open access institutional repositories.  LSU is 
>considering the Dataverse Network, on which ASIST just presented a 
>Webinar.  If interested, see the URL below:
>
>   http://en.wikipedia.org/wiki/Dataverse
>
>There is more than cakes being cooked in various kitchens.  There is a 
>lot of data also being cooked.
>
>Stephen J. Bensman
>LSU Libraries
>USA
>
>-----Original Message-----
>From: ASIS&T Special Interest Group on Metrics 
>[mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of David Wojick
>Sent: Friday, July 18, 2014 1:56 PM
>To: SIGMETRICS at LISTSERV.UTK.EDU
>Subject: Re: [SIGMETRICS] Peer Review Scandals
>
>Adminstrative info for SIGMETRICS (for example unsubscribe):
>http://web.utk.edu/~gwhitney/sigmetrics.html
>
>This is a common confusion. A typical peer review takes a few hours 
>because it just involves reading the paper. The primary objective is to 
>say whether the results are important enough to publish in the 
>reviewing journal.
>Replication means repeating the research, which may take days, weeks, 
>months or more, depending on the project. Reading and research are very 
>different things, hence so are review and replication..
>
>As for your second claim, failure to replicate does not show that the 
>original research is unsound. This is another common confusion. There 
>may be a lot of procedural subtlety in the original research, which is 
>not conveyed in the journal article, which is very brief. As a result 
>the replication attempt may fail simply because something was done differently.
>This has been discussed at length at The Scholarly Kitchen. My wife 
>recently pointed out an amusing example from baking, which is applied 
>chemistry. Forty people each made an angel food cake from the same 
>recipe and all the resulting cakes had in common was that each had a 
>hole in the middle. Journal articles seldom provide even a recipe, so 
>failure to replicate is not telling.
>
>David Wojick
>http://insidepublicaccess.com/
>http://scholarlykitchen.sspnet.org/author/dwojick/
>
>At 02:31 PM 7/18/2014, you wrote:
> >Adminstrative info for SIGMETRICS (for example unsubscribe):
> >http://web.utk.edu/~gwhitney/sigmetrics.html
> >
> >David Wojick claimed:
> >|--------------------------------------------------------------------------|
> >|"[. . .]                                                                  |
> >|                                                                          |
> >|Of course peer review has nothing to do with replication."                |
> >|--------------------------------------------------------------------------|
> >
> >It is dubious to claim that being approved by reviewers should not 
> >involve replication.
> >
> >|--------------------------------------------------------------------
> >|----
> ---|
> >|"My guess is there are between 5 and 10 million peer reviews a year,
> but it|
> >|only takes 4 or 5 anecdotes, some way off base, to generate broad
> claims   |
> >|of wholesale corruption, that is hurting science. This is what
> social      |
> >|movements feed on, and there is plenty to go
> around.                       |
> >| 
>     |
> >|[. . 
> .]"                                                                   |
> >|--------------------------------------------------------------------
> >|----
> ---|
> >
> >Lack of replication harms science.
> >
> >Regards,
> >C. Gloster