Crowd-Sourced Peer Review: Substitute or Supplement?

[Stevan Harnad]

> Only Scientific Members who are affiliated with an accredited university and have at least
> 5 publications assigned to their ORCID account may officially review an article at ScienceOpen. 

So  being affiliated with an accredited university and having at least 5 publications makes 
anyone a specialist qualified to review anyone's research?

The criterion for peership might have to be a trifle more exacting than that even before we
raise once again the niggling question of answerability...

Harnad, S. (1990) Scholarly Skywriting and the Prepublication Continuum of Scientific Inquiry 
Psychological Science 1: 342 - 343 (reprinted in Current Contents 45: 9-13, November 11 1991). 
http://cogprints.org/1581/

Stevan Harnad

On Aug 22, 2014, at 4:41 AM, Jennifer Smith <jesmith at sgul.ac.uk> wrote:

> Hi All
>  
> As list members probably know, Frontiers In … have a slightly different method of reviewing than traditional peer review model.
>  
> I noticed recently another alternative model for reviewing, at ScienceOpen (based on having an academic ID, in this case, ORCiD):
>  
> “Comments and Reviews require registration via ORCID
>  
> Everybody can read, download and share your article. Commenting, rating and reviewing,
> however, requires previous registration via ORCID. Only Scientific Members who are
> affiliated with an accredited university and have at least 5 publications assigned to their
> ORCID account may officially review an article at ScienceOpen. Commenting requires
> at least 1 publication. Please refer to our User Categories for a detailed description. In
> any case, please consult our Peer Review Guidelines and Guidelines for Commenting
> before writing a review or commenting on papers.”
>  
> http://about.scienceopen.com/how-does-it-work/#more-9
>  
> Interesting times to see what develops and is taken up.
>  
> Kind regards,
>  
> Jennifer
>  
> Jennifer Smith
> Research Publications Librarian
> Library
> Information Services
> St George's University of London
> E: jesmith at sgul.ac.uk
> T: +44 (0)20 8725 5393
>  
> From: Repositories discussion list [mailto:JISC-REPOSITORIES at JISCMAIL.AC.UK] On Behalf Of Stevan Harnad
> Sent: 21 August 2014 20:19
> To: JISC-REPOSITORIES at JISCMAIL.AC.UK
> Subject: Crowd-Sourced Peer Review: Substitute or Supplement?
>  
> Harnad, S. (2014) Crowd-Sourced Peer Review: Substitute or supplement for the current outdated system? LSE Impact Blog 8/21 
>  
> EXCERPT:
>  
> If, as rumoured, google builds a platform for depositing unrefereed research papers for “peer-reviewing” viacrowd-sourcing, can this create a substitute for classical peer-review or will it merely supplement classical peer review with crowd-sourcing?
>  
> ... no one knows whether crowd-sourced peer-review, even if it could work, would be scaleable or sustainable.
> 
> The key questions are hence:
> 1. Would all (most? many?) authors be willing to post their unrefereed papers publicly (and in place of submitting them to journals!)?
> 
> 2. Would all (most? many?) of the posted papers attract referees? competent experts?
> 
> 3. Who/what decides whether the refereeing is competent, and whether the author has adequately complied? (Relying on a Wikipedia-style cadre of 2nd-order crowd-sourcers who gain authority recursively in proportion to how much 1st-order crowd-sourcing they have done — rather than on the basis of expertise —  sounds like a way to generate Wikipedia quality, but not peer-reviewed quality…)
> 
> 
> 4. If any of this actually happens on any scale, will it be sustainable?
> 
> 5. Would this make the landscape (unrefereed preprints, referee comments, revised postprints) as navigable and useful as classical peer review, or not?
> My own prediction (based on nearly a quarter century of umpiring both classical peer review and open peer commentary) is that crowdsourcing will provide an excellent supplement to classical peer review but not a substitute for it. Radical implementations will simply end up re-inventing classical peer review, but on a much faster and more efficient PostGutenberg platform. We will not realize this, however, until all of the peer-reviewed literature has first been made open access. And for that it is not sufficient for Google merely to provide a platform for authors to put their unrefereed papers, because most authors don’t even put their refereed papers in their institutional repositories until it is mandated by their institutions and funders.
>  
> http://openaccess.eprints.org/index.php?/archives/1121-Crowd-Sourced-Peer-Review-Substitute-or-Supplement.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20140822/4e544a3b/attachment.html>