Crowd-Sourced Peer Review: Substitute or Supplement?

[Stevan Harnad]

Harnad, S. (2014) *Crowd-Sourced Peer Review: Substitute or supplement for
the current outdated system?
<http://blogs.lse.ac.uk/impactofsocialsciences/2014/08/21/crowd-sourced-peer-review-substitute-or-supplement/>
**LSE
Impact Blog* 8/21

EXCERPT:

If, as rumoured
<http://www.wired.co.uk/news/archive/2014-08/13/google-science-would-we-need-it>,
google builds a platform for depositing unrefereed research papers for
“peer-reviewing” viacrowd-sourcing
<https://en.wikipedia.org/wiki/Crowdsourcing>, can this create a *substitute
<https://www.google.ca/?gws_rd=ssl#q=harnad+%22peer+review%22+substitute+supplement>*
for
classical peer-review or will it merely *supplement
<https://www.google.com/webhp?tbm=blg&gws_rd=ssl#q=site:openaccess.eprints.org+%22peer+review%22+substitute+supplement&tbm=blg>*
classical
peer review with crowd-sourcing?

... no one knows whether crowd-sourced peer-review, even if it could work,
would be scaleable or sustainable.

The key questions are hence:










*1. Would all (most? many?) authors be willing to post their unrefereed
papers publicly (and in place of submitting them to journals!)?2. Would all
(most? many?) of the posted papers attract referees? competent experts?3.
Who/what decides whether the refereeing is competent, and whether the
author has adequately complied? (Relying on a Wikipedia-style cadre of
2nd-order crowd-sourcers
<http://www0.cs.ucl.ac.uk/staff/L.Capra/publications/ubicrowd11.pdf> who
gain authority recursively in proportion to how much 1st-order
crowd-sourcing they have done — rather than on the basis of expertise —
 sounds like a way to generate Wikipedia quality, but not peer-reviewed
quality…)4. If any of this actually happens on any scale, will it be
sustainable?5. Would this make the landscape (unrefereed preprints, referee
comments, revised postprints) as navigable and useful as classical peer
review, or not?*

My own prediction (based on nearly a quarter century of umpiring
<http://users.ecs.soton.ac.uk/harnad/Temp/bbs.valedict.html> both classical
peer review and open peer commentary) is that crowdsourcing will provide an
excellent supplement to classical peer review but not a substitute for it.
Radical implementations will simply end up re-inventing classical peer
review, but on a much faster and more efficient PostGutenberg platform. We
will not realize this, however, until all of the peer-reviewed literature
has first been made open access. And for that it is not sufficient for
Google merely to provide a platform for authors to put their unrefereed
papers, because most authors don’t even put their refereed papers in their
institutional repositories until it is mandated by their institutions and
funders.

http://openaccess.eprints.org/index.php?/archives/1121-Crowd-Sourced-Peer-Review-Substitute-or-Supplement.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20140821/d0b621a2/attachment.html>