Crowd-Sourced Peer Review: Substitute or Supplement?

Stevan Harnad amsciforum at GMAIL.COM
Thu Aug 21 15:18:31 EDT 2014

Harnad, S. (2014) *Crowd-Sourced Peer Review: Substitute or supplement for
the current outdated system?
Impact Blog* 8/21


If, as rumoured
google builds a platform for depositing unrefereed research papers for
“peer-reviewing” viacrowd-sourcing
<>, can this create a *substitute
classical peer-review or will it merely *supplement
peer review with crowd-sourcing?

... no one knows whether crowd-sourced peer-review, even if it could work,
would be scaleable or sustainable.

The key questions are hence:

*1. Would all (most? many?) authors be willing to post their unrefereed
papers publicly (and in place of submitting them to journals!)?2. Would all
(most? many?) of the posted papers attract referees? competent experts?3.
Who/what decides whether the refereeing is competent, and whether the
author has adequately complied? (Relying on a Wikipedia-style cadre of
2nd-order crowd-sourcers
<> who
gain authority recursively in proportion to how much 1st-order
crowd-sourcing they have done — rather than on the basis of expertise —
 sounds like a way to generate Wikipedia quality, but not peer-reviewed
quality…)4. If any of this actually happens on any scale, will it be
sustainable?5. Would this make the landscape (unrefereed preprints, referee
comments, revised postprints) as navigable and useful as classical peer
review, or not?*

My own prediction (based on nearly a quarter century of umpiring
<> both classical
peer review and open peer commentary) is that crowdsourcing will provide an
excellent supplement to classical peer review but not a substitute for it.
Radical implementations will simply end up re-inventing classical peer
review, but on a much faster and more efficient PostGutenberg platform. We
will not realize this, however, until all of the peer-reviewed literature
has first been made open access. And for that it is not sufficient for
Google merely to provide a platform for authors to put their unrefereed
papers, because most authors don’t even put their refereed papers in their
institutional repositories until it is mandated by their institutions and
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the SIGMETRICS mailing list