Fwd: Peer Review Selectivity Determines Quality, Not Open Access vs. Toll Access

[Stevan Harnad]

---------- Forwarded message ----------
From: Stevan Harnad <amsciforum at gmail.com>
Date: Fri, Nov 21, 2008 at 10:55 PM
Subject: Peer Review Selectivity Determines Quality, Not Open Access vs.
Toll Access
To: American Scientist Open Access Forum <
AMERICAN-SCIENTIST-OPEN-ACCESS-FORUM at listserver.sigmaxi.org>


In "Open Access: The question of
quality<http://poynder.blogspot.com/2008/11/open-access-question-of-quality_21.html>,"
Richard Poynder writes:

*"Open Access scientometrics... raise the intriguing possibility that if
research becomes widely available on the Web the quality of papers published
in OA journals may start to overtake, not lag, the quality of papers
published in TA journals... Why? Because if these tools were widely adopted
the most important factor would no longer be which journal you managed to
get your paper published in, but how other researchers assessed the value of
your work — measured by a wide range of different indicators, including for
instance when and how they downloaded it, how they cited it, and the
different ways in which they used it."*

All true, but how does it follow from this that OA journals will overtake TA
journals? As Richard himself states, publishing in an OA journal ("Gold OA")
is not the only way to make one's article OA: One can publish in a TA
journal and self-archive ("Green OA"). OA scientometrics apply to all OA
articles, Green and Gold; so does the OA citation advantage.

Is Richard perhaps conflating TA journals in general with *top*-TA journals
(which may indeed lose some of their metric edge because OA scientometrics
is, as Richard notes, calculated at the article rather than the journal
level)? The only overtaking I see here is OA overtaking TA, not OA journals
overtaking TA journals. (Besides, there are top-OA journals too, as Richard
notes, and bottom-rung TA ones too.)

It should also be pointed out that the top journals differ from the rest of
the journals not just in their impact factor (which, as Richard points out,
is a blunt instrument, being based on the journal average rather than
individual-article citation count) but in their degree of selectivity (peer
review standards). If I am selecting members for a basketball team, and I
only accept the tallest 5%, I am likely to have a taller team than the team
that is less selective on height. Selectivity is correlated with impact
factor, but it is also correlated with quality itself. The Seglen effect
(that about 80% of citations go to the top 20% of articles) is not just a
within-journal effect: it is true across all articles across all journals.
There is no doubt variation within the top journals, but not only are their
articles cited more on average, but they are also better quality on average
(because of their greater selectivity). And the within-journal variation
around the mean is likely to be tighter in those more selective journals
than the less-selective journals.

OA will give richer and more diverse metrics; it will help the cream
(quality) to rise to the top
(citations)<http://openaccess.eprints.org/index.php?/archives/486-guid.html>
unconstrained
by whether the journal happens to be TA or OA. But it is still the rigor and
selectivity of peer
review<http://www.nature.com/nature/webmatters/invisible/invisible.html>
that
does the quality triage in the quality hierarchy among the c. 25,000 peer
reviewed journals, not OA.

(And performance evaluation committees are probably right to place higher
weight on more selective journals -- and on journals with known,
longstanding track-records.)

*Stevan Harnad <http://www.eprints.org/openaccess/>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20081121/b6872dd5/attachment.html>