Citation analysis of author-choice OA journals

[Stevan Harnad]

Confirmation Bias and the Open Access Advantage:
Some Methodological Suggestions for Davis's Citation Study

Stevan Harnad

Full text: http://openaccess.eprints.org/index.php?/archives/451-guid.html

SUMMARY: Davis (2008) -- http://arxiv.org/pdf/0808.2428v1 -- analyzes
citations from 2004-2007 in 11 biomedical journals. For 1,600 of the 11,000
articles (15%), their authors paid the publisher to make them Open Access
(OA). The outcome, confirming previous studies (on both paid and unpaid OA),
is a significant OA citation Advantage, but a small one (21%, 4% of it
correlated with other article variables such as number of authors,
references and pages). The author infers that the size of the OA advantage
in this biomedical sample has been shrinking annually from 2004-2007, but
the data suggest the opposite. In order to draw valid conclusions from these
data, the following five further analyses are necessary:
    (1) The current analysis is based only on author-choice (paid) OA. Free
OA self-archiving needs to be taken into account too, for the same journals
and years, rather than being counted as non-OA, as in the current analysis.
    (2) The proportion of OA articles per journal per year needs to be
reported and taken into account.
    (3) Estimates of journal and article quality and citability in the form
of the Journal Impact Factor and the relation between the size of the OA
Advantage and journal as well as article "citation-bracket" need to be taken
into account.
    (4) The sample-size for the highest-impact, largest-sample journal
analyzed, PNAS, is restricted and is excluded from some of the analyses. An
analysis of the full PNAS dataset is needed, for the entire 2004-2007
period.
    (5) The analysis of the interaction between OA and time, 2004-2007, is
based on retrospective data from a June 2008 total cumulative citation
count. The analysis needs to be redone taking into account the dates of both
the cited articles and the citing articles, otherwise article-age effects
and any other real-time effects from 2004-2008 are confounded.
The author proposes that an author self-selection bias for providing OA to
higher-quality articles (the Quality Bias, QB) is the primary cause of the
observed OA Advantage, but this study does not test or show anything at all
about the causal role of QB (or of any of the other potential causal
factors, such as Accessibility Advantage, AA, Competitive Advantage, CA,
Download Advantage, DA, Early Advantage, EA, and Quality Advantage, QA). The
author also suggests that paid OA is not worth the cost, per extra citation.
This is probably true, but with OA self-archiving, both the OA and the extra
citations are free.



Confirmation Bias and the Open Access Advantage: Some Methodological
Suggestions for Davis's Citation Study
http://openaccess.eprints.org/index.php?/archives/451-guid.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20080824/b7e25a06/attachment.html>