Perils of Press-Release Journalism: NSF and Chronicle of Higher Education

[Stevan Harnad]

*In response to my
critique<http://openaccess.eprints.org/index.php?/archives/535-guid.html>
of
his Chronicle of Higher Education
posting<http://chronicle.com/news/article/6026/fee-based-journals-get-better-results-study-in-fee-based-journal-reports?commented=1#c033840>
on
Evans and Reimer's (2009) Science
article<http://www.sciencemag.org/cgi/content/full/323/5917/1025>
(which
I likewise critiqued<http://openaccess.eprints.org/index.php?/archives/533-guid.html>,
though much more mildly), I got an email from Paul Basken asking me to
explain what, if anything he had got wrong, since his posting was based
entirely on a press release from
NSF<http://www.nsf.gov/news/news_summ.jsp?cntn_id=114225>.
Sure enough, the silly spin originated from the NSF Press release (though
the buck stops with E & R's vague and somewhat tendentious description and
interpretation of some of their findings). Here is the NSF Press Release,
enhanced with my comments, for your delectation and verdict:
------------------------------
**

If you offer something of value to people for free while someone else
charges a hefty sum of money for the same type of product, one would
logically assume that most people would choose the free option. According to
new research in today's edition of the journal Science, if the product in
question is access to scholarly papers and research, that logic might just
be wrong. These findings provide new insight into the nature of scholarly
discourse and the future of the open source publication movement[sic,
emphasis added].

*(1) If you offer something valuable for free, people will choose the free
option *unless they've already paid for the paid option (especially if they
needed -- and could afford -- it earlier).*

(2) Free access after an embargo of a year is not the same "something" as
immediate free access. Its "value" for a potential user is lower. (That's
one of the reasons institutions keep paying for subscription/license access
to journals.)

(3) Hence it is not in the least surprising that immediate print-on-paper
access + (paid) online access (IP + IO) generates more citations than
immediate (paid) print-on-paper access (IP) alone.

(4) Nor is it surprising that immediate (paid) print-on-paper access +
online access + delayed free online access (IP +IO + DF) generates more
citations than just immediate (paid) print-on-paper + online access (IO +
IP) alone -- even if the free access is provided a year later than the paid
access.

(5) Why on earth would anyone conclude that the fact that the increase in
citations from IP to IP + IO is 12% and the increase in citations from IP +
IO to IP + IO + DF is a further 8% implies anything whatsoever about
people's preference for paid access over free access? Especially when the
free access is not even immediate (IF) but delayed (DF)?*

Most research is published in scientific journals and reviews, and
subscriptions to these outlets have traditionally cost money--in some cases
a great deal of money. Publishers must cover the costs of producing
peer-reviewed publications and in most cases also try to turn a profit. To
access these publications, other scholars and researchers must either be
able to afford subscriptions or work at institutions that can provide
access.

In recent years, as the Internet has helped lower the cost of publishing,
more and more scientists have begun publishing their research in open source
outlets online. Since these publications are free to anyone with an Internet
connection, the belief has been that more interested readers will find them
and potentially cite them. Earlier studies had postulated that being in an
open source format could more than double the number of times a journal
article is used by other researchers.

*What on earth is an "open source outlet"? ("Open source" is a software
matter.) Let's assume what's meant is "open access"; but then is this
referring to (i) publishing in an open access journal, to (ii) publishing in
a subscription journal but also self-archiving the published article to make
it open access, or to (iii) self-archiving an unpublished paper?

What (many) previous
studies<http://opcit.eprints.org/oacitation-biblio.html> had
measured (not "postulated") was that (ii) publishing in a subscription
journal (IP + IO) and also self-archiving the published article to make it
Open Access (IP + IO + OA) could more than double the citations, compared to
IP + IO alone.*

To test this theory, James A. Evans, an assistant professor of sociology at
the University of Chicago, and Jacob Reimer, a student of neurobiology also
at the University of Chicago, analyzed millions of articles available
online, including those from open source publications and those that
required payment to access.

*No, they did nothing of the sort; and no "theory" was tested.

Evans & Reimer (E & R) only analyzed articles from subscription access
journals before and after they became accessible online (to paid subscribers
only) (i.e., IP vs IP + IO) as well as before and after the online version
was made accessible free for all (after a paid-access-only embargo of up to
a year or more: i.e., IP +IO vs IP + IO + DF). Their methodology was based
on comparing citation counts for articles within the same journals before
and after being made free online at various intervals.*

The results were surprising. On average, when a given publication was made
available online after being in print for a year, being published in an open
source format increased the use of that article by about 8 percent. When
articles are made available online in a commercial format a year after
publication, however, usage increases by about 12 percent.

*In other words, the citation count increase from just (paid) IP to (paid)
IP + IO was 12% and the citation count increase from just (paid) IP + IO to
(paid) IP + IO + DF was 8%. Not in the least surprising: Making paid-access
articles accessible online increases their citations, and making them free
online (even if only after a delay of a year) increases them still more.

What *is* surprising is the rather absurd spin that this press release
appears to be trying to put on this unsurprising finding.*

"Across the scientific community," Evans said in an interview, "it turns out
that open access does have a positive impact on the attention that's given
to the journal articles, but it's a small impact."

*We already knew that OA increased citations, as the many prior published
studies <http://opcit.eprints.org/oacitation-biblio.html> have shown.  Most
of those studies, however, were based on *immediate* OA (i.e., IF), not
embargoed OA. What E & R do show, interestingly, is that* even delaying OA
for a year still increases citations*, though not nearly as much as
immediate OA (IF).*

Yet Evans and Reimer's research also points to one very positive impact of
the open source movement that is sometimes overlooked in the debate about
scholarly publications. Researchers in the developing world, where research
funding and libraries are not as robust as they are in wealthier countries,
were far more likely to read and cite open source articles.

*A large portion of the citation increase from (delayed) OA turns out to
come from Developing Countries (refuting
Frandsen<http://www.hprints.org/hprints-00328270/en/>'s
recent report to the contrary). (A similar comparison, within the US, of
citations from the Have-Not Universities (with the smaller journal
subscription budgets) compared to the Harvards may well reveal the same
effect closer to home, though probably at a smaller scale.)*

The University of Chicago team concludes that outside the developed world,
the open source movement "widens the global circle of those who can
participate in science and benefit from it."

*And it will be interesting to test for the same effect comparing the
Harvards and the Have-Nots in the US -- but a more realistic estimate might
come from looking at immediate OA (IF) rather than just embargoed OA (DF).*

So while some scientists and scholars may chose to pay for scientific
publications even when free publications are available, their colleagues in
other parts of the world may find that going with open source works is the
only choice they have.

*It would be interesting to hear the authors of this NSF press release -- or
E & R, for that matter -- explain how this paradoxical "preference" for paid
access over free access was tested during the access embargo period...

*Stevan Harnad <http://www.eprints.org/openaccess/>*
American Scientist Open Access
Forum<http://amsci-forum.amsci.org/archives/American-Scientist-Open-Access-Forum.html>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20090225/dd8837a9/attachment.html>