How to compare research impact of toll- vs. open-access research

Stevan Harnad harnad at ECS.SOTON.AC.UK
Thu Jun 12 09:23:19 EDT 2003

On Thu, 12 Jun 2003, Sally Morris wrote:

> It is important to point out that Steve Lawrence's article, despite its
> title, does not (and indeed, could not) distinguish between articles which
> are freely available and those for which a fee has been paid.  All it
> measures is the increase in rate of citation if the articles are accessible
> online to the citer. We recently checked this with the author himself.

Sally is quite right to point out that the Lawrence citation-impact study
did not specifically compare online-for-fee vs. online-for-free. It
compared online-or-on-paper-for-fee vs. online-for-free. (Unless I am
mistaken, all the computer science conference papers in his "for-free"
set were indeed for-free, i.e., open-access.)

The correct, controlled studies that need to be conducted are the two
I described in an earlier posting on this thread (and again summarize
below) -- although I am confident (and I am sure everyone who hasn't a
vested interest in the opposite outcome and either has experience
with free online availability or has thought about it would agree)
that the outcome will be pretty much the same as the Lawrence study,
if not stronger. For although access is not a *sufficient* condition
for reading/usage/citation, it is certainly a *necessary* one. And
currently it is a brute fact that for papers accessible only for-fee,
access to the majority of their potential users on the planet is denied
-- because the toll-access fees for the planet's 20,000 peer-reviewed
journals are not universally affordable, and cannot be, for anywhere
near all potential users. That is the Gutenberg fact that is poised to
be remedied by self-archiving in our PostGutenberg era.

The correct, controlled studies are:

        (1) Comparing citation counts for self-archived and non-self-archived
        articles equated for year, volume, and issue in the same toll-access
        journals, all hybrid journals (i.e. having paper edition and
        toll-access online editions), across a diverse sample of disciplines,
        using paracite to seek full-text free-access versions.

        (2) Comparing citation counts for self-archived and non-self-archived
        articles equated for year, volume, and issue in the same toll-access
        journals, all hybrid journals (i.e. having paper edition and
        toll-access online editions) in the Physics ArXiv, using citebase.

Our research team at Southampton will be conducting these studies, but
as the sooner the data are in the better, we invite anyone else to scoop
us! There are plenty of urgent things to do for open access apart from
convincing the sceptics that the Lawrence results do indeed confirm what
all self-archivers already know: That freely accessible research will be
read, used and cited far more than research that is not freely accessible.

> Of course, this means that there is a feedback loop from the 'Big Deals'
> which the largest publishers are able to provide to libraries and consortia.
> Availability means more citations, more citations means increased
> attractiveness both to authors and to cash-strapped libraries.

If toll-access publishers use the Lawrence results to promote online
access to institutions that so far only seek on-paper access, that's
fine.* In its own limited way, that too is increasing impact.

        *Indeed, the benefits of fee-based access (over non-access) are
        precisely what the Kurtz study shows in the very special case of
        For the astrophysics community is perhaps unique in being

                (1) a relatively circumscribed community worldwide, mostly located
                only at the well-funded universities (perhaps because of the
                resource-intensiveness of the research?),

                (2) having a relatively small, "closed" literature, involving
                a small specific set of journals in which all the relevant
                papers and citations appear. As a consequence

                (3) virtually all astrophysicists have institutional site-licensed
                access (i.e., for-fee) to the entire astrophysics literature,
                as confirmed by Tim Brody's finding that (unlike all other fields
                of physics, and unlike all other disciplines)

                (4) astrophyicists self-archive only their unrefereed preprints,
                to get them out early (like other phsyicists); they do not bother
                to self-archive their postprints, knowing that they will all be
                available to everyone for-fee (as Peter Boyce of ASA has been
                telling me for years).

        I am sure there will be postings from astrophysicists who report that
        they do *not* have access to all of the astro journal literature
        for-fee, but astro is still probably the field that is *closest*
        to the kind of "universal for-fee access" that some publishers hope
        to provide, but that will not prove even faintly affordable for most
        fields other than astro

Stevan Harnad

> ----- Original Message -----
> From: "Stevan Harnad" <harnad at>
> Sent: Monday, June 09, 2003 1:19 PM
> Subject: Re: THES article on research access Friday June 6 2003
> > On Fri, 6 Jun 2003, [identity removed] wrote:
> >
> > > You mention in your recent post and THES article that
> > >>  "For the citation counts of papers whose full texts are already
> > >>  freely accessible on the web are more than 300 per cent higher than
> > >>  those that are not."
> > > Do you have a citation for that that you can point me at?
> >
> > Lawrence, S. (2001a) Online or Invisible? Nature 411 (6837): 521.
> >
> >
> > Lawrence, S. (2001b) Free online availability substantially increases a
> > paper's impact. Nature Web Debates.
> >
> >
> > See also:
> > Kurtz, Michael J.; Eichhorn, Guenther; Accomazzi, Alberto; Grant,
> > Carolyn S.; Demleitner, Markus; Murray, Stephen S.; Martimbeau,
> > Nathalie; Elwell, Barbara. (submitted) The NASA Astrophysics Data
> > System: Sociology, Bibliometrics, and Impact.
> >
> >
> > and the work of Andrew Odlyzko:
> >
> >
> > and Tim Brody's remarkable impact correlation generator, which
> > can predict later citation impact from earlier usage (download)
> > impact using variable time-windows and ranges for the Physics
> > ArXiv (you need the latest java to be able to use it) at:
> >
> >
> > Finally, see:
> >
> >
> > Stevan Harnad

More information about the SIGMETRICS mailing list