Perils of Press-Release Journalism: NSF and Chronicle of Higher Education

Fri Mar 6 16:24:45 EST 2009

Dear David and Christina: I don't know whether it is relevant to your
discussion, but the following reference from the 1991 NEJM sounds like
it should be.

IMPORTANCE OF THE LAY PRESS IN THE TRANSMISSION OF MEDICAL KNOWLEDGE TO
THE SCIENTIFIC COMMUNITY

  <javascript:void>  <javascript:void>   Holdings a UIUC Catalog b
ILLINET Catalog c UIC Catalog  

Format this record for printingE-mail this recordAdd this record to your
Marked List.Save to EndNote Web librarySave to EndNote, Reference
Manager, or similar bibliographic management toolmore options
<http://apps.isiknowledge.com/full_record.do?product=WOS&search_mode=Gen
eralSearch&qid=1&SID=1A2NKmLJc29M9 at G5LK2&page=1&doc=2#output_options#out
put_options> 

 <javascript:void> 

Author(s): PHILLIPS DP
<http://apps.isiknowledge.com/DaisyOneClickSearch.do?product=WOS&search_
mode=DaisyOneClickSearch&db_id=&SID=1A2NKmLJc29M9 at G5LK2&name=PHILLIPS%20
DP&ut=A1991GK53800030&pos=1> , KANTER EJ
<http://apps.isiknowledge.com/DaisyOneClickSearch.do?product=WOS&search_
mode=DaisyOneClickSearch&db_id=&SID=1A2NKmLJc29M9 at G5LK2&name=KANTER%20EJ
&ut=A1991GK53800030&pos=2> , BEDNARCZYK B
<http://apps.isiknowledge.com/DaisyOneClickSearch.do?product=WOS&search_
mode=DaisyOneClickSearch&db_id=&SID=1A2NKmLJc29M9 at G5LK2&name=BEDNARCZYK%
20B&ut=A1991GK53800030&pos=3> , TASTAD PL
<http://apps.isiknowledge.com/DaisyOneClickSearch.do?product=WOS&search_
mode=DaisyOneClickSearch&db_id=&SID=1A2NKmLJc29M9 at G5LK2&name=TASTAD%20PL
&ut=A1991GK53800030&pos=4>  

Source: NEW ENGLAND JOURNAL OF MEDICINE    Volume: 325    Issue: 16
Pages: 1180-1183    Published: OCT 17 1991   

Times Cited: 105
<http://apps.isiknowledge.com/CitingArticles.do?product=WOS&search_mode=
CitingArticles&SID=1A2NKmLJc29M9 at G5LK2&db_id=WOS&parentQid=1&parentDoc=2
&recid=77483183>      References: 15
<http://apps.isiknowledge.com/CitedRefList.do?product=WOS&search_mode=Ci
tedRefList&SID=1A2NKmLJc29M9 at G5LK2&db_id=WOS&parentQid=1&parentDoc=2&rec
id=77483183>       Citation Map <javascript:void();>        

Abstract: Background. Efficient, undistorted communication of the
results of medical research is important to physicians, the scientific
community, and the public. Information that first appears in the
scientific literature is frequently retransmitted in the popular press.
Does popular coverage of medical research in turn amplify the effects of
that research on the scientific community? 

Methods. To test the hypothesis that researchers are more likely to cite
papers that have been publicized in the popular press, we compared the
number of references in the Science Citation Index to articles in the
New England Journal of Medicine that were covered by The New York Times
with the number of references to similar articles that were not covered
by the Times. We also performed the comparison during a three-month
period when the Times was on strike but continued to prepare an "edition
of record" that was not distributed; doing so enabled us to address the
possibility that coverage in the Times was simply a marker of the most
important articles, which would therefore be cited more frequently, even
without coverage in the popular press.

Results. Articles in the Journal that were covered by the Times received
a disproportionate number of scientific citations in each of the 10
years after the Journal articles appeared. The effect was strongest in
the first year after publication, when Journal articles publicized by
the Times received 72.8 percent more scientific citations than control
articles. This effect was not present for articles published during the
strike; articles covered by the Times during this period were no more
likely to be cited than those not covered. Conclusions. Coverage of
medical research in the popular press amplifies the transmission of
medical information from the scientific literature to the research
community.

Document Type: Note 

Language: English 

KeyWords Plus: CITATION ANALYSIS 

Reprint Address: PHILLIPS, DP (reprint author), UNIV CALIF SAN DIEGO,
DEPT SOCIOL, LA JOLLA, CA 92093 USA 

Publisher: MASS MEDICAL SOC, 10 SHATTUCK, BOSTON, MA 02115 

Subject Category: Medicine, General & Internal 

IDS Number: GK538 

ISSN: 

The PDF for this article was sent to me by Barbara Gastel. If you don't
have access to the full text let me know. I presume you can access the
list of 100 plus citing papers. If not let me know. Best wishes. Gene
Garfield

________________________________

From: ASIS&T Special Interest Group on Metrics
[mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of David Wojick
Sent: Friday, February 27, 2009 6:28 AM
To: SIGMETRICS at LISTSERV.UTK.EDU
Subject: Re: [SIGMETRICS] Perils of Press-Release Journalism: NSF and
Chronicle of Higher Education

Thanks Christina, I will follow up on these leads.

However, what I am looking for is something more global, analogous to
citation analysis but in two steps. First, on the supply side, which
research is being reported and spreading, throughout the news system?
This is akin to publication. Second, who in the scientific community is
reading about which research? This is analogous to citation. This
writing-reading transaction system is much harder to track than
citations, but it might be just as important, if not more so. In fact
how to track it is the biggest research challenge. The write-read news
system is bigger, faster, more turbulent and much less tangible,
especially the reading part. 

Plus there is the significant difference that in the news system many
publications are about topics rather than specific results. The negative
impact of biofuel production for example. Yet this too is the spread of
scientific ideas among scientists.

My conjecture is that the news system is far more important than
scholarly publication when it comes to generating first awareness of
research results and new ideas within the scientific community.
Scholarly publication probably plays an intermediate role, that is it is
something one reads after becoming interested as a result of news, and
before one contacts the author. If so then the role of scholarly
publication may be misunderstood, but this is presently just a guess.

My best regards,

David

David Wojick, Ph.D.

Senoir consultant for innovation
http://www.osti.gov

Feb 26, 2009 09:52:56 AM, SIGMETRICS at listserv.utk.edu wrote:

	Certainly there has been research on how press releases figure
into the diffusion of scientific information.  For example, there have
been bibliometric studies that included press release coverage in a
regression equations regarding citedness.  There have also been STS and
public understanding of science (I refuse to use the unpleasant
abbreviation) papers about this in general as well as the particular
case surrounding cold fusion.  There are also studies in scholarly
communication that discuss the Ingelfinger rule and the like.

	Actually, an editorial in today's Nature is about this issue
with blogs, pre-prints, and press embargos:  
http://www.nature.com/nature/journal/v457/n7233/full/4571058a.html

	Based on conversations with PLOS and Nature editors, it seems
likely that they will both add more information to article pages
regarding web commentary on blogs and other social computing
technologies.  Their goal is to provide a more 360 view of
article/author impact than journal article citations do alone.

	As far as how to study, I think there have even been some
relevant questions on the GSS as well as smaller surveys,
qualitative/ethnographic studies, critical/historical studies, etc.  I'm
not saying it's a done deal, but it certainly has been addressed.

	Christina K. Pikas, MLS 
	R.E. Gibson Library & Information Center
	The Johns Hopkins University Applied Physics Laboratory 
	Voice  240.228.4812 (Washington), 443.778.4812 (Baltimore) 
	Fax 443.778.5353 

	From: ASIS&T Special Interest Group on Metrics
[mailto:SIGMETRICS at listserv.utk.edu] On Behalf Of David Wojick
	Sent: Thursday, February 26, 2009 6:53 AM
	To: SIGMETRICS at listserv.utk.edu
	Subject: Re: [SIGMETRICS] Perils of Press-Release Journalism:
NSF and Chronicle of Higher Education

	Adminstrative info for SIGMETRICS (for example unsubscribe):
http://web.utk.edu/~gwhitney/sigmetrics.html 

	Steve raises an important scientometric issue, quite apart from
the issue of what Evans did or found. This is the role of press
releases, and the news articles they engender, in the diffusion of
scientific infromation. The question is how to observe and measure such
diffusion? The number of information transactions, or A reading about
B's results, via news is several orders of magnitude greater than via
journal articles. I don't think we even know how many orders of
magnitude. Yet this is in some respects the most important mode of
scientific knowledge diffusion.

	How this news based diffusion affects the dynamics of science is
likewise unknown. Is anyone studying this formally? I am doing so
informally. The web is providing some new approaches, such a blog
tracking and the occurrence of embedded URLs. The spread of
characteristic language is also a likely avenue. This is much more like
true diffusion analysis than is citation and co-author network analysis,
in that it goes beyond tracking large, discrete transactions to looking
at a vague spreading cloud of information.

	Steve also raises the issue of the spread of misinformation via
diffusion of news. This has been studied in the context of general
social thought, especially rumors. It is certainly significant in the
realm of science and public policy, where the Evans case lies. I study
this phenomenon in the climate change debate and in energy policy.
Whether it is important in science per se I do not know. It is not even
clear how one would approach it, but it seems like an important research
topic. Perhaps it should be approached as the diffusion and dynamics of
controversy or disagreement.

	Cheers,

	David

	David Wojick, Ph.D.

	http://www.osti.gov

	Feb 25, 2009 06:40:34 PM, SIGMETRICS at listserv.utk.edu wrote:

		Adminstrative info for SIGMETRICS (for example
unsubscribe): http://web.utk.edu/~gwhitney/sigmetrics.html In response
to my critique
<http://openaccess.eprints.org/index.php?/archives/535-guid.html>  of
his Chronicle of Higher Education posting
<http://chronicle.com/news/article/6026/fee-based-journals-get-better-re
sults-study-in-fee-based-journal-reports?commented=1#c033840>  on Evans
and Reimer's (2009) Science article
<http://www.sciencemag.org/cgi/content/full/323/5917/1025>  (which I
likewise critiqued
<http://openaccess.eprints.org/index.php?/archives/533-guid.html> ,
though much more mildly), I got an email from Paul Basken asking me to
explain what, if anything he had got wrong, since his posting was based
entirely on a press release from NSF
<http://www.nsf.gov/news/news_summ.jsp?cntn_id=114225> . Sure enough,
the silly spin originated from the NSF Press release (though the buck
stops with E & R's vague and somewhat tendentious description and
interpretation of some of their findings). Here is the NSF Press
Release, enhanced with my comments, for your delectation and verdict: 

________________________________

			If you offer something of value to people for
free while someone else charges a hefty sum of money for the same type
of product, one would logically assume that most people would choose the
free option. According to new research in today's edition of the journal
Science, if the product in question is access to scholarly papers and
research, that logic might just be wrong. These findings provide new
insight into the nature of scholarly discourse and the future of the
open source publication movement[sic, emphasis added].

		(1) If you offer something valuable for free, people
will choose the free option unless they've already paid for the paid
option (especially if they needed -- and could afford -- it earlier).

		(2) Free access after an embargo of a year is not the
same "something" as immediate free access. Its "value" for a potential
user is lower. (That's one of the reasons institutions keep paying for
subscription/license access to journals.)

		(3) Hence it is not in the least surprising that
immediate print-on-paper access + (paid) online access (IP + IO)
generates more citations than immediate (paid) print-on-paper access
(IP) alone.

		(4) Nor is it surprising that immediate (paid)
print-on-paper access + online access + delayed free online access (IP
+IO + DF) generates more citations than just immediate (paid)
print-on-paper + online access (IO + IP) alone -- even if the free
access is provided a year later than the paid access.

		(5) Why on earth would anyone conclude that the fact
that the increase in citations from IP to IP + IO is 12% and the
increase in citations from IP + IO to IP + IO + DF is a further 8%
implies anything whatsoever about people's preference for paid access
over free access? Especially when the free access is not even immediate
(IF) but delayed (DF)? 

		Most research is published in scientific journals and
reviews, and subscriptions to these outlets have traditionally cost
money--in some cases a great deal of money. Publishers must cover the
costs of producing peer-reviewed publications and in most cases also try
to turn a profit. To access these publications, other scholars and
researchers must either be able to afford subscriptions or work at
institutions that can provide access.

		In recent years, as the Internet has helped lower the
cost of publishing, more and more scientists have begun publishing their
research in open source outlets online. Since these publications are
free to anyone with an Internet connection, the belief has been that
more interested readers will find them and potentially cite them.
Earlier studies had postulated that being in an open source format could
more than double the number of times a journal article is used by other
researchers.

		What on earth is an "open source outlet"? ("Open source"
is a software matter.) Let's assume what's meant is "open access"; but
then is this referring to (i) publishing in an open access journal, to
(ii) publishing in a subscription journal but also self-archiving the
published article to make it open access, or to (iii) self-archiving an
unpublished paper?

		What (many) previous studies
<http://opcit.eprints.org/oacitation-biblio.html>  had measured (not
"postulated") was that (ii) publishing in a subscription journal (IP +
IO) and also self-archiving the published article to make it Open Access
(IP + IO + OA) could more than double the citations, compared to IP + IO
alone. 

		To test this theory, James A. Evans, an assistant
professor of sociology at the University of Chicago, and Jacob Reimer, a
student of neurobiology also at the University of Chicago, analyzed
millions of articles available online, including those from open source
publications and those that required payment to access.

		No, they did nothing of the sort; and no "theory" was
tested.

		Evans & Reimer (E & R) only analyzed articles from
subscription access journals before and after they became accessible
online (to paid subscribers only) (i.e., IP vs IP + IO) as well as
before and after the online version was made accessible free for all
(after a paid-access-only embargo of up to a year or more: i.e., IP +IO
vs IP + IO + DF). Their methodology was based on comparing citation
counts for articles within the same journals before and after being made
free online at various intervals. 

		The results were surprising. On average, when a given
publication was made available online after being in print for a year,
being published in an open source format increased the use of that
article by about 8 percent. When articles are made available online in a
commercial format a year after publication, however, usage increases by
about 12 percent.

		In other words, the citation count increase from just
(paid) IP to (paid) IP + IO was 12% and the citation count increase from
just (paid) IP + IO to (paid) IP + IO + DF was 8%. Not in the least
surprising: Making paid-access articles accessible online increases
their citations, and making them free online (even if only after a delay
of a year) increases them still more.

		What is surprising is the rather absurd spin that this
press release appears to be trying to put on this unsurprising finding. 

		"Across the scientific community," Evans said in an
interview, "it turns out that open access does have a positive impact on
the attention that's given to the journal articles, but it's a small
impact."

		We already knew that OA increased citations, as the many
prior published studies
<http://opcit.eprints.org/oacitation-biblio.html>  have shown.  Most of
those studies, however, were based on immediate OA (i.e., IF), not
embargoed OA. What E & R do show, interestingly, is that even delaying
OA for a year still increases citations, though not nearly as much as
immediate OA (IF). 

		Yet Evans and Reimer's research also points to one very
positive impact of the open source movement that is sometimes overlooked
in the debate about scholarly publications. Researchers in the
developing world, where research funding and libraries are not as robust
as they are in wealthier countries, were far more likely to read and
cite open source articles.

		A large portion of the citation increase from (delayed)
OA turns out to come from Developing Countries (refuting Frandsen
<http://www.hprints.org/hprints-00328270/en/> 's recent report to the
contrary). (A similar comparison, within the US, of citations from the
Have-Not Universities (with the smaller journal subscription budgets)
compared to the Harvards may well reveal the same effect closer to home,
though probably at a smaller scale.) 

		The University of Chicago team concludes that outside
the developed world, the open source movement "widens the global circle
of those who can participate in science and benefit from it."

		And it will be interesting to test for the same effect
comparing the Harvards and the Have-Nots in the US -- but a more
realistic estimate might come from looking at immediate OA (IF) rather
than just embargoed OA (DF). 

		So while some scientists and scholars may chose to pay
for scientific publications even when free publications are available,
their colleagues in other parts of the world may find that going with
open source works is the only choice they have.

		It would be interesting to hear the authors of this NSF
press release -- or E & R, for that matter -- explain how this
paradoxical "preference" for paid access over free access was tested
during the access embargo period...

		Stevan Harnad <http://www.eprints.org/openaccess/> 
		American Scientist Open Access Forum
<http://amsci-forum.amsci.org/archives/American-Scientist-Open-Access-Fo
rum.html> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20090306/667fe09b/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 43 bytes
Desc: image001.gif
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20090306/667fe09b/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.gif
Type: image/gif
Size: 1151 bytes
Desc: image002.gif
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20090306/667fe09b/attachment-0001.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.gif
Type: image/gif
Size: 836 bytes
Desc: image003.gif
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20090306/667fe09b/attachment-0002.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.gif
Type: image/gif
Size: 853 bytes
Desc: image004.gif
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20090306/667fe09b/attachment-0003.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.gif
Type: image/gif
Size: 752 bytes
Desc: image005.gif
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20090306/667fe09b/attachment-0004.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.gif
Type: image/gif
Size: 208 bytes
Desc: image006.gif
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20090306/667fe09b/attachment-0005.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image007.gif
Type: image/gif
Size: 257 bytes
Desc: image007.gif
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20090306/667fe09b/attachment-0006.gif>