Perils of Press-Release Journalism: NSF and Chronicle of Higher Education

David E. Wojick dwojick at HUGHES.NET
Sat Mar 7 11:08:13 EST 2009

Thank you Loet, but I am not talking about newspapers. That was 
merely the example that Gene supplied. I am talking about all 
significant forms of scientific communication, in the sense of all 
the forms whereby scientists learn about new science.

For example, I get most of my initial information about 
scientometrics papers from this listserv. In many cases I only read 
the abstract. I get a great deal of knowledge about other topics that 
I am involved in from the magazine part of Science, which I read 
carefully each week. For particular topics I get a lot from Google, 
Google Scholar, and the various OSTI products like and In one case (climate change) I rely mostly on 

I am often led to journal articles but I read no specific journals 
regularly, so journals are never my original awareness source. The 
journal article typically shows up somewhere between steps 2 and 10, 
although I may not get to an article in many cases.

Taken all together this is the diffusion system of science. Journals 
play an essential role but from a transaction point of view it is 
quite small. Yet this small subsystem, the journals, gets 99% of our 
attention. We have very little idea what is going on in the rest of 
the diffusion system, probably because we have very little data on 
it. But it is there and it is important.

(Sorry but I did not use or mention arrows so I do not know what you 
are referring to by the second of my arrows.)

All my best,

Dear David,

The non-ISI-sources for citations are available in the JCR. This 
covers the second of your arrows. The first can be traced because 
major newspapers have excellent search engines on their archives. 
Alternatively, one can use Lexis Nexis. I do this regularly with 
students from communication studies. We then distinguish among three 
types of communication: scientific communication, public 
communication (newspapers), and political communication 
(parliamentary proceedings). The causality is very different in 
different domains.

For example, in the case of obesitas scientific communications feed 
into the other domains. In the case of issues like violence in 
computer games, public concern is leading both scientific interest 
and political debate. However, I would not overestimate the feedback 
arrow in specific cases. I don't remember having read anything about 
citation analysis in a newspaper which I did not already know from 
the scientific communication.

Best wishes,


PS. Below is the citation impact environment of the Wall Street 
Journal in 2006 using JCR data:

Loet Leydesdorff
Amsterdam School of Communications Research (ASCoR),
Kloveniersburgwal 48, 1012 CX Amsterdam.
Tel.: +31-20- 525 6598; fax: +31-20- 525 3681
<mailto:loet at>loet at ; 

From: ASIS&T Special Interest Group on Metrics 
[mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of David Wojick
Sent: Saturday, March 07, 2009 1:07 PM
Subject: Re: [SIGMETRICS] Perils of Press-Release Journalism: NSF and 
Chronicle of Higher Education

Thank you Gene, this is indeed relevant and a good example of looking 
at the role of news in the diffusion of science within science.

However, I do question their use of the term "disproportionate" in 
their conclusion that "Articles in the Journal that were covered by 
the Times received a disproportionate number of scientific citations 
in each of the 10 years after the Journal articles appeared." The 
term "significantly larger" is better because disproportionate sound 
like a criticism.

In any case my point is that channels of communication other than 
reading journal articles play a major role in the diffusion of ideas 
within science. Not just news outlets, but channels like this 
listserv are very important. My conjecture is that when it comes to 
first awareness of a new result reading the journal is probably a 
relative small part of the system of information transactions. That 
is, most scientists learn about new results in other ways. The 
structure of the magazine/journal Science is very revealing in this 
context. The number of news-like articles about research results is 
much higher than the number of actual articles, perhaps 20 times 

What we need is to look at this non-journal diffusion as 
systematically as we look at the systems of citation and 
co-authorships. First awareness and citation or co-authorship are 
opposite extremes in the system of transactions. The approach will 
have to be different because the tracking of individual transactions 
is impossible. A reading a story about B for example. This is more 
like true diffusion analysis, in groundwater for example, than like 
network analysis. I think the flow of new language, which always 
characterizes the flow of new knowledge, will be an important 
approach. It can also be in real time.

So far as I know this kind of diffusion research is not being done, 
so the dynamics of new scientific thinking is not being seen, in an 
important respect. That was my original point.

My best regards,


David Wojick, Ph.D.

Mar 6, 2009 04:31:17 PM, SIGMETRICS at LISTSERV.UTK.EDU wrote:

Dear David and Christina: I don't know whether it is relevant to your 
discussion, but the following reference from the 1991 NEJM sounds 
like it should be.










<http://javascript:void> Holdingsa UIUC Catalogb ILLINET Catalogc UIC Catalog









Source: NEW ENGLAND JOURNAL OF MEDICINE    Volume: 325    Issue: 
16    Pages:        1180-1183    Published:        OCT 17 1991 


Times Cited: 
<>105     References: 
<>15     <http://javascript:void();> 
Citation Map      


Abstract:        Background. Efficient, undistorted communication of 
the results of        medical research is important to physicians, 
the scientific community,        and the public. Information that 
first appears in the scientific        literature is frequently 
retransmitted in the popular press. Does        popular coverage of 
medical research in turn amplify the effects of        that research 
on the scientific community?


Methods. To test the hypothesis that researchers are more likely 
 to cite papers that have been publicized in the popular press, we 
 compared the number of references in the Science Citation Index to 
 articles in the New England Journal of Medicine that were covered by 
 The New York Times with the number of references to similar articles 
 that were not covered by the Times. We also performed the comparison 
 during a three-month period when the Times was on strike but 
continued        to prepare an "edition of record" that was not 
distributed;        doing so enabled us to address the possibility 
that coverage in the        Times was simply a marker of the most 
important articles, which would        therefore be cited more 
frequently, even without coverage in the        popular press.


Results. Articles in the Journal that were covered by the Times 
 received a disproportionate number of scientific citations in each 
of        the 10 years after the Journal articles appeared. The 
effect was        strongest in the first year after publication, when 
Journal articles        publicized by the Times received 72.8 percent 
more scientific citations        than control articles. This effect 
was not present for articles        published during the strike; 
articles covered by the Times during this        period were no more 
likely to be cited than those not covered.        Conclusions. 
Coverage of medical research in the popular press        amplifies 
the transmission of medical information from the scientific 
 literature to the research community.


Document Type: Note       


Language:        English


KeyWords Plus:        CITATION ANALYSIS


Reprint Address:        PHILLIPS, DP (reprint author), UNIV CALIF SAN 


Publisher: MASS        MEDICAL SOC, 10 SHATTUCK, BOSTON,         MA 


Subject Category:        Medicine, General & Internal


IDS Number:        GK538






The PDF for this article was sent to me by Barbara Gastel. If you 
don't have access to the full text let me know. I presume you can 
access the list of 100 plus citing papers. If not let me know. Best 
wishes. Gene Garfield

From: ASIS&T Special Interest Group on Metrics 
[mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of David Wojick
Sent: Friday, February 27, 2009 6:28 AM
Subject: Re: [SIGMETRICS] Perils of Press-Release Journalism: NSF and 
Chronicle of Higher Education

Thanks Christina, I will follow up on these leads.

However, what I am looking for is something more global, analogous to 
citation analysis but in two steps. First, on the supply side, which 
research is being reported and spreading, throughout the news system? 
This is akin to publication. Second, who in the scientific community 
is reading about which research? This is analogous to citation. This 
writing-reading transaction system is much harder to track than 
citations, but it might be just as important, if not more so. In fact 
how to track it is the biggest research challenge. The write-read 
news system is bigger, faster, more turbulent and much less tangible, 
especially the reading part.

Plus there is the significant difference that in the news system many 
publications are about topics rather than specific results. The 
negative impact of biofuel production for example. Yet this too is 
the spread of scientific ideas among scientists.

My conjecture is that the news system is far more important than 
scholarly publication when it comes to generating first awareness of 
research results and new ideas within the scientific community. 
Scholarly publication probably plays an intermediate role, that is it 
is something one reads after becoming interested as a result of news, 
and before one contacts the author. If so then the role of scholarly 
publication may be misunderstood, but this is presently just a guess.

My best regards,


David Wojick, Ph.D.

Senoir consultant for innovation

Feb 26, 2009 09:52:56 AM, SIGMETRICS at wrote:

Certainly there has been research on how press releases figure into 
the diffusion of scientific information.  For example, there have 
been bibliometric studies that included press release coverage in a 
regression equations regarding citedness.  There have also been STS 
and public understanding of science (I refuse to use the unpleasant 
abbreviation) papers about this in general as well as the particular 
case surrounding cold fusion.  There are also studies in scholarly 
communication that discuss the Ingelfinger rule and the like.

Actually, an editorial in today's Nature is about this issue with 
blogs, pre-prints, and press embargos: 

Based on conversations with PLOS and Nature editors, it seems likely 
that they will both add more information to article pages regarding 
web commentary on blogs and other social computing technologies. 
Their goal is to provide a more 360 view of article/author impact 
than journal article citations do alone.

As far as how to study, I think there have even been some relevant 
questions on the GSS as well as smaller surveys, 
qualitative/ethnographic studies, critical/historical studies, etc. 
I'm not saying it's a done deal, but it certainly has been addressed.

Christina K. Pikas, MLS
R.E. Gibson Library & Information Center
The Johns Hopkins University Applied Physics Laboratory
Voice  240.228.4812 (Washington), 443.778.4812 (Baltimore)
Fax 443.778.5353

From: ASIS&T Special Interest Group on Metrics 
[mailto:SIGMETRICS at] On Behalf Of David Wojick
Sent: Thursday, February 26, 2009 6:53 AM
Subject: Re: [SIGMETRICS] Perils of Press-Release Journalism: NSF and 
Chronicle of Higher Education

Steve raises an important scientometric issue, quite apart from the 
issue of what Evans did or found. This is the role of press releases, 
and the news articles they engender, in the diffusion of scientific 
infromation. The question is how to observe and measure such 
diffusion? The number of information transactions, or A reading about 
B's results, via news is several orders of magnitude greater than via 
journal articles. I don't think we even know how many orders of 
magnitude. Yet this is in some respects the most important mode of 
scientific knowledge diffusion.

How this news based diffusion affects the dynamics of science is 
likewise unknown. Is anyone studying this formally? I am doing so 
informally. The web is providing some new approaches, such a blog 
tracking and the occurrence of embedded URLs. The spread of 
characteristic language is also a likely avenue. This is much more 
like true diffusion analysis than is citation and co-author network 
analysis, in that it goes beyond tracking large, discrete 
transactions to looking at a vague spreading cloud of information.

Steve also raises the issue of the spread of misinformation via 
diffusion of news. This has been studied in the context of general 
social thought, especially rumors. It is certainly significant in the 
realm of science and public policy, where the Evans case lies. I 
study this phenomenon in the climate change debate and in energy 
policy. Whether it is important in science per se I do not know. It 
is not even clear how one would approach it, but it seems like an 
important research topic. Perhaps it should be approached as the 
diffusion and dynamics of controversy or disagreement.



David Wojick, Ph.D.


Feb 25, 2009 06:40:34 PM, SIGMETRICS at wrote:

my <>critique of 
his <>Chronicle 
of Higher Education posting on Evans and Reimer's 
(2009) <>Science 
article (which I 
likewise <>critiqued, 
though much more mildly), I got an email from Paul Basken asking me 
to explain what, if anything he had got wrong, since his posting was 
based entirely on 
a <>press release 
from NSF. Sure enough, the silly spin originated from the NSF Press 
release (though the buck stops with E & R's vague and somewhat 
tendentious description and interpretation of some of their 
findings). Here is the NSF Press Release, enhanced with my comments, 
for your delectation and verdict:

If you offer something of value to people for free while someone else 
charges a hefty sum of money for the same type of product, one would 
logically assume that most people would choose the free option. 
According to new research in today's edition of the journal Science, 
if the product in question is access to scholarly papers and 
research, that logic might just be wrong. These findings provide new 
insight into the nature of scholarly discourse and the future of 
the open source publication movement[sic, emphasis added].

(1) If you offer something valuable for free, people will choose the 
free option unless they've already paid for the paid option 
(especially if they needed -- and could afford -- it earlier).

(2) Free access after an embargo of a year is not the same 
"something" as immediate free access. Its "value" for a potential 
user is lower. (That's one of the reasons institutions keep paying 
for subscription/license access to journals.)

(3) Hence it is not in the least surprising that immediate 
print-on-paper access + (paid) online access (IP + IO) generates more 
citations than immediate (paid) print-on-paper access (IP) alone.

(4) Nor is it surprising that immediate (paid) print-on-paper access 
+ online access + delayed free online access (IP +IO + DF) generates 
more citations than just immediate (paid) print-on-paper + online 
access (IO + IP) alone -- even if the free access is provided a year 
later than the paid access.

(5) Why on earth would anyone conclude that the fact that the 
increase in citations from IP to IP + IO is 12% and the increase in 
citations from IP + IO to IP + IO + DF is a further 8% implies 
anything whatsoever about people's preference for paid access over 
free access? Especially when the free access is not even immediate 
(IF) but delayed (DF)?

Most research is published in scientific journals and reviews, and 
subscriptions to these outlets have traditionally cost money--in some 
cases a great deal of money. Publishers must cover the costs of 
producing peer-reviewed publications and in most cases also try to 
turn a profit. To access these publications, other scholars and 
researchers must either be able to afford subscriptions or work at 
institutions that can provide access.

In recent years, as the Internet has helped lower the cost of 
publishing, more and more scientists have begun publishing their 
research in open source outlets online. Since these publications are 
free to anyone with an Internet connection, the belief has been that 
more interested readers will find them and potentially cite them. 
Earlier studies had postulated that being in an open source format 
could more than double the number of times a journal article is used 
by other researchers.

What on earth is an "open source outlet"? ("Open source" is a 
software matter.) Let's assume what's meant is "open access"; but 
then is this referring to (i) publishing in an open access journal, 
to (ii) publishing in a subscription journal but also self-archiving 
the published article to make it open access, or to (iii) 
self-archiving an unpublished paper?

What (many) <>previous 
studies had measured (not "postulated") was that (ii) publishing in a 
subscription journal (IP + IO) and also self-archiving the published 
article to make it Open Access (IP + IO + OA) could more than double 
the citations, compared to IP + IO alone.

To test this theory, James A. Evans, an assistant professor of 
sociology at the University of Chicago, and Jacob Reimer, a student 
of neurobiology also at the University of Chicago, analyzed millions 
of articles available online, including those from open source 
publications and those that required payment to access.

No, they did nothing of the sort; and no "theory" was tested.

Evans & Reimer (E & R) only analyzed articles from subscription 
access journals before and after they became accessible online (to 
paid subscribers only) (i.e., IP vs IP + IO) as well as before and 
after the online version was made accessible free for all (after a 
paid-access-only embargo of up to a year or more: i.e., IP +IO vs IP 
+ IO + DF). Their methodology was based on comparing citation counts 
for articles within the same journals before and after being made 
free online at various intervals.

The results were surprising. On average, when a given publication was 
made available online after being in print for a year, being 
published in an open source format increased the use of that article 
by about 8 percent. When articles are made available online in a 
commercial format a year after publication, however, usage increases 
by about 12 percent.

In other words, the citation count increase from just (paid) IP to 
(paid) IP + IO was 12% and the citation count increase from just 
(paid) IP + IO to (paid) IP + IO + DF was 8%. Not in the least 
surprising: Making paid-access articles accessible online increases 
their citations, and making them free online (even if only after a 
delay of a year) increases them still more.

What is surprising is the rather absurd spin that this press release 
appears to be trying to put on this unsurprising finding.

"Across the scientific community," Evans said in an interview, "it 
turns out that open access does have a positive impact on the 
attention that's given to the journal articles, but it's a small 

We already knew that OA increased citations, as 
the <>many prior 
published studies have shown.  Most of those studies, however, were 
based on immediate OA (i.e., IF), not embargoed OA. What E & R do 
show, interestingly, is that even delaying OA for a year still 
increases citations, though not nearly as much as immediate OA (IF).

Yet Evans and Reimer's research also points to one very positive 
impact of the open source movement that is sometimes overlooked in 
the debate about scholarly publications. Researchers in the 
developing world, where research funding and libraries are not as 
robust as they are in wealthier countries, were far more likely to 
read and cite open source articles.

A large portion of the citation increase from (delayed) OA turns out 
to come from Developing Countries 
(refuting <>Frandsen's 
recent report to the contrary). (A similar comparison, within the US, 
of citations from the Have-Not Universities (with the smaller journal 
subscription budgets) compared to the Harvards may well reveal the 
same effect closer to home, though probably at a smaller scale.)

The University of Chicago team concludes that outside the developed 
world, the open source movement "widens the global circle of those 
who can participate in science and benefit from it."

And it will be interesting to test for the same effect comparing the 
Harvards and the Have-Nots in the US -- but a more realistic estimate 
might come from looking at immediate OA (IF) rather than just 
embargoed OA (DF).

So while some scientists and scholars may chose to pay for scientific 
publications even when free publications are available, their 
colleagues in other parts of the world may find that going with open 
source works is the only choice they have.

It would be interesting to hear the authors of this NSF press release 
-- or E & R, for that matter -- explain how this paradoxical 
"preference" for paid access over free access was tested during the 
access embargo period...

<>Stevan Harnad
Scientist Open Access Forum


"David E. Wojick, Ph.D., PE" <WojickD at>
Senior Consultant for Innovation
Office of Scientific and Technical Information
US Department of Energy
391 Flickertail Lane, Star Tannery, VA 22654 USA provides my bio and 
past client list. 
presents some of my own research on information structure and 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: WallStreetJ.JPG
Type: image/jpeg
Size: 70066 bytes
Desc: not available
URL: <>

More information about the SIGMETRICS mailing list