FW: Citation Inflation - Science News

[Eugene Garfield]

   For readers outside the US,  Science News is a popular weekly digest
published by the Society  for Science and the Public. This digest
relates to a paper by Bryan Neff and Julian Olden in the June issue of
BioScience which may be familiar to some readers.. If you access the
link that follows you can also read another piece by the same author
(Raloff) on Citation Amnesia. EG 

 

 

http://www.sciencenews.org/view/generic/id/60283/title/Citation_inflatio
n


Citation Inflation - Science News


Many journals - and the authors who publish their novel data and
analyses in them - rely on "impact factors
<http://en.wikipedia.org/wiki/Impact_factor> " as a gauge of the
importance and prestige of their work. These ratios quantify how often a
journal's papers were cited by peer-reviewed
<http://www.lib.utexas.edu/lsl/help/modules/peer.html>  studies during
the subsequent two years. And there's been a presumption that the more a
journal's papers have been cited, the more important they must have
been. However, a new analysis turns up subtle ways that journals can
game the system to artificially inflate that impact factor.

Because the authors of the new study can't read the intent of a
journal's editors, they can't identify whether any particular
publication tinkered with its policies to deliberately foster that
inflation - or whether changes made for other purposes merely had that
ancillary effect. What they can say is that many journals' impact
factors no longer accurately reflect what this yardstick was designed to
measure. 

An impact factor of 2 would indicate that on average, each paper
published in the preceding two years had been cited twice in major
peer-reviewed journals. To investigate claims of impact inflation, Bryan
Neff <http://www.uwo.ca/biology/Faculty/neff/>  of the University of
Western Ontario <http://www.uwo.ca/> , in London, and Julian Olden
<http://www.fish.washington.edu/people/olden/>  of the University of
Washington <http://www.washington.edu/> , in Seattle, looked at all 70
peer-reviewed ecological journals that had consistently yielded an
impact factor of at least 1 over the preceding five years. They used the
Web of Science
<http://thomsonreuters.com/products_services/science/science_products/a-
z/web_of_science> , one of the three primary journal-indexing sources,
to download citation records for all papers published in the 70 journals
between 1998 and 2007.

After excluding 14 outlier articles for their excessive citations (i.e.
"falling more than seven standard deviations from the regression line"),
Neff and Olden focused their study on the remaining 72,298 papers. Over
the 10 year period, published works cited from 8 to 330 other papers,
with an average of 52. But this rate has not been static. On average,
they found, "papers have cited 0.67 additional papers each year."
Moreover, about 14 percent of citations, based on a random sampling of
100 publications, were to papers published in the preceding two years. 

So newer papers tend to cite more publications - and proportionately
more recent publications - than did the older journal articles, Neff and
Olden report in the June BioScience. Together, these factors contributed
most to impact inflation over the surveyed decade, they say.

The duo also looked at whether journals published review papers, which
tend to be more heavily cited down the road than do single-study papers.
"We found that there were strong positive relationships between the
proportion of review papers published and the impact factor of the
journal," they say, "as well as the change in impact factor over the
past five years."

This suggests that impact factors "should control for the proportion of
reviews published," Neff and Olden argue, or should compute them
separately for journals that publish reviews and those that do not. In
fact, they contend that too many papers are essentially lazy and cite
review papers rather than looking up and citing the original papers
mentioned in the review. Citing this primary literature instead would
"also help to alleviate the disproportionate effect reviews appear to
have on impact factors," they maintain.

Some editors may also encourage authors to cite recent papers that
appeared in their publication, which would help boost impact factors,
observes Alan Wilson of Auburn University <http://www.auburn.edu/>  in
Alabama. 

But one potentially important inflationary factor rests outside those
editors' hands: the growing proliferation of new journals. Impact
calculations sort of assume that the pool of journals from which
citations emerge will remain constant over time. In fact, that pool is
increasing. And as the pool of potential citers expands, so does the
likelihood that any given journal's papers will be mentioned. 

Sixty one of the 70 journals that Neff and Olden studied reported an
increase in impact factor over the 10 years studied. However, the
researchers found, only one-third of the total beat their calculated
inflation rate of 0.23 per year. Four percent matched the inflation rate
and 62 percent fell below it. Journals whose impact factors climbed
faster than the rate of inflation often were those with the highest
starting impact factor. It's like the rich getting richer fastest. 

Three years ago, Wilson investigated the inflation issue and found a
link between journal financing and impact factor. Nonprofit journals
tended to be oldest and have the lowest impact factors. For-profit
journals or those published by nonprofit organizations but via a
for-profit publisher had higher impact factors. 

Subscription rates of for-profit journals tend to cost substantially
more than for journals published by nonprofits. And because authors try
to publish in the highest-impact journal that will take their papers, a
pressure can develop to submit manuscripts to the more expensive
journals, he said. Moreover, libraries across the nation have had to cut
back on their journal budgets. Here too, impact factor may be developing
into an increasingly distorted metric for deciding how they can get the
biggest bang for their subscriptions bucks.

Neff and Olden cite other papers in recent years showing similar
indications of impact-factor inflation in other fields, although their
new rate for ecology journals seems to lead the pack. Clearly, the two
argue, there must be a more equitable way to measure publishing value
than this simple - and easy to manipulate - metric.

*         Email <about://view/send/id/60283/title/Citation_inflation> 

*         |

*         Print

*         |

*          


Found in: Ecology <about://view/interest/id/2526/topic/Ecology> ,
Education <about://view/interest/id/2533/topic/Education> , Environment
<about://view/interest/id/2337/topic/Environment>  and Science & Society
<about://view/interest/id/2347/topic/Science_%2B_Society> 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20100616/7b67dcd5/attachment.html>