CHE Impact Factor Article
Stephen J Bensman
notsjb at LSU.EDU
Mon Oct 10 14:02:48 EDT 2005
|-----------------+--------------------------------------------------------------------------------+-------------+-------------+------------->
| (Embedded image | |(Embedded |(Embedded |(Embedded |
| moved to file: | |image moved |image moved |image moved |
| pic13829.gif) | |to file: |to file: |to file: |
| | |pic14924.gif)|pic30188.gif)|pic21763.gif)|
| | | | | |
| |(Embedded image moved to file: pic05030.gif) | | | |
|Search The Site |(Embedded image moved to file: pic20326.gif)The Chronicle of Higher Education | | | |
| | | | | |
|More options | |(Embedded image moved to file: pic29011.gif) | | | |
|Back issues | | | | |
|Home |(Embedded image moved to file: pic30771.gif)Research | | | |
| | | | | |
| | | | | |
| | | | | |
|News | | | | |
| (Embedde| | | | |
| d image | From the issue dated October 14, 2005| | | |
| moved to| | | | |
| file: |The Number That's Devouring Science | | | |
| pic30900| | | | |
| .gif) | | | | |
| |The impact factor, once a simple way to rank scientific journals, has become an | | | |
|Today's news |unyielding yardstick for hiring, tenure, and grants | | | |
|Current issue | | | | |
|Special issues & | | | | |
|data (Embedded |Related materials | | | |
|image moved to | | | | |
|file: | | | | |
|pic17713.gif) |Text: How the impact factor is calculated | | | |
|Admissions & | | | | |
|Student Aid | | | | |
|Campus |Table: A list of the journals with the highest impact factors reveals one of the| | | |
|Architecture |measurement's quirks | | | |
|Campus Services | | | | |
|Endowments | | | | |
|Executive |Article: Impact Factors Run Into Competition | | | |
|Compensation | | | | |
|Legal Issues | | | | |
|Personal Finance |Colloquy: Join a live, online discussion with Anurag A. Agrawal, an assistant | | | |
|More ... |professor of ecology and evolutionary biology at Cornell University, about the | | | |
|The Faculty |ranking of scientific journals by their "impact factors" and the effect of that | | | |
|Research & Books |practice on research, hiring, and promotion, on Wednesday, October 12, at 1 | | | |
|Government & |p.m., U.S. Eastern time. | | | |
|Politics Money & | | | | |
|Management | | | | |
|Information | | | | |
|Technology | | | | |
|Students |(Embedded image moved to file: pic06411.gif) | | | |
|Athletics | Printer | | | |
|International | friendly | | | |
|Community | (Embedded image moved to file: pic21153.gif) | | | |
|Colleges Short | E-mail | | | |
|Subjects Gazette | article | | | |
|Corrections |(Embedded image moved to file: pic21520.gif) | | | |
| | Subscribe | | | |
| | (Embedded image moved to file: pic29790.gif) | | | |
| | Order | | | |
| | reprints | | | |
|Opinion & Forums | | | | |
| (Embedd| | | | |
| ed | | | | |
| image |By RICHARD MONASTERSKY | | | |
| moved | | | | |
| to | | | | |
| file: |In the beginning, during the late 1950s, it was just an innocent idea in Eugene | | | |
| pic1895|Garfield's head. A Philadelphia researcher who described himself as a | | | |
| 8.gif) |"documentation consultant," Mr. Garfield spent his free time thinking about | | | |
| |scientific literature and how to mine information from it. | | | |
|The Chronicle | | | | |
|Review Forums | | | | |
|Colloquy |He eventually dreamed up something he called an "impact factor," essentially a | | | |
| |grading system for journals, that could help him pick out the most important | | | |
| |publications from the ranks of lesser titles. To identify which journals | | | |
| |mattered most to scientists, he proposed tallying up the number of citations an | | | |
| |average article in each journal received. | | | |
|Careers | | | | |
| (Embedde| | | | |
| d image |This accounting method sounds harmless enough. Outside academe, few people have | | | |
| moved to|even heard of it. Mr. Garfield, though, now compares his brainchild to nuclear | | | |
| file: |energy: a force that can help society but can unleash mayhem when it is misused.| | | |
| pic17578| | | | |
| .gif) | | | | |
| |Indeed, impact factors have assumed so much power, especially in the past five | | | |
|News & Advice My |years, that they are starting to control the scientific enterprise. In Europe, | | | |
|Career homepage |Asia, and, increasingly, the United States, Mr. Garfield's tool can play a | | | |
|Search jobs by |crucial role in hiring, tenure decisions, and the awarding of grants. | | | |
|position type by | | | | |
|discipline/field | | | | |
|by state/region |"The impact factor may be a pox upon the land because of the abuse of that | | | |
|by institution |number," says Robert H. Austin, a professor of physics at Princeton University. | | | |
|Tools & Resources| | | | |
|Employer Profiles| | | | |
| |Impact-factor fever is spreading, threatening to skew the course of scientific | | | |
| |research, say critics. Investigators are now more likely to chase after | | | |
| |fashionable topics — the kind that get into high-impact journals — than to | | | |
| |follow important avenues that may not be the flavor of the year, says Yu-Li | | | |
| |Wang, a professor of physiology at the University of Massachusetts Medical | | | |
|Sponsored |School. "It influences a lot of people's research direction." | | | |
|information & | | | | |
|solutions | | | | |
| |That influence has also led to a creeping sense of cynicism about the business | | | |
| |of science publications. Journal editors have learned how to manipulate the | | | |
| |system, sometimes through legitimate editorial choices and other times through | | | |
| |deceptive practices that artificially inflate their own rankings. Several | | | |
| |ecology journals, for example, routinely ask authors to add citations to | | | |
|Chronicle/Gartner|previous articles from that same journal, a policy that pushes up its impact | | | |
|Leadership Forum |factor. Authors who have received such requests say that the practice veers | | | |
| |toward extortion and represents a violation of scientific ethics. | | | |
| | | | | |
| | | | | |
| |What's more, investigations into impact factors have revealed problems with the | | | |
|Services |basic data used by ISI, the company that tabulates citation statistics and | | | |
| (Embedde|journals' impact factors. Started by Mr. Garfield in Philadelphia, ISI was | | | |
| d image |bought in 1992 by the Thomson Corporation, which has tried to transform the | | | |
| moved to|citation enterprise into a more profitable operation by buying up databases and | | | |
| file: |promoting its products. With alarming frequency, editors are finding fault with | | | |
| pic08365|the impact factors that Thomson has issued. | | | |
| .gif) | | | | |
| | | | | |
|Help Contact us |"This was a serious concern," says Alan Nevill, a professor of biostatistics at | | | |
|Subscribe Manage |the University of Wolverhampton, in England, who took issue with the | | | |
|your account |calculations that ISI made regarding the Journal of Sports Science, which he | | | |
|Advertise with us|edits. "Academia is being held ransom by these citations." | | | |
|Rights & | | | | |
|permissions | | | | |
| |Far From Its Roots | | | |
| | | | | |
| (Embedded image | | | | |
| moved to file: |It wasn't supposed to be this way. "We never predicted that people would turn | | | |
| pic13007.gif) |this into an evaluation tool for giving out grants and funding," says Mr. | | | |
| |Garfield. | | | |
| | | | | |
| | | | | |
| |Although he first mentioned the term "impact factor" in a publication in 1955, | | | |
| |it wasn't until the 1960s that Mr. Garfield and a colleague fully developed the | | | |
| |concept to help them select the most important journals for a new citation | | | |
| |index, which has grown into one of the most widely used citation tools in | | | |
| |science and the social sciences. It didn't make sense, they reasoned, to include| | | |
| |only the journals that get the most citations, because that would eliminate | | | |
| |smaller publications. So they invented a type of measurement that reflects the | | | |
| |average number of citations per article for each journal. | | | |
| | | | | |
| | | | | |
| |The basic definition has changed little since then, although the process of | | | |
| |calculating impact factors has become highly automated through the use of | | | |
| |computer algorithms, which trolled through 27 million citations last year. In | | | |
| |June, ISI issued its latest set of impact factors, for 5,968 science journals | | | |
| |and 1,712 social-science journals. | | | |
| | | | | |
| | | | | |
| |To calculate the most recent factor for the journal Nature, for example, the | | | |
| |company tallied the number of citations in 2004 to all of the articles that | | | |
| |Nature published in 2002 and 2003. Those citations were divided by the number of| | | |
| |articles the journal published in those two years, yielding an impact factor of | | | |
| |32.182 — the ninth-highest of all journals. It is a number that editors and | | | |
| |publishers across the world lust after; more than half of all science journals | | | |
| |listed by ISI score below 1. | | | |
| | | | | |
| | | | | |
| |Impact factors caught on because they are an objective measure that serves many | | | |
| |purposes. Librarians can use them to decide which journals to purchase and which| | | |
| |to cancel. Editors and publishers can chart their journals' impact factors to | | | |
| |gauge their progress relative to competitors. And scientists can examine the | | | |
| |numbers to see where their research papers are likely to get the most attention.| | | |
| | | | | |
| | | | | |
| |Higher-ranking journals, it turns out, do get a message out better. Matthew B. | | | |
| |Stanbrook, an assistant professor of medicine at the University of Toronto, | | | |
| |tracked what happened after 12 medical journals published a joint statement on | | | |
| |research authorship and sponsorship in 2001 — an unusual situation that provided| | | |
| |direct comparisons. Over the following 26 months, the highest-impact journal | | | |
| |received 100 times as many citations to the article as the lowest one of the 12,| | | |
| |Dr. Stanbrook reported at a conference on peer review and publishing last month | | | |
| |in Chicago. "There's a measurable value associated with a high-impact journal, | | | |
| |which indicates why those journals are important," he says. | | | |
| | | | | |
| | | | | |
| |Over the years, impact factors have proved so attractive to scientists that they| | | |
| |started applying them not only to journals but also to researchers. Ideally, | | | |
| |evaluators would look at the number of citations an individual paper receives or| | | |
| |a scientist accumulates over his or her career — but that process takes time and| | | |
| |money. Impact factors provide a shortcut. | | | |
| | | | | |
| | | | | |
| |They also help in the modern world of ultraspecialized science. Members of a | | | |
| |tenure committee or a hiring panel find it increasingly difficult to assess the | | | |
| |papers of a candidate working outside their own subdiscipline, so they use the | | | |
| |impact factor of the journal in which the paper appeared as a measure of the | | | |
| |paper's quality. By that logic, evaluators rate a paper more highly if it | | | |
| |appears in a high-impact journal, regardless of what the paper actually says. | | | |
| | | | | |
| | | | | |
| |Europeans cite another reason that impact factors are popular there. In some | | | |
| |countries, the community of researchers in a particular field is so small that | | | |
| |they all know each other and either collaborate or compete. Using impact factors| | | |
| |to assess individual scientists is seen as an improvement over tapping into an | | | |
| |old-boy network to make hiring and grant decisions. | | | |
| | | | | |
| | | | | |
| |Fuzzy Math | | | |
| | | | | |
| | | | | |
| |But relying on impact factors to evaluate a person is statistically dimwitted, | | | |
| |say critics of its spreading influence. The measurement is just an average of | | | |
| |all the papers in a journal over a year; it doesn't apply to any single paper, | | | |
| |let alone to any author. For example, a quarter of the articles in Nature last | | | |
| |year drew 89 percent of the citations to that journal, so a vast majority of the| | | |
| |articles received far fewer than the average of 32 citations reflected in the | | | |
| |most recent impact factor. | | | |
| | | | | |
| | | | | |
| |Mr. Garfield and ISI routinely point out the problems of using impact factors | | | |
| |for individual papers or people. "That is something we have wrestled with quite | | | |
| |a bit here," says Jim Pringle, vice president for development at Thomson | | | |
| |Scientific, the division that oversees ISI. "It is a fallacy to think you can | | | |
| |say anything about the citation pattern of an article from the citation pattern | | | |
| |of a journal." | | | |
| | | | | |
| | | | | |
| |Such warnings have not helped. In several countries in Europe and Asia, | | | |
| |administrators openly use impact factors to evaluate researchers or allocate | | | |
| |money: | | | |
| | In England, hiring panels routinely consider impact factors, says Mr. | | | |
| | Nevill. | | | |
| | According to Spanish law, researchers are rewarded for publishing in | | | |
| | journals defined by ISI as prestigious, which in practice has meant in the| | | |
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic05030.gif
Type: image/gif
Size: 43 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic20326.gif
Type: image/gif
Size: 1218 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0001.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic29011.gif
Type: image/gif
Size: 43 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0002.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic30771.gif
Type: image/gif
Size: 3746 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0003.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic06411.gif
Type: image/gif
Size: 1391 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0004.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic21153.gif
Type: image/gif
Size: 1261 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0005.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic21520.gif
Type: image/gif
Size: 1342 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0006.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic29790.gif
Type: image/gif
Size: 1609 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0007.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic14924.gif
Type: image/gif
Size: 43 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0008.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic30188.gif
Type: image/gif
Size: 43 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0009.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic21763.gif
Type: image/gif
Size: 43 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0010.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic04940.gif
Type: image/gif
Size: 43 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0011.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic20851.jpg
Type: image/jpeg
Size: 16486 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic18662.gif
Type: image/gif
Size: 43 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0012.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic13829.gif
Type: image/gif
Size: 1710 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0013.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic30900.gif
Type: image/gif
Size: 572 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0014.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic17713.gif
Type: image/gif
Size: 75 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0015.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic18958.gif
Type: image/gif
Size: 568 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0016.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic17578.gif
Type: image/gif
Size: 568 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0017.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic08365.gif
Type: image/gif
Size: 572 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0018.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic13007.gif
Type: image/gif
Size: 1710 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0019.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic11477.gif
Type: image/gif
Size: 12018 bytes
Desc: not available
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20051010/9a5996d2/attachment-0020.gif>
More information about the SIGMETRICS
mailing list